EP4020458B1 - Procédé de génération par gabarit de variantes d'une chanson. - Google Patents

Procédé de génération par gabarit de variantes d'une chanson.

Info

Publication number
EP4020458B1
EP4020458B1 EP21212764.1A EP21212764A EP4020458B1 EP 4020458 B1 EP4020458 B1 EP 4020458B1 EP 21212764 A EP21212764 A EP 21212764A EP 4020458 B1 EP4020458 B1 EP 4020458B1
Authority
EP
European Patent Office
Prior art keywords
song
harmony
template
parameters
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP21212764.1A
Other languages
German (de)
English (en)
Other versions
EP4020458A1 (fr
Inventor
Dieter Rein
Jürgen Jaron
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Bellevue Investments GmbH and Co KGaA
Original Assignee
Bellevue Investments GmbH and Co KGaA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US17/134,708 external-priority patent/US11741922B2/en
Application filed by Bellevue Investments GmbH and Co KGaA filed Critical Bellevue Investments GmbH and Co KGaA
Publication of EP4020458A1 publication Critical patent/EP4020458A1/fr
Application granted granted Critical
Publication of EP4020458B1 publication Critical patent/EP4020458B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G10H1/0008—Associated control or indicating means
    • G10H1/0025—Automatic or semi-automatic music composition, e.g. producing random music, applying rules from music theory or modifying a musical piece
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/101—Music Composition or musical creation; Tools or processes therefor
    • G10H2210/145—Composing rules, e.g. harmonic or musical rules, for use in automatic composition; Rule generation algorithms therefor
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/101—Music Composition or musical creation; Tools or processes therefor
    • G10H2210/151—Music Composition or musical creation; Tools or processes therefor using templates, i.e. incomplete musical sections, as a basis for composing
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/571—Chords; Chord sequences
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/571—Chords; Chord sequences
    • G10H2210/576—Chord progression

Definitions

  • This disclosure relates generally to methods of editing and generating audio content and, in more particular, to methods utilizing a combination of collected and stored expert knowledge and machine learning in an artificial intelligence-based (“AI") selection engine for automatic audio song construction and for automatic variant generation of the audio song construction according to a combination of collected and stored expert knowledge and machine learning in an artificial intelligence-based (“AI”) selection engine.
  • AI artificial intelligence-based
  • US2020/090632 A1 shows a combination of expert knowledge and machine learning as AI selection engine for automatic song construction.
  • a system and method that allows a user to complete the song or music generation process and produce a result that is a musically pleasing complete song or music piece.
  • a system and method that provides the user with expert guidance in creating the work, where the guidance is based on collected and evolving expert knowledge and machine learning in an AI-based system for selection and positioning of suitable audio loops consistent with a user's end goal.
  • an approach that allows the user to automatically generate multiple variants of a generated music work utilizing expert knowledge and machine learning in an AI-based system.
  • FIG. 1 At least a portion of the instant invention will be implemented in form of software running on a user's computer 100 or other device with a CPU such as a table computer, smart phone, etc.
  • a CPU such as a table computer, smart phone, etc.
  • the word "computer” or CPU will be used generically to refer to any programmable device such as those listed in the previous sentence.
  • Such a computer will have some amount of program memory and storage (whether internal or accessible via a network) as is conventionally utilized by such units.
  • an external camera 110 of some sort be utilized with - and will be preferably be connectible to - the computer so that video and/or graphic information can be transferred to and from the computer ( Figure 1 ).
  • the camera 110 will be a digital video camera, although that is not a requirement, as it is contemplated that the user might wish to utilize still images from a digital still camera in the creation of his or her multimedia work.
  • the camera might be integrated into the computer or some other electronic device and, thus, might not be a traditional single-purposes video or still camera.
  • the camera will preferably be digital in nature, any sort of camera might be used, provided that the proper interfacing between it and the computer is utilized.
  • a microphone 130 might be utilized so that the user can add voice-over narration to a multimedia work and a CD or DVD burner 120 could be useful for storing in-progress or completed works. Further, it might also be possible and is shown in Figure 1 that the process of the instant invention might be implemented on portable tablet computer devices 140 or on mobile devices, such as smart phones 150.
  • FIG. 2 this figure illustrates the skeletal structure of a song or a music piece 200.
  • This structure functions as the starting point for the functionality of the instant invention.
  • a song or music piece generated by an embodiment of the software product will consist of a plurality of individual song parts which is illustrated by part 1 210 and part 2 220 in Figure 2 , where the denomination of part N 230 is used to show that a potential song or music piece might consist of an arbitrary number of parts.
  • Each part has a specific runtime at a given tempo, which might be selected and defined by the user, alternatively the run time might be strictly defined as being, for example, 4 or 8 measures or multiples thereof. Additionally, these parts might be further specified by, for example, designating them as being an intro or an ending.
  • Figure 2 also generally indicates that each part of a song or music piece preferably consists of an arbitrary number of instruments, wherein audio loops that sound these instruments are supplied accessible to the user and the AI engine discussed below, an audio loop being a digital section of sound material that usually may be seamlessly repeated, i.e., "looped". Further details with respect to the figure are presented below.
  • Figure 3 gives additional details of the process of constructing a skeleton.
  • the structure or skeleton of a song is depicted there.
  • a song is constructed of 8 individual sections, which might be an intro 345, an ending 350, all of the user supplied sections with their content 200 and 210, and, in this particular example, a mixture of variations of these supplied parts (355, 360 and 365).
  • parts might be added to the skeleton to lengthen the runtime of the work.
  • the skeleton basically includes an intro and an ending and in between the user parts plus variations of these parts and new parts, the variations and new parts preferably being chosen and defined and automatically added by the expert engine.
  • other song parts might be available including, for example, a song bridge, a song refrain / chorus, pre-chorus, etc.
  • the AI system uses the skeleton model as a starting point and the skeleton model has the song structure planned, i.e., the parts, style, instruments and so on.
  • the AI system analyzes each empty part and its desired content (as specified by the skeleton) and fills it with loops consistent with the requirements of those parts.
  • the AI system has access to all of the loops and information about all the loops in the loops database and their associated auditory properties.
  • the AI system may or may not select a loop from a style that is normally used to, for example, to generate a blues song if "Blues" is the selected style. But, if the AI system determines that the auditory properties of a different loop in the database are sonically compatible or even better suited then it may select that loop for insertion into the skeleton, independent from the style association of that loop.
  • the selected audio loop is played during the whole runtime of the part to which its parent instrument belongs, however it is also possible that the user may select and de-select or switch individual audio loops during the runtime of the particular part.
  • the instant invention provides and utilizes an evolving and growing database of audio loops, wherein the audio loops are categorized according to one or more particular styles, for example EDM, 50s, Drum'n Bass and House. Each individual style features a plurality of different instruments associated with it and each instrument has a specific number of associated audio loops, i.e., audio loops in which the instrument sounds when the loop is played (e.g., recorded). Of course, there might be one or multiple instruments recorded in a loop.
  • the loop might not contain traditional audio recordings of an acoustic instrument but might contain computer generated sounds that resemble (or not) traditional instruments. Either way, when it is said that an instrument is recorded in a loop that term should be broadly construed to cover instances where there is a digital audio recording of that instrument as well as cases where the audio material in the loop is computer generated.
  • This database will preferably be updated on a regular basis with new styles and the associated instruments and loops being added, existing styles with the associated instruments and loops being updated or deleted, etc. Preferably these updates will be delivered over the Internet for free or in exchange for a particular payment option.
  • this figure illustrates the interaction of the participating parts of the process to generate a complete music work utilizing expert knowledge and machine learning in an AI-based system in a compressed form.
  • the process will begin with a fragmented song or music piece, comprised of at least one, and preferably two, selected and defined song parts, usually a verse 300 and a chorus or refrain 310 of a music piece.
  • These song parts are generated and provided with audio content 320 by a user, the content preferably being audio loops selected by the user.
  • This song fragment might be the starting point for the process; alternatively, the starting point might be the selection of a music style, or a seed song from a song library.
  • the starting data (e.g., loops / variation /style/ volume or intensity or dynamic, etc.) is selected by the user and provided to the combination of the expert engine and the AI system 330.
  • This combination will preferably proceed to sequentially process the existing user-defined starting point and as a result provides a complete song 340 for review by the user.
  • the user might influence the song creation process by specifying a desired tempo or preferences regarding the dynamic progression of a song (for example - the beginning should be calm - in the middle it is aggressive and at the end energetic).
  • an expert engine will analyze the user defined or selected song parts to identify a preferred style. Based on the style a song length will be chosen by the expert engine that is predominantly associated with that style. That is, the expert engine will contain rules, guidelines, preferences, etc., provided by experts who are familiar with a particular style of musical works, collectively "rules" hereinafter. That being said, it should be understood that these guidelines might (or might not) be strict rules, but instead are more general preferences.
  • the rules that are accessible by the expert engine might include things such as the length of the song, preferential chord change patterns, preferable instruments, preferable tempos, preferable percussion patterns, etc. Thus, the user may, for example, override the suggested song length and choose an alternative length if that is desired. Given the identified or selected style, the following sorts of decisions can be made by the expert engine as part of the music work creation process:
  • the expert system is a conglomerate of information gathered from music experts who were asked to provide information about how a particular song in a selected style should be built within the confines of a loop-based music generation system. So, the expert system is a curated system that is constantly updated and therefore living i.e., constantly updated system which is incremented by new additions or changes to the contents of the expert system.
  • the AI system uses the skeleton model as a starting point and the skeleton model has the song structure planned, i.e., parts, style, instruments and so on. To fill in the gaps of the skeleton model the AI system analyzes each part and its desired content and fills it, even if the style of the loop is different from the requested style. So, the AI system is not limited to use the designated style in selecting loops - there are no style boundaries with the AI system in terms of loop selection. Even though the skeleton model is requesting specific audio loops of styles, the AI system potentially utilizes other information to select loops that might not be associated with the requested style, it might even be associated with a style that is not musically combined or used with the requested song style.
  • the AI system identifies a loop from a different style that is compatible and well suited for the requested part of the skeleton model, that loop might still be selected and inserted into the skeleton model.
  • an embodiment of the AI system When an embodiment of the AI system is requested to fill in a song skeleton, it will first look at the user-provided audio loops (provided by the user in the parts, e.g., parts 200 and 210 ). It will also determine with which instruments each loop is associated, i.e., which instruments each loop expresses. The AI will analyze the performance parameters of the initial existing loops and select loops for insertion for which the values (in view of all performance parameters) are within a defined threshold and belong to a particular instrument. It may be clear now that the type and preferences of the song skeleton does not necessarily control the loop selection process to any great extent. The loop definition and selection process are primarily based on the user-chosen audio loops and their associated data values - meaning, the type of instruments associated with the audio loops and therewith skeleton model also contributes to the loop selection process.
  • the completed song preferably consists of an intro section 345 and ending section 350, any pre-existing user generated song parts 210, 220 and a plurality of additional song parts that have been generated according to structural requirements defined by the expert engine and filled with appropriate content provided and proposed by the AI system, so, for example, the AI system may select different audio loops for the vocal instrument selected in original part 2 220 and the expert engine may determine it structurally inconsistent to have different audio loops in the vocals instrument and may delete the vocal instrument completely, thereby generating a variation 360 of original part 2.
  • the expert system may change the harmony of the part, the bpm (beats per minute) of the song part, create a harmonic progression for the selected part, change the intensity of effects over the running time of the selected part - all these options resulting in a variation of an original part.
  • the process of generating a complete music work utilizing expert knowledge and machine learning in an AI-based system additionally provides a global setting for the variance whose spectrum might be definable by the user and which ensures that for each iteration of the process different but still fitting audio loops are selected and inserted.
  • the AI system knows the auditory properties of these three loops and if the variance level is low the AI system in its selection of new loops for part 2 in the skeleton model only selects audio loops for the instruments whose auditory properties are very similar (tempo in part 1 instrument 1 loop 1 is 78 - so the loop for part 2 instrument 1 is selected to have a tempo ⁇ 2 (76 or 80) bpm). If the variance level is high this threshold area widens, e.g., it might widen with each step in the variance level ⁇ 4.
  • a variation in the loop selection process might be introduced by adding a random amount to each value that is used in the selection of a new loop. That is, in some embodiments a parameter that represents "calmness" might be provided for each loop. This parameter might represent, e.g., the amount of dynamic variation in that loop. Assume for purposes of discussion that the value of an initial existing loop is 0.60. If, for example, the randomization amount is set to 5%, when the AI system searches for loops to include in the skeleton, the search might be broadened to include loops with calmness values between 0.57 and 0.63 and, in some embodiments, values at the extremes might be preferentially selected, with the selection among multiple qualifying loops preferably being random in some embodiments.
  • the user might be provided with a graphical user interface that shows, e.g., the ten most prominent parameters of either each loop or the whole generated song.
  • This GUI could also provide a control that allows the user to prioritize each parameter, with the prioritization ranging from 100% to 0%, with 0% meaning that a parameter is never considered and 100% meaning the parameter is the most important, perhaps sole, parameter considered.
  • the user will be able to instruct the AI system, e.g., to select more loops with a female singing voice (control to 100%) and completely ignore loops with aggressive mood (control to 0%).
  • a setting of 50% for the parameter style could instruct the AI system to "mix" it up regarding the loop selection, i.e., the designated parameter is somewhat important, but the AI is permitted to stray from the base style determination set by the skeleton model.
  • FIG. 4 this figure is depicting an embodiment that illustrates the functionality of the machine learning AI system 410.
  • the system for machine-based learning is constantly being trained with the complete available database of styles and the therewith associated audio loops 400. Constantly meaning that, as has been disclosed, the loop and style availability changes over time, with new content being added to the database and content base or removed from it with the AI constantly evaluating these changes.
  • the AI has complete knowledge of every audio loop, its auditory properties, length, tempo, dynamic and key. Additionally, the AI does know which audio loops are available locally for the user and which have to be purchased online.
  • the AI Based on the song skeleton 440 generated by the expert engine 430 which utilizes any user provided existing song parts and their associated audio loops 420 and based on the information of the selected loops in the existing song parts 420 the AI selects suitable audio loops from the database for the expert engine generated song parts making up the song skeleton and automatically fills 460 the available data structures of the generated song skeleton 440 resulting in a completed song 470.
  • the AI provides the determined audio loops for selection by the user 450, therewith providing more experienced users the ability to take on a more active part in the song generation process by manually deciding which audio loops to insert 460 into the song skeleton 440.
  • the audio loop selection of the AI system is not bound by the initial style selection of the user, which results in a much bigger selection horizon compared to the manual selection of audio loops by the user.
  • the utilization of the AI disrupts the predefined "only audio loops from the selected style" selection form music generation in an innovative, efficient and time saving manner.
  • the selection of suitable audio loops is realized utilizing all the known data from the available initial song parts.
  • the existence of, in one embodiment, at least one song part that comprises of a length of 4 to 8 measures is sufficient for the functionality of the instant invention.
  • Each of the potential song parts has instruments (for example drums, bass, guitar, synthesizer, vocals) selected and for each instrument at least one audio loop is inserted and selected.
  • Each audio loop is stored in a database that additionally stores its individual auditory properties as well as its association to an instrument and style.
  • the AI system initially determines the instruments from the existing song parts and determines compatible and usable instruments in a next step - this is realized by an ad hoc judgment regarding the usability of different new audio loops from individual instruments with respect to the already inserted and selected instruments and audio loops.
  • This ad hoc judgment provides a determination above a specific threshold level which instrument and audio loop is acceptable in the specific song part of the generated song skeleton.
  • the different and new audio loops are selected, provided to the user, and ultimately inserted into the generated song skeleton, this being carried out either automatically or depending upon user approval.
  • FIG. 5 this figure is an illustration of one preferred workflow of an example useful for understanding the invention.
  • the user 500 initiates the song creation or generation process 525 from within a software program used for this purpose.
  • the user 500 begins the manual creation of a song or music piece by defining individual song parts (at least one) 530 and providing these song parts with style selection, instrument selection and audio loop selection.
  • the expert engine 510 begins the analysis of any already generated song parts 535 and as a result of that analysis generates a song skeleton 540, representing a full song consisting of a plurality of individual song parts.
  • the song skeleton generation process is initiated by the expert engine by determining the user selected style or styles from any previously song parts or any of the information provided by the user.
  • the expert engine makes the following sorts of choices - it should be understood that these choices do not need to be carried out sequentially, it might also be possible that the expert engine could be configured to process the mentioned points in a different order.
  • the expert engine determines a suitable length of the target song; this might depend, for example, on the initially selected style.
  • a 12-bar blues will typically have a verse that is 12 measures long, e.g., in the key of C the progression could be, by way of example only, C-C-C-C-F-F-C-C-G-F-C-C.
  • the expert engine determines the song scheme or song skeleton - for example, "Intro - Verse 1 - Verse 1 Variation - Refrain 1 - Verse 2 - Verse 2 Variation - Refrain 2 - Verse 3 - Verse 3 Variation - Refrain 3 - Ending". Then pluralities of different instruments are selected, where these instruments are selected to be compatible with the selected style.
  • the expert engine also determines harmony sequences that are typical for the selected initial style - for example for a blues style a chord change pattern of C-C-F- F-C-G-F-C (in the key of C) would be a fitting harmony or chord sequence, where the letters represent chords in one variation of a traditional 8-bar blues.
  • the 8-bar blues and the expert engine would be free to choose among them.
  • One way to do this might be to choose a skeleton randomly from among those available.
  • a familiar rock chord progression in the key of G
  • the dynamics structure for the song skeleton will be determined where, for example, the dynamics will be being defined as having a step wise increase from the beginning to the end of the song, e.g., the song would tend to get louder.
  • the expert engine also could specify the level of the variation for each to be created song part, for example a definition of how many of the audio loops in each song part could be exchanged and how much the audio loops will be allowed to differ from the audio loops selected by the user.
  • the data that is utilized as the basis for the expert engine is gathered from a data foundation collected from experienced song producers, wherein in a preferred embodiment for each individual style a specific data set is created and stored in a database (preferably xml-based).
  • the data will be constantly under review by these experts and will preferably be updated and varied at any time.
  • the machine-based learning AI 520 takes into account the data from the expert engine, the generated song skeleton 540 and any user created song parts 545 or any other information given by the user and begins with the selection of audio loops 550 and inserts these audio loops into the song skeleton 550 therewith creating the song 560.
  • the user might have the desire to generate a plurality of different variants of the created song, wherein these variants are generated with the resulting variants stemming exclusively from an approach utilizing provided harmony templates according to the instant invention.
  • the variant generation is started with a song that has been generated with either the disclosed mixture of the AI system and the expert knowledge or with a song that has been exclusively generated by the user - the common denominator being the fact that the initial song generated utilized expert knowledge and machine learning in an AI-based system is acceptable to the user and being utilized for the variant generation process.
  • a user might have the desire to generate variants of this particular song, because more (of the same) music is needed by the user to, for example, produce the music for a movie.
  • the instant invention discloses the use of harmony templates for the variant generation.
  • harmony templates are generated by a human producer and supplied to the user as part of the song creation system disclosed in paragraphs [0022] to [0047] or are provided to the user on a regular basis as soon as new harmony templates are generated.
  • the user is able to implement the provided harmony templates corresponding to the style of the song and enhance the existing song and therewith generate the desired number of song variants.
  • Each individual harmony template produces one song variant.
  • a loop affected by this harmony template has parameters that include its length, 16 beats, and its suggested use within a musical work, i.e., as part of the song chorus.
  • It's starting pitch is "2" which means, in this specific example of a layout, that the loop is generally compatible with the key of "A", although it is contemplated that an ending pitch might also be provided in the event that the starting and ending pitch of the loop are associated with different keys.
  • the "harmony template” provides a series of chords that would be compatible with the melody or general tone sequence of this loop.
  • one chord for each beat of the loop is provided.
  • the harmony template contains 8 beats of the "A" chord followed by 8 beats of the "D" chord.
  • the "A" chord will be associated with the first 8 beats of the loop and the "D" chord with the last 8 beats.
  • the eight beats might be expressed musically as a single occurrence of the chord which is sustained for 8 beats or it could be two or more soundings of the chord, e.g., one sounding of the chord every 4 th beat of the loop, etc.
  • the sequence of chords might be modified automatically if, for instance, the loop were to be transposed to a different key.
  • FIG. 6 this figure illustrates one workflow of the variant generation process of the instant invention.
  • the user who wants to generate song variants chooses an existing, previously generated song or music work 600 for which the instant invention shall generate the song variant.
  • the user initiates the song variant generation 610 by interacting with the graphical user interface of the software program.
  • the user will be able to select a plurality of different song variant generation approaches, wherein in this particular embodiment the variant generation is implemented via the selection and application of harmony templates.
  • the instant invention will determine the style parameter associated with the chosen song/music work and selects each harmony template associated with the same style parameter setting from that database.
  • the selected harmony templates are then provided to the user for further selection 620.
  • each harmony template will preferably be provided with a plurality of associated audio loops that are compatible with the harmony template. That will make it possible for the user to preview each harmony template so that the quality and appropriateness of each harmony template can be reviewed before the user makes a final selection.
  • the instant invention will extract the harmony template parameters and associated settings from the selected harmony template 630.
  • parameters that that might be associated with the loop include the chords, the length (in beats) and the defined chord sequence.
  • the chords parameter is a global parameter for the complete harmony template defining the chords that are used within the harmony template, wherein the length (in beats) and the chord sequence parameter are specific for each individual song part configured in the harmony template.
  • “global parameter” is meant that this parameter applies to all of the loops that are to be played in conjunction with the selected harmony template.
  • the instant invention will determine the parameters in the song that are compatible with the features of the harmony template. That is, the chords used in the song will be determined, as well as the individual song parts and the associated parameter values for these song parts, e.g., the length (in beats) and chord sequence.
  • the instant invention will apply the determined harmony template parameter values to the song 650 replacing the corresponding values stored within the song.
  • both the harmony template as well as the completed song have the same structural layout and parameter, wherein when applying the harmony template parameter values the corresponding parameter values of the song will be adapted accordingly. That is, according to this embodiment, the existing chords of the song/music work will be set to the values in the harmony template, with the idea being that these values will represent global information regarding the chords' variability within the template as well as that of the song/music work. Also relevant for the variant generation is the application of the length (in beats) and chord sequence values from the harmony template to the music work which is implemented the same way as has been disclosed with the chord's parameter value.
  • the instant invention in some embodiments will go through the individual parameter values for each part of the harmony template and will apply the parameter values to the corresponding parts of the existing song/music work. That is, the length and the chord sequence of each corresponding part of the existing song/music work will be modified accordingly. With this modification, the instant invention will ultimately generate the song variant 660 according to the selected harmony template.
  • FIG. 7 this figure illustrates in an example useful for understanding the invention_a flowchart of the harmony template based variant generation as part of the song generation process.
  • the starting point for this flowchart is the selection of an existing song/music work 700 by the user. Initially in this part of the song generation process the user is working with an already existing generated song/music work and is provided with the option to transform the existing song/music work 710. If the user decides to initiate a transform process of the existing song/music work a new transformed output song 720 will be generated from the existing song/music work, wherein in this embodiment the audio loops of the existing song/music work will be replaced with different audio loops selected according to a specific selection algorithm.
  • the user is provided with the option of selecting a harmony variant 730. If the user rejects the option for the generation of a harmony variant the instant invention stores the existing song/music work 750. If the user decides to accept the generation of a harmony variant the user is provided with a selection option of all available harmony templates 740 for the style of the existing song/music work. In a next preferred step, the user selects one of the available and provided harmony templates to generate a new output song 760 after application of the harmony template.
  • the user is also given the option of initiating the harmony variant 730 generation to produce a transformed output song 720. If the user does not accept the option, the process will store the transformed output song/music work as the output song/music work 760. If the user decides to initiate the generation of a new variation using the harmony template 730, after determining the quality of the transformed output song/music work, the process will provide the available harmony templates 740 for the style of the song to the user for selection. In a next preferred step, the user selects one of the available and provided harmony templates to generate a new output song 760 after application of the harmony template.
  • FIG. 8 this figure is an illustration depicting the structural setup of the harmony templates of this embodiment of the instant invention.
  • the specific structure of the harmony template 740 contains a plurality of different parameters and associated values and different levels wherein only a limited number of these parameter are relevant for a preferred embodiment of the instant invention - the other parameters are provided for providing a preview functionality to the user.
  • each harmony template will typically contain the following parameters and associated parameter values, the bpm value 805, volume 810, cut mode 815, chords 820 and parts 825.
  • the bpm value 805 represents the assigned beats per minutes value of the template
  • the volume parameter 810 represents the volume of the template
  • the cut mode 815 defines the specific mode or criteria according to which the audio loops in the harmony template are cut.
  • the chords 820 represents a listing of the available chords that are a part of the template and that are after selection of a specific harmony template applied to the existing song/music work.
  • the database harmony template structure contains certain preferred fields 825, which contain parameters related to the template.
  • the parameters include the chord sequence 830, the name 835, the length of the template in beats 840, the pitch 845 and the instrument 850.
  • Each instrument 850 is associated with an audio loop 855, with each loop featuring an associated name 860, a volume value 865 and an active 870 setting.
  • the harmony template could be viewed as a short song/music work, wherein, as has been disclosed, most of the content of a harmony template, is directed to provide the user a preview functionality prior to selection and application of the template to the existing song/music work.
  • Particularly relevant for the application of the template in most cases are the chords 820 parameter, the parts 825 and the beats length 840 parameter of each part and the chord sequence 830 for each part.
  • the other parameters e.g., the bpm value 805, volume 810, cut mode 815, name 835 of the part, pitch 845 of the part, instrument 850 and loop 855 parameter including the name 860, volume 865 and active 870 are all directed to providing the user the preview functionality. In some cases, the parameters will not be knowable or alterable by the user.
  • the bpm value 805 represents the value according to well-known bpm value determination algorithms, the volume value 810 can take a plurality of different ranges, however in a preferred embodiment it ranges from the values of 0 (very low) to 100 (very high).
  • the cut mode 815 defines the specific mode or criteria according to which the audio loops in the harmony template are cut, wherein possible options are according beat and/or bar.
  • the pitch parameter of the parts 845 represents the pitch level of each specific part and is preferably ranged according to an internal listing numerically from 0 to 10 for example.
  • the instrument 850 parameter and the associated loop parameter 855 represent the specific sections (instrument) and the content (loop) parameter of the harmony template.
  • Each audio loop 855 of each instrument 850 of each part 825 has an associated name 860, volume 865 and an active parameter 870, wherein the active parameter features an on/off setting, which determines if that loop is active or not.
  • FIG. 9 this figure depicts a flowchart of the application process of the harmony template to the initial song/music work.
  • the harmony template 740 is selected by the user for application and the instant invention selects the necessary harmony template features 910, respectively the chords parameter 820, the beats length parameter 840 and the chord sequence parameter 830 and applies the values of the features to the existing song/music work 920 to therewith after application generate the output song/music work 930.
  • the existing selected song/music work has, as has been disclosed, the same or a similar structure as the harmony template which allows the instant invention to apply the data values of the named harmony template features to the corresponding features of the existing song/music work.
  • the instant invention adapts the chords parameter value of the existing song to the parameter value of the harmony template.
  • the beats length and the chord sequence parameter are, as has been disclosed, parameters and associated values that are connected to each part that makes up the template.
  • the method parses through each part of the harmony template sequentially, selects the associated values for beats length 840 and chord sequence 830 and applies these values to the same parts of the existing song 920 resulting in the output song/music work 930, wherein applying these values comprises of selecting audio loops from the audio loop database that are featuring the required chords for the chord sequence 830. It should be noted that each audio loop in the database is stored in at least eight chord variants.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Auxiliary Devices For Music (AREA)

Claims (4)

  1. Procédé de génération de variantes d'œuvres musicales dans un ordinateur numérique programmé pour exécuter des étapes de programme comprenant :
    a. l'accès à une œuvre musicale numérique existante (600), ladite œuvre musicale numérique ayant une pluralité de paramètres musicaux qui lui sont associés ;
    b. la détermination du paramètre de style associé à l'œuvre musicale numérique existante à partir de ladite pluralité de paramètres musicaux ;
    c. fournir une base de données numérique de boucles audio, dans laquelle chaque boucle audio est stockée dans au moins huit variantes d'accords, la base de données numérique contenant en outre une pluralité de modèles d'harmonie, dans lesquels chaque modèle d'harmonie fournit une série d'accords compatibles avec la mélodie ou la séquence de notes de chaque boucle audio et dans lesquels chaque modèle d'harmonie contient une pluralité de paramètres différents, comprenant des paramètres de génération de variantes et des paramètres de prévisualisation de modèles ;
    d. en fonction dudit paramètre de style déterminé associé à ladite œuvre musicale numérique, sélectionner chaque modèle d'harmonie (620) associé au même paramètre de style ;
    e. recevoir d'un utilisateur (620) une sélection d'un modèle d'harmonie parmi ladite pluralité de modèles d'harmonie sélectionnés dans ladite base de données, ledit modèle d'harmonie sélectionné ayant une pluralité de paramètres de modèle d'harmonie qui lui sont associés ;
    f. identifier les valeurs des paramètres du modèle d'harmonie associés (630) à chacun desdits paramètres du modèle d'harmonie associés ;
    g. déterminer les paramètres de la chanson (640) à partir de ladite œuvre musicale numérique qui sont compatibles avec les mêmes paramètres du modèle d'harmonie ; et
    h. appliquer automatiquement les valeurs des paramètres du modèle d'harmonie associées auxdits paramètres du modèle d'harmonie identifiés auxdits paramètres correspondants de l'œuvre musicale sélectionnée parmi ladite œuvre musicale numérique sélectionnée (650), remplaçant ainsi les valeurs des paramètres de l'œuvre musicale par les valeurs des paramètres du modèle d'harmonie, générant ainsi une variante de l'œuvre musicale numérique.
  2. Procédé selon la revendication 1, dans lequel lesdits paramètres de génération de variantes comprennent la classification des parties, la sélection des accords, la longueur des mesures et la séquence des accords, les paramètres de longueur des mesures et de séquence des accords étant spécifiques à chaque partie de chanson configurée dans le modèle d'harmonie.
  3. Procédé selon la revendication 2, dans lequel ledit paramètre de sélection d'accords est un paramètre global et dans lequel la longueur des battements et la séquence d'accords sont des paramètres associés à la classification des parties.
  4. Procédé selon la revendication 3, dans lequel lesdits paramètres de prévisualisation du modèle comprennent la valeur bpm, le volume, le mode de coupe, la hauteur et l'instrument.
EP21212764.1A 2020-12-28 2021-12-07 Procédé de génération par gabarit de variantes d'une chanson. Active EP4020458B1 (fr)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US17/134,708 US11741922B2 (en) 2018-09-14 2020-12-28 Method and system for template based variant generation of hybrid AI generated song

Publications (2)

Publication Number Publication Date
EP4020458A1 EP4020458A1 (fr) 2022-06-29
EP4020458B1 true EP4020458B1 (fr) 2026-02-11

Family

ID=78822599

Family Applications (1)

Application Number Title Priority Date Filing Date
EP21212764.1A Active EP4020458B1 (fr) 2020-12-28 2021-12-07 Procédé de génération par gabarit de variantes d'une chanson.

Country Status (1)

Country Link
EP (1) EP4020458B1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4411721A1 (fr) * 2023-02-03 2024-08-07 Soundtrap AB Génération de séquence d'accords en boucle

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070261535A1 (en) * 2006-05-01 2007-11-15 Microsoft Corporation Metadata-based song creation and editing
EP2495720A1 (fr) * 2011-03-02 2012-09-05 YAMAHA Corporation Génération de sons par combinaison de matériaux sonores
US20120312145A1 (en) * 2011-06-09 2012-12-13 Ujam Inc. Music composition automation including song structure
US20170092248A1 (en) * 2015-09-30 2017-03-30 Apple Inc. Automatic composer

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11972746B2 (en) * 2018-09-14 2024-04-30 Bellevue Investments Gmbh & Co. Kgaa Method and system for hybrid AI-based song construction
GB2581319B (en) * 2018-12-12 2022-05-25 Bytedance Inc Automated music production

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070261535A1 (en) * 2006-05-01 2007-11-15 Microsoft Corporation Metadata-based song creation and editing
EP2495720A1 (fr) * 2011-03-02 2012-09-05 YAMAHA Corporation Génération de sons par combinaison de matériaux sonores
US20120312145A1 (en) * 2011-06-09 2012-12-13 Ujam Inc. Music composition automation including song structure
US20170092248A1 (en) * 2015-09-30 2017-03-30 Apple Inc. Automatic composer

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"The Real Book passage", 1 June 2005, THE REAL BOOK, HAL-LEONARD CORPORATION, US, PAGE(S) 1 - 5, ISBN: 978-0-634-06021-2, XP007915306 *

Also Published As

Publication number Publication date
EP4020458A1 (fr) 2022-06-29

Similar Documents

Publication Publication Date Title
EP3792909B1 (fr) Procédé et système de construction de chanson hybride basée sur l'ia
US12327539B2 (en) Method and system for energy-based song variant construction with demixing
CN111512359B (zh) 模块化自动音乐制作服务器
CN108369799B (zh) 采用基于语言学和/或基于图形图标的音乐体验描述符的自动音乐合成和生成的机器、系统和过程
US7792782B2 (en) Internet music composition application with pattern-combination method
US10496250B2 (en) System and method for implementing an intelligent automatic music jam session
US11741922B2 (en) Method and system for template based variant generation of hybrid AI generated song
WO2020000751A1 (fr) Procédé et appareil de composition automatique, et dispositif informatique et support d'informations
US10325581B2 (en) Singing voice edit assistant method and singing voice edit assistant device
CN117765902A (zh) 乐曲伴奏的生成方法、装置、设备、存储介质及程序产品
Zhang Utilizing computational music analysis and AI for enhanced music composition: exploring pre-and post-analysis
CN114974184A (zh) 音频制作方法、装置、终端设备及可读存储介质
US11615138B2 (en) Method and system for hybrid AI-based song variant construction
EP4020458B1 (fr) Procédé de génération par gabarit de variantes d'une chanson.
EP4020256A1 (fr) Procédé et système de construction de variante de chanson hybride basée sur l'ia
JP6611633B2 (ja) カラオケシステム用サーバ
EP4024392A1 (fr) Procédé et système de construction de chanson basés sur l'énergie
CN1770258B (zh) 表演风格确定设备和方法
EP4418258A1 (fr) Procédé et système de génération de variant de chanson basé sur l'énergie
EP2793222B1 (fr) Procédé de mise en oeuvre d'une session automatique de musique jam.
DK202170064A1 (en) An interactive real-time music system and a computer-implemented interactive real-time music rendering method
EP4614492A1 (fr) Système musical génératif utilisant des algorithmes à base de règles et des modèles ai
US20240304167A1 (en) Generative music system using rule-based algorithms and ai models
EP4462420A1 (fr) Système et procédé de création de musique basée sur l'ia générative
US20230188108A1 (en) System and method for increasing energy level of songs

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20221213

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20230417

R17C First examination report despatched (corrected)

Effective date: 20230426

RIC1 Information provided on ipc code assigned before grant

Ipc: G11B 27/031 20060101ALN20250303BHEP

Ipc: G10H 1/00 20060101AFI20250303BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RIC1 Information provided on ipc code assigned before grant

Ipc: G10H 1/00 20060101AFI20250919BHEP

Ipc: G11B 27/031 20060101ALN20250919BHEP

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: G10H 1/00 20060101AFI20250919BHEP

Ipc: G11B 27/031 20060101ALN20250919BHEP

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

RIC1 Information provided on ipc code assigned before grant

Ipc: G10H 1/00 20060101AFI20251007BHEP

Ipc: G11B 27/031 20060101ALN20251007BHEP

INTG Intention to grant announced

Effective date: 20251022

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: CH

Ref legal event code: F10

Free format text: ST27 STATUS EVENT CODE: U-0-0-F10-F00 (AS PROVIDED BY THE NATIONAL OFFICE)

Effective date: 20260211

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602021047642

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20260122

Year of fee payment: 6