EP3012832B1 - Verfahren und System für die synthetische Modellierung eines Klangsignals - Google Patents

Verfahren und System für die synthetische Modellierung eines Klangsignals Download PDF

Info

Publication number
EP3012832B1
EP3012832B1 EP14189713.2A EP14189713A EP3012832B1 EP 3012832 B1 EP3012832 B1 EP 3012832B1 EP 14189713 A EP14189713 A EP 14189713A EP 3012832 B1 EP3012832 B1 EP 3012832B1
Authority
EP
European Patent Office
Prior art keywords
model
differential equation
function
input sound
sound sample
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Not-in-force
Application number
EP14189713.2A
Other languages
English (en)
French (fr)
Other versions
EP3012832A1 (de
Inventor
Markus Abel
Andre Bergner
Nikolaos Stefanakis
Karsten Ahnert
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ambrosys GmbH
Universitaet Postdam
Original Assignee
Ambrosys GmbH
Universitaet Postdam
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ambrosys GmbH, Universitaet Postdam filed Critical Ambrosys GmbH
Priority to EP14189713.2A priority Critical patent/EP3012832B1/de
Publication of EP3012832A1 publication Critical patent/EP3012832A1/de
Application granted granted Critical
Publication of EP3012832B1 publication Critical patent/EP3012832B1/de
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H5/00—Instruments in which the tones are generated by means of electronic generators
    • G10H5/007—Real-time simulation of G10B, G10C, G10D-type instruments using recursive or non-linear techniques, e.g. waveguide networks, recursive algorithms
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
    • G10H2210/061—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for extraction of musical phrases, isolation of musically relevant segments, e.g. musical thumbnail generation, or for temporal structure analysis of a musical piece, e.g. determination of the movement sequence of a musical work
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2250/00—Aspects of algorithms or signal processing methods without intrinsic musical character, yet specifically adapted for or used in electrophonic musical processing
    • G10H2250/315—Sound category-dependent sound synthesis processes [Gensound] for musical use; Sound category-specific synthesis-controlling parameters or control means therefor
    • G10H2250/435—Gensound percussion, i.e. generating or synthesising the sound of a percussion instrument; Control of specific aspects of percussion sounds, e.g. harmonics, under the influence of hitting force, hitting position, settings or striking instruments such as mallet, drumstick, brush or hand
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2250/00—Aspects of algorithms or signal processing methods without intrinsic musical character, yet specifically adapted for or used in electrophonic musical processing
    • G10H2250/471—General musical sound synthesis principles, i.e. sound category-independent synthesis methods
    • G10H2250/511—Physical modelling or real-time simulation of the acoustomechanical behaviour of acoustic musical instruments using, e.g. waveguides or looped delay lines

Definitions

  • the invention relates to a method and a system for synthetic modeling of a sound signal.
  • Document US 4,736,663 A discloses a digital system for synthesizing individual voices of musical instruments.
  • the system comprises means for solving a system of simultaneous finite difference equations, where time is represented by real time in the computations.
  • Musical sounds of the voice can be produced by repetitiously solving the difference equations that model the instrument in real time, using an array of elemental means named "universal processing elements”.
  • Document WO 2009/134166 A1 discloses a musical instrument synthesizer which contains a MIDI sequence input unit, a MIDI analysis unit, and a unit for generating string and hammer parameters which are connected to a filter and a sound reproducer.
  • the computer program product can be stored on a non-transitory storage medium.
  • the differential equation can be represented by one or more functions.
  • the function(s) can also be called basis function(s) and can be provided from a pool of functions.
  • Each function can depend on one or more variables and / or one or more parameters.
  • the optimality condition may be determined by a least square method using Newton's method, for example.
  • the input sound signal can be any sound signal, for example a drum sound, a horn sound (e.g. from a brass instrument), a clarinet sound and a cymbal sound.
  • the model of the differential equation can be a stochastic model and the at least one function can be a non-linear function.
  • This allows improved modeling of non-linear sound signals, wherein the frequency of the signal depends on the amplitude.
  • a damping of the sound can be modeled efficiently.
  • Using non-linear functions is appealing to the nature of speech as well as to that of many physical instruments which exhibit nonlinear behavior and therefore, their resynthesis has not been solved to a satisfactory manner by linear methods.
  • the use of a nonlinear model in conjunction to percussive instruments would appear justified, since it is generally accepted that there are several nonlinear mechanisms involved in the physics of membranophones and cymbals. These include the dependence of frequency to amplitude, irregularities in the decay profile of distinct modes of vibration and nonlinear coupling between different modes.
  • the at least one parameter of the at least one function can be changed dynamically according to a predefined function or to a user input.
  • the method may further comprise decomposing the input sound sample into several components.
  • the method may further comprise determining a number of dimensions of the model of the differential equation.
  • the time derivative of the input sound sample may be a first order or a higher order (second order, third order, ...) time derivative.
  • the method can comprise a step of transforming a real valued component of the input sound sample into a complex value.
  • the at least one parameter of the at least one function can be a constant.
  • the at least one function can be a time-dependent function.
  • An envelope of the input sound signal can be modeled by a dynamic equation, wherein the dynamic equation comprises at least one further differential equation.
  • the envelope By modeling the envelope, parts of the sound signal varying fast in time can be captured.
  • the method may also comprise a step of outputting a sound signal which is generated by evolving the model of the differential equation in time.
  • the differential equation may be integrated forward in time.
  • the disclosure refers to the usage of a computing device.
  • the computing device may comprise one or more processors configured to execute instructions.
  • the computing device may comprise a memory in form of volatile memory (e.g. RAM - random access memory) and / or non-volatile memory (e.g. a magnetic hard disk, a flash memory).
  • the device may further comprise means for connecting and / or communicating with other (computing) devices, for example by a wired connection (e.g. LAN - local area network, Firewire (IEEE 1394) and / or USB - universal serial bus) or by a wireless connection (e.g. WLAN - wireless local area network, Bluetooth and / or WiMAX - Worldwide Interoperability for Microwave Access).
  • a wired connection e.g. LAN - local area network, Firewire (IEEE 1394) and / or USB - universal serial bus
  • a wireless connection e.g. WLAN - wireless local area network, Bluetooth and / or WiMAX - Worldwide
  • the computing device may comprise a device for registering user input, for example a keyboard, a mouse and / or a touch pad.
  • the device may comprise a display device or may be connected to a display device.
  • the display device may be a touch-sensitive display device (e.g. a touch screen).
  • ODE ordinary differential equations
  • a model for f( ⁇ ) has to be chosen, then involved parameters can be evaluated by using L2 approximations, i.e., least-squares methods. Assuming that the standard function f( ⁇ ) has been properly defined, then, given an appropriate initial condition, forward integration of the standard system can be used for re-synthesis of the original sound.
  • drum sounds do not exhibit harmonic structure and this rises the number of dimensions M which are required for the proper modeling of the dynamics.
  • M the number of dimensions required for the proper modeling of the dynamics.
  • simple filtering can be used in order to isolate parts of the signal spectrum that corresponds to distinct modes.
  • decompositions in terms of wavelets or other basis functions can be used in order to decompose the signal.
  • FIG. 1(a) and (b) illustrates the spectral magnitude and the envelope decay for this first mode for each one of the three realizations.
  • a first thing to observe here is the effect of tension modulation which becomes evident by the broadening of the spectral peak in the recording corresponding highest impact level in sub figure (a).
  • a non-constant decaying behavior becomes apparent at the initial part of the signal in Fig. 1(b) .
  • the mode seems to decay at a constant rate only after some significant amount of time has passed, which is probably a consequence of the average membrane tension relaxing to its nominal value.
  • the mode appears to decay fairly constantly for the two realizations corresponding to the two lowest impact strengths. Additional differences within these three recordings concern the attack time (symbolized by t p in Fig. 1(c) ). In particular, we observe a slight decrease of the time of maximum amplitude as we go from the lowest impact level to the highest one. These observations dictate that the same modal component is not only quantitatively but also qualitatively different within realizations corresponding to different impact levels.
  • a synthesis model should fulfill: first, the model used for synthesis should account for the various types of nonlinearities that characterize the mode vibration, second, it should be able to reproduce the qualitative diversity that characterizes the physical system at different impact levels.
  • a straightforward question then arises; is it possible to construct and use a single model which will be valid for the entire dynamic range of consideration, or should one use a different model for different dynamic levels? While the variance of the impact level is probably the most important expressive attribute in drum performance, we believe that this question is far more than trivial. In what follows these challenges are addressed from the perspective of a simple nonlinear dynamical model.
  • drum sounds do not exhibit harmonic structure and this raises the embedding dimension which is required for the proper modeling of the dynamics.
  • a first order complex ODE is an excellent dynamic system for capturing and resynthesizing the characteristics of distinct modal components. The latter may be isolated from the recorded signal by simple low-pass or band-pass filtering.
  • a low dimensional dynamical model can be used as the means for synthesizing decaying oscillations.
  • the basic form of the dynamical model is a non-autonomous first order complex ODE.
  • the function is defined explicitly below, together with the reconstruction method which shows a way to systematically build it from data.
  • drum modes do not decay monotonically; they undergo an attack region where the amplitude builds-up, reaching at a maximum level before it starts to decay.
  • the exact characteristics of the attack region are expected to be depended on the type of the excitator that is used (e.q. mallet or hand) and the correct synthesis of the attack part of the signal is known to be very crucial for achieving a naturally sounding result.
  • the parameters in ⁇ must be optimized so that the generated trajectory is quantitatively similar to an observation of the recorded modal component. Observe that the presented model is a linear function of all the parameters in ⁇ expect from the smoothing constants ⁇ 1 ,..., ⁇ M . In order to simplify things a bit, we admit in the following the use of fixed values for the smoothing constants in e . This way the parameter set is reduced and also the problem is brought into a form which is appropriate for applying linear reconstruction.
  • An embedding in particular a differential embedding, is a quite abstract construct in order to describe a map between two spaces, in our case, differential manifolds.
  • the manifold is the object under consideration, here; it corresponds to the phase space occupied by the trajectory of our dynamical system - the percussive instrument. So, we are searching a model in a certain underlying space, given the data, i.e. the sound signal recorded from an instrument. This model lives on a differentiable manifold which in turn might be embedded in a function space C 1 n (the space of vector of functions, which are once differentiable, in correspondence to Eq. 2.1).
  • the real-valued discrete signal must be converted into a complex-valued signal.
  • This so called analytic representation of the discrete signal can be derived in the frequency domain, by setting the negative frequency indexes to zero and then performing the inverse Fourier transform.
  • the complex-valued signal must be filtered in order to isolate the modal components of interest. For this purpose, low-pass or band-pass filtering may be applied as an attempt for separating the modal component of interest from the remaining part of the signal.
  • An additional crucial step before reconstruction is the calculation of the derivative of the input signal.
  • the choice of the method for the calculation of the derivative is of no trivial concern, especially when the signal is contaminated by noise, in which case more sophisticated techniques are required.
  • the signals to be differentiated are expected to be relatively smooth because all high frequency content is filtered out. Therefore, simple numerical methods of differentiation are expected to work as well.
  • a last step considers the association of an impact level parameter with each observation.
  • VCM Varying Coefficient Model
  • b b R + jb I with b R ⁇ 0 is responsible for the control of the attack time.
  • Figure 2 illustrates the synthesis result for a central frequency of 150 Hz, a decay rate equal to -15, a velocity value equal to 1 and attack time equal to 0.025 sec.
  • the above dynamical system is thus absolutely predictable and sets the basis for adding further functionalities in the ODE as the means for enriching the sonic attributes of the output signal (y(t)).
  • a very interesting type of non-linearity can be created by using the envelope of the signal,
  • y ⁇ 0 + j 2 ⁇ f 0 y + b y t + ⁇ + c
  • a non-zero value of c R requires the recalculation of the depended parameters b R and r p * so that the target attack-time and maximum amplitude will hold.
  • the two non-linearities are transformed into terms of the form d cos( ⁇ ) r and e sin( ⁇ ) in the expression of the amplitude and the frequency respectively.
  • they enforce a kind of amplitude and frequency modulation into the dynamical system, with a modulator frequency which varies dynamically according to ⁇ ( t ).
  • This is an extremely useful mechanism for synthesis purposes as it can lead to a straightforward increase of the bandwidth, or it can be used to create traditional synthesis effects such as tremolo and vibrato.
  • the underlying mechanisms presented here reveal a significant potential in enlarging the bandwidth of the synthesized sound in connection to external input or to coupling with other ODEs.
  • classical deterministic oscillators such as sines, square-waveforms or saw-tooth waveforms with user defined frequency characteristics can be directly inserted into the ODEs in order to modulate its frequency or amplitude.
  • the output signal from a second ODE can be used in order to modulate the amplitude or the frequency of the first ODE.
  • FM synthesis Frequency Modulation synthesis
  • the concept can be of course generalized to a case of N ODEs where the user is free to determine parameters involving their linear or non-linear coupling characteristics. This idea is further illustrated in the following.
  • the presented approach reveals a connection to additive synthesis; each equation in the system of ODEs may be associated to a different mode of vibration and a synthesis result can be obtained as a superposition of modes.
  • ( ⁇ ) denotes the Hadamard product. All bold capital letters are N ⁇ N coefficient matrices and m, l are N ⁇ 1 coefficient vectors. These entities carry the synthesis control parameters.
  • a simple graphical user interface depicted in Figure 7 serves as prototype GUI for giving to the user access to all the control parameters of interest.
  • the user is allowed to vary continuously the synthesis parameters from a minimum to a maximum value with the use of sliders.
  • An exception is the synthesis parameters m n and l n which are associated to the rate of pitchbend and non-linear decay in Eq. (5.10).
  • a pop-up window to set integer values is designed for that purpose.
  • the GUI is divided in two panels, the "Oscillator” panel and the “Coupling” panel. All sliders in the “Oscillator” panel refer to processes that are intrinsic to each oscillator. Namely, “attack time”, “decay rate”, “frequency”, “pitch bend”, “nonlinear decay” and “amplitude control” apply changes to the diagonal of matrices B, A, C R , C I and P respectively while the “intensity” slider is used for controlling the value of r p,n .
  • the "Coupling” panel includes the sliders that are necessary for controlling frequency modulation ("FM”). Amplitude modulation ("AM”) and additive coupling (“AC”), therefore affecting the elements of matrices E, D and A respectively which are again confined between a minimum and maximum value in the real domain.
  • An additional set of radio buttons is used for selecting the index of the "interfering oscillator. For example, if n is the index of the selected radio button in the "Oscillator” panel and j the corresponding index in the "Coupling” panel, the changes applied to the sliders will be reflected to the n-th row and j-th column of A, D and E.
  • Fig. 8(a) The work-flow of the data collection approach is shown in Fig. 8(a) .
  • Any conventional digital recording equipment can be used in order to capture the natural sound of the percussive instrument. If done with a computer, a microphone will be required, and a sound card equipped with a microphone pre-amplifier. Alternatively, stand-alone devices can also be used.
  • the musician is requested to perform several hits of varying impact level. This is advantageous in order to create a generic model which will output realistic synthesis results for a wide dynamic range.
  • Some pre-processing of the recorded or exported audio file is also required.
  • an automatic onset detection algorithm is built which can be used in order to crop a large audio file, containing multiple realizations of the physical instrument, into multiple smaller audio files, each one containing a single realization of the physical instrument. After these files have been created, they are named and saved in the hard disk for further processing.
  • the work-flow for the reconstruction process is shown in Fig. 8(b) .
  • the first step in the reconstruction process is to import the necessary audio samples from the instrument's database.
  • various temporal and frequency characteristics of the samples are plotted. Based on these plots, decisions such as the duration of the input signal, the number and the band-pass limits of the different frequency zones, the number and the type of the basis functions and many other parameters are taken.
  • the imported samples need to be processed according to the reconstruction parameters defined by the user, providing so the input data that will be used for the reconstruction process.
  • more than one audio samples of the same instrument might be required for the analysis, each time that a model is built. Special attention must be paid in that case so that all samples have the same duration and the same time onset.
  • Fig. 8(c) The work-flow of the synthesis process is shown in Fig. 8(c) .
  • the model corresponding to a particular instrument is loaded.
  • various parameters can be specified by the user. These concern the duration of the synthesized signal, the sampling rate and the impact level. Additional parameters which can be taken into account concern the decay rate and the pitch of the synthesized sound.
  • the ODEs are integrated forward in time and the synthesis result is plotted and heard.
  • N coupled ODEs previously shown in Eq. 4.1.
  • N different functions f n ( ⁇ ) are defined and it is assumed that each function can be decomposed into M n basis terms g n ( ⁇ ).
  • the problem is expressed in matrix vector form.
  • the number of time points I are considered to be greater than the number of basis vectors M and therefore, matrix W is overdetermined.
  • the square of the energy of the reconstruction error is normalized with respect to the energy of the target measurement; a value of E LS equal to 0 therefore denotes a perfect reconstruction result.
  • This task can be mathematically formulated as min ⁇ y ⁇ ⁇ Wc ⁇ 2 subject to Ac ⁇ B , where matrix A and vector B can be defined so as to express any linear inequalities.
  • Constrained optimization can be used in order to restrict the sign of some coefficients in regions where the ODE remains stable. Additionally, one might use it in order to control the rate of decay or the fundamental frequency of the ODE, especially in low order ODEs where the coefficients of the basis functions have a well understood physical meaning. Apart from the use of inequalities, one can also fix the value of the ODE coefficients to a desired value.
  • Constrained optimization can be performed by using the built-in Matlab routine lsqlin.
  • sparse reconstruction techniques can be mathematically formulated as min ⁇ y ⁇ ⁇ Wc ⁇ 2 + ⁇ ⁇ c ⁇ o , where
  • o measures the number of non-zero elements in vector c. While the applicability of this optimization strategy in signal compression and dimension reduction is apparent, it serves by helping to define ODEs with a small only number of terms, and therefore, with reduced computational complexity.
  • MP Matching Pursuit
  • OMP Orthogonal Matching Pursuit
  • the MP algorithm (shown in Table 3) is an iterative technique based on three fundamental steps; an initialization step, a selection step and a residual update step.
  • the algorithm requires that all basis vectors g m are normalized to have equal energy.
  • the algorithm begins by looking among all candidate basis vectors in order to find the one which has the highest correlation with the target measurement. A coefficient is returned by projecting the measurement onto the selected basis vector, and the contribution of the given vector to the measurement is removed, returning a residual.
  • the same process is then repeated, but the initial measurement is now replaced by the residual calculated at the previous iteration. The process is repeated until a maximum number of iterations K max has been reached.
  • OMP shown in Table 4, has exactly the same initialization and selection step as MP, but the coefficients corresponding to the selected vectors are updated simultaneously by projecting the initial target measurement onto the space spanned by the entire set of selected vectors.
  • the number of columns of the matrix with the collection of selected vectors D ⁇ i increases by one at each iteration.
  • the value of the coefficients in OMP is thus allowed to vary at each iteration, whereas in MP, once a coefficient value has been defined, it is not allowed to change any more.
  • OMP has in general better convergence properties than MP, but it includes a matrix inversion step which makes it slower than MP.

Landscapes

  • Physics & Mathematics (AREA)
  • Nonlinear Science (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Electrophonic Musical Instruments (AREA)

Claims (12)

  1. Verfahren zum synthetischen Modellieren eines Tonsignals, wobei das Verfahren von einem Prozessor einer Rechenvorrichtung durchgeführt wird und die folgenden Schritte umfasst:
    - Bereitstellen eines Eingangstonmusters als Ziel-Signal,
    - Analysieren des Eingangstonmusters, um ein Modell einer Differenzialgleichung zu bestimmen, wobei die Differenzialgleichung durch mindestens eine Funktion dargestellt ist, wobei die mindestens eine Funktion von mindestens einer Variablen und mindestens einem Parameter abhängt,
    - Bestimmen der mindestens einen Funktion, des mindestens einen Parameters und einer Anfangsbedingung der Differenzialgleichung derart, dass die Differenzialgleichung ein dynamisches Modell des Ziel-Signals bereitstellt, und
    - Bestimmen der Qualität des dynamischen Modells unter Verwendung einer Optimalitätsbedingung,
    dadurch gekennzeichnet, dass
    - mindestens eine zeitverzögerte Version des Eingangstonmusters bereitgestellt wird, und die mindestens eine zeitverzögerte Version des Eingangstonmusters analysiert wird, um das Modell der Differenzialgleichung zu bestimmen, und/oder
    - eine Zeitableitung des Eingangstonmusters bestimmt wird und die Zeitableitung des Eingangstonmusters analysiert wird, um das Modell der Differenzialgleichung zu bestimmen, wobei die Qualität des Modells der Differenzialgleichung mittels einer Optimalitätsbedingung bestimmt wird.
  2. Verfahren nach Anspruch 1, wobei das Modell der Differenzialgleichung ein stochastisches Modell ist und die mindestens eine Funktion eine nichtlineare Funktion ist.
  3. Verfahren nach Anspruch 1 oder 2, wobei der mindestens eine Parameter der mindestens einen Funktion gemäß einer vordefinierten Funktion oder einer Benutzereingabe dynamisch geändert wird.
  4. Verfahren nach einem der vorhergehenden Ansprüche, ferner umfassend Zerlegen des Eingangstonmusters in mehrere Komponenten.
  5. Verfahren nach einem der vorhergehenden Ansprüche, ferner umfassend das Bestimmen einer Anzahl von Dimensionen des Modells der Differenzialgleichung.
  6. Verfahren nach einem der vorhergehenden Ansprüche, ferner umfassend das Transformieren einer realwertigen Komponente des Eingangstonmusters in einen komplexen Wert.
  7. Verfahren nach einem der Ansprüche 1, 2 und 4 bis 6, wobei der mindestens eine Parameter der mindestens einen Funktion eine Konstante ist.
  8. Verfahren nach einem der vorhergehenden Ansprüche, wobei die mindestens eine Funktion eine zeitabhängige Funktion ist.
  9. Verfahren nach einem der Ansprüche 1 bis 7, wobei eine Hüllkurve des Eingangstonmusters durch eine dynamische Gleichung modelliert wird, wobei die dynamische Gleichung mindestens eine weitere Differenzialgleichung umfasst.
  10. Verfahren nach einem der vorhergehenden Ansprüche, ferner umfassend das Ausgeben eines Tonsignals, welches durch Entwickeln des Modells der Differenzialgleichung nach der Zeit erzeugt wird.
  11. System zum synthetischen Modellieren eines Tons, umfassend einen Prozessor und einen Speicher und eingerichtet zum:
    - Bereitstellen eines Eingangstonmusters als Ziel-Signal,
    - Analysieren des Eingangstonmusters, um ein Modell einer Differenzialgleichung zu bestimmen, wobei die Differenzialgleichung durch mindestens eine Funktion dargestellt ist, wobei die mindestens eine Funktion von mindestens einer Variablen und mindestens einem Parameter abhängt,
    - Bestimmen der mindestens einen Funktion, des mindestens einen Parameters und einer Anfangsbedingung der Differenzialgleichung derart, dass die Differenzialgleichung ein dynamisches Modell des Ziel-Signals bereitstellt, und
    - Bestimmen der Qualität des dynamischen Modells unter Verwendung einer Optimalitätsbedingung,
    dadurch gekennzeichnet, dass das System ferner eingerichtet ist zum:
    - Bereitstellen mindestens einer zeitverzögerten Version des Eingangstonmusters, und Analysieren der mindestens einen zeitverzögerten Version des Eingangstonmusters, um das Modell der Differenzialgleichung zu bestimmen, und/oder
    - Bestimmen einer Zeitableitung des Eingangstonmusters und Analysieren der Zeitableitung des Eingangstonmusters, um das Modell der Differenzialgleichung zu bestimmen, und, falls zutreffend, Bestimmen der Qualität des Modells der Differenzialgleichung mittels einer Optimalitätsbedingung.
  12. Computerprogrammprodukt, das, wenn es von einem Prozessor ausgeführt wird, das Verfahren nach einem der Ansprüche 1 bis 10 ausführt.
EP14189713.2A 2014-10-21 2014-10-21 Verfahren und System für die synthetische Modellierung eines Klangsignals Not-in-force EP3012832B1 (de)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP14189713.2A EP3012832B1 (de) 2014-10-21 2014-10-21 Verfahren und System für die synthetische Modellierung eines Klangsignals

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP14189713.2A EP3012832B1 (de) 2014-10-21 2014-10-21 Verfahren und System für die synthetische Modellierung eines Klangsignals

Publications (2)

Publication Number Publication Date
EP3012832A1 EP3012832A1 (de) 2016-04-27
EP3012832B1 true EP3012832B1 (de) 2019-03-06

Family

ID=51753093

Family Applications (1)

Application Number Title Priority Date Filing Date
EP14189713.2A Not-in-force EP3012832B1 (de) 2014-10-21 2014-10-21 Verfahren und System für die synthetische Modellierung eines Klangsignals

Country Status (1)

Country Link
EP (1) EP3012832B1 (de)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119902092B (zh) * 2025-04-01 2025-06-13 山东大学 基于多模态重构融合的锂离子电池热失控预警方法及系统

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4736663A (en) * 1984-10-19 1988-04-12 California Institute Of Technology Electronic system for synthesizing and combining voices of musical instruments
DE19917434C1 (de) * 1999-04-19 2000-09-28 Rudolf Rabenstein Vorrichtung zur Signalberechnung und -erzeugung, insbesondere zur digitalen Klangsynthese
US7548837B2 (en) * 2004-01-14 2009-06-16 Apple Inc. Simulation of string vibration
RU2364956C1 (ru) * 2008-04-29 2009-08-20 Дмитрий Эдгарович Эльяшев Синтезатор музыкального инструмента с физическим моделированием

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
EP3012832A1 (de) 2016-04-27

Similar Documents

Publication Publication Date Title
US12080311B2 (en) Machine-learned differentiable digital signal processing
US10068557B1 (en) Generating music with deep neural networks
Bilbao Numerical sound synthesis: finite difference schemes and simulation in musical acoustics
RU2679254C1 (ru) Устройство и способ для обработки аудиосигнала для получения обработанного аудиосигнала с использованием целевой огибающей во временной области
Aramaki et al. Analysis-synthesis of impact sounds by real-time dynamic filtering
Stöter et al. Common fate model for unison source separation
Hermus et al. Perceptual audio modeling with exponentially damped sinusoids
Olivero et al. A class of algorithms for time-frequency multiplier estimation
Christensen et al. Optimal filter designs for separating and enhancing periodic signals
JP2019515323A (ja) スペクトログラムに対する構造テンソルを用いた調波打楽器残差音声分離装置および方法
JP2019078864A (ja) 楽音強調装置、畳み込みオートエンコーダ学習装置、楽音強調方法、プログラム
Shier et al. Real-time timbre remapping with differentiable DSP
Issanchou et al. Nonsmooth contact dynamics for the numerical simulation of collisions in musical string instruments
Serra Introducing the phase vocoder
Fitz A new algorithm for bandwidth association in bandwidth-enhanced additive sound modeling
Karjalainen et al. Efficient modeling and synthesis of bell-like sounds
EP3012832A1 (de) Verfahren und System für die synthetische Modellierung eines Klangsignals
Virtanen Algorithm for the separation of harmonic sounds with time-frequency smoothness constraint
Wells Modal decompositions of impulse responses for parametric interaction
Boccardi et al. Sound morphing with Gaussian mixture models
Mohamad et al. Pickup position and plucking point estimation on an electric guitar via autocorrelation
Dubnov Polyspectral analysis of musical timbre
Martínez Ramírez Deep learning for audio effects modeling
Karjalainen et al. Making of a computer carillon
Zambon et al. Simulation of piano sustain-pedal effect by parallel second-order filters

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

17P Request for examination filed

Effective date: 20161019

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

RIC1 Information provided on ipc code assigned before grant

Ipc: G10H 5/00 20060101AFI20170711BHEP

17Q First examination report despatched

Effective date: 20170719

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20180605

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20180924

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: UNIVERSITAET POTSDAM

Owner name: AMBROSYS GMBH

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: AT

Ref legal event code: REF

Ref document number: 1105622

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190315

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602014042233

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: SE

Ref legal event code: TRGR

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20190306

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190606

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190607

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190606

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1105622

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190306

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190706

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602014042233

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190706

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

26N No opposition filed

Effective date: 20191209

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191021

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191031

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191031

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20191031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191031

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20191021

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191021

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191021

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20141021

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: SE

Payment date: 20211021

Year of fee payment: 8

Ref country code: DE

Payment date: 20211101

Year of fee payment: 8

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20211021

Year of fee payment: 8

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602014042233

Country of ref document: DE

REG Reference to a national code

Ref country code: SE

Ref legal event code: EUG

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221031

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230503

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221022