ES2364005T3 - PROCEDURE, DEVICE AND MEANS OF THE COMPUTER PROGRAM CODE FOR VOICE CONVERSION. - Google Patents

Abstract

A method for converting a speech signal from a source speaker into a converted voice signal, comprising the steps of: - a training stage, in which: - given a training database of parallel source and target data , for each tone period of said training database: - model each tone period by means of a glotal wave and a glotal tract filter according to the Lu and Smith model, to obtain a set of Liljencrants parameters - Fant LF, said set of parameters LF comprises an excitation force parameter E e and a set of parameters T, T p, T e, T a, T c that model a glottal wave and a set of omnipolar vocal tract filter coefficients (α1 ... α p); - converting said parameters T, T p, T e, T a, T c into parameters R g, R k, R a; - converting said omnipolar vocal tract filter coefficients (α 1 ... α p) into in-line spectral frequencies on the Bark scale lsf 1 ... lsf p; - define a glotal vector G to convert; - defining an LSF vocal tract vector to be converted, said LSF vocal tract vector comprises said in-line spectral frequencies on the Bark scale lsf 1 ... lsf p; - apply the elimination of noise from the wavelet to obtain an estimate of the glotal aspiration noise; - from the set of LSF vocal tract vectors obtained for each tone period of said training database, estimate a function of continuous probabilistic linear transformation of the vocal tract using the minimum quadratic error criterion; The procedure is characterized in that said modeling step further comprises the steps of: - modeling said estimation of the aspiration noise by modulating the Gaussian noise of the unit variance close to zero with the aforementioned modeled glotal wave and adjusting its ANE energy to coincide with the aforementioned suction noise estimation; said glotal vector G to be converted comprises said excitation force parameter E e, said parameters R, R g, R k, R a and said ANE energy of the aspiration noise estimation, the method further comprises: - a conversion stage in which a given test speech wave is modeled and transformed into a set of parameters E e ', R g', R k, R a ', ANE', LSF '; - a synthesis stage in which a converted speech wave is synthesized from said set of converted parameters E e ', R g', R k, R a ', ANE', LSF '.

Description

Field of the Invention

The present invention relates to procedures and systems for voice conversion.

State of the art

Voice Conversion is aimed at transforming the speech of a source speaker to sound like a different target speaker. Text-to-speech synthesizers, dialogue systems and speech repair are among the many applications that can benefit greatly from the development of voice conversion technology.

Representations of the most widely used voice signals are the Source - Filter Model and the Sinusoidal Model. The Source - Filter representation (G, Fant, Acoustic Theory of Speech Production, ISBN 9027916004) is based on a simple production model composed of a glotal source wave that excites a variable filter in the time charged at its output by radiation from the lips. The main challenge in the Source - Filter model is the estimation of the glottal wave and the parameters of the vocal tract filter of the speech signal.

Among the existing parameterizations of the glottal wave, the Liljencrants-Fant (LF) model (The LF-model revisited. Transformation and frequency domain analysis, STL - QPSR, vol. 36, No. 2 - 3, 1995, p. 119 - 156) has become the model of choice for research on the glottal source. He has proven to be able to model a wide range of phonations that occur naturally and the effects of variations in their parameters are easy to understand. It exploits the linearity of the properties of the invariance over time of the Source - Filter representation and assumes the switching of the filters of the vocal tract and the radiation of the lips to combine the source excitation model and the radiation of the lips in the parameterization of the derivative of the glottal wave.

Linear Prediction (LP) is a popular technique used to obtain a combined parameterization of the components of the glottal source, the vocal tract and the radiation of the lips in a single omnipolar filter H (z). Said filter is excited, as shown in Figure 1, by a sequence of pulses separated in the fundamental period To during the sound speech and by the white Gaussian noise during the non-sound speech. If the speech signal were really the response of an omnipolar filter, the LP error or residual would be a separate pulse train at the moments of sound excitation and the model of the impulse / noise voice source would be accurate. In practice, however, the LP residue looks more like a white noise signal with higher values around the moments of excitation. Although the excitation of the LP filter with the LP residue results in speech that is indistinguishable from the original, using a pulse train as sound excitation produces speech with a very buzzing quality. The strength of the LP rests on its ability to automatically estimate a set of filter coefficients that compactly represent the speech spectrum cover, making it popular in applications where the spectral characteristics of the voice wave need to be captured with A small number of parameters. Its main drawback, on the other hand, comes from the over-simplified model of the glottal wave that avoids its use in systems that require high quality voice outputs.

As an alternative to LP, H. Lu et al. Have proposed a convex optimization procedure to automatically estimate the vocal tract filter and the glottal wave together (Joint estimation of vocal tract filter and glottal source waveform via convex optimization, Proc, 1999 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Paltz, New York, Oct. 17-20, 1999). The best model of the glottal source used by this approach results in speech that has a better quality than that of LP. In addition, the parameterization of the glottal wave allows its parametric modification, which can be exploited in voice conversion applications.

Sinusoidal Models assume that the speech wave is composed of the sum of a small number of sinusoids with varying amplitudes, frequencies and phases over time. This model was mainly developed by McAulay and Quatieri (Speech Analysis / Synthesis Based on a Sinusoidal Representation, IEEE Transactions on Acoustics, Speech and Signal Processing, p. 744-754, 1986) in the mid-1980s and has proven capable of producing High quality speech even after tone and time scale transformations. However, due to the high number of sinusoidal amplitudes, frequencies and phases involved, the sinusoidal model is less flexible than the Source - Filter representation to modify spectral characteristics.

To obtain high-quality converted speech, voice conversion (VC) implementations of the current technique mainly employ variations and extensions of the original sinusoidal model. In addition, they generally adopt a Source-Filter formulation based on LP to carry out the spectral transformations.

Spectral covers are generally encoded in line spectral frequencies (LSF) for voice conversion, since LSFs have proven to have very good linear interpolation characteristics and relate well to format allocation and bandwidth. Since the frequency resolution of the human ear is higher at low frequencies than at high frequencies, spectral covers are often wrapped in a nonlinear scale, for example, the Bark scale, taking into account the non-uniform sensitivity of the human ear . Usually only the spectral covers of segments of sound speech are transformed, since the non-sound sounds contain little information of the vocal tract and its spectral covers present high variations. Among the different conversion techniques of existing spectral covers, it has been found that linear transformations of continuous probability are more robust and efficient. These can be obtained through at least a minimization of the quadratic error of the source and target training databases or using general frameworks of maximum probability transformation (Ye, H. and Young, S. Quality-enhanced Voice Morphing using Maximum Likelihood Transformations, IEEE Audio Speech and Language Processing, vol. 14, No. 4, p. 1301-1312, 2006). A problem of the sharing of all spectral cover conversion procedures is the expansion of the spectral peaks, the expansion of the forming bandwidths and the over-smoothing caused by the averaging effect of the parameter interpolations. The phenomenon causes the converted speech sound to be slightly muted. To resolve this issue, post-filtering is often applied as a post-processing stage for narrow bandwidth bandwidths and noise in the spectral valleys is suppressed, such as in Ye, H. and Young, S. Quality-enhanced Voice Morphing using Maximum Likelihood Transformations, IEEE Audio Speech and Language Processing, vol. 14, no. 4, p. 1301-1312, 2006.

As for the conversion of LP residues, VC sinusoidal systems have developed procedures for prediction and selection of residues (D. Suendermann, A. Bonafonte, H. Ney, and J. Hoege, A study on residual prediction techniques for voice conversion , in Proc. ICASSP, 2005, p. 13-16) based on the correlation between the spectral cover and the LP residues. These procedures reintroduce the spectral target detail lost after the cover conversion. Since the residues contain the errors introduced by the LP parameterization, it has been found that the waste prediction techniques improve the conversion efficiency. However, LP residues do not constitute an exact voice source model and waste prediction alone is not able to modify the quality of the voice source. This does not allow its use in applications that require voice clarity modifications such as, for example, speech repair.

Patent application WO 2008/018653 A1 presents an additional voice conversion technique that uses the Liljencrants-Fant parameters of the glottal wave.

Summary of the Invention

It is therefore an object of the present invention to provide a voice conversion method based on a Source-Filter model that uses a more accurate representation of the glottal source than LP residues. This allows the use of continuous probabilistic linear transformations for the conversion of the voice source.

In particular, an object of the present invention is a method for converting a speaker's voice signal into a converted voice signal, which comprises a training stage, a conversion stage and a synthesis stage.

The training stage comprises, given a training database of parallel source and target data, for each tone period of said training database: the modeling of each tone period by means of a glottal wave and a filter of the vocal tract according to the Lu and Smith model, to obtain a set of LF parameters, said set of LF parameters comprises an excitation force parameter and a set of T parameters that models a glottal wave, and a set of omnipolar vocal tract filter coefficients; which convert said parameters T into parameters R; converting said omnipolar vocal tract filter coefficients into in-line spectral frequencies on the Bark scale; defining a glotal vector to convert; defining a vocal tract vector to be converted, said vocal tract vector comprising said spectral frequencies in line on the Bark scale, applying elimination of noise from the wavelet to obtain an estimate of a glotal aspiration noise.

The training stage also comprises, from a set of vocal tract vectors obtained for each tone period of said training database, the estimation of a continuous probabilistic linear transformation function of vocal tract using the error criterion minimum quadratic.

The previous modeling stage also includes the steps of modeling said estimation of aspiration noise by modulating the Gaussian noise of unit variance close to zero with the aforementioned modeled glotal wave and adjusting its energy to coincide with that of said estimation of aspiration noise. In addition, the glotal vector to be converted comprises said excitation force parameter, said R parameters and said aspiration noise estimation energy.

In the conversion stage, a test voice wave is modified and transformed into a set of converted parameters.

In the synthesis stage, a converted voice wave is synthesized from said set of converted parameters.

Preferably, the training stage further comprises: from the set of glottal vectors obtained for each tone period of said training database, the estimation of a continuous probabilistic linear transformation function of the glottal wave using the error criterion minimum quadratic.

The modeling step of each tone period by means of a glottal wave and a vocal tract filter according to the Lu and Smith model, preferably comprises the steps of: modeling the glottal wave using the Rosenberg-Klatt model; using convex optimization to obtain a set of Rosenberg-Klatt glottal wave parameters and all omnipolar vocal tract filter coefficients, wherein said step of using convex optimization comprises an adaptive pre-emphasis step to estimate and eliminate the contribution of the Spectrum tilt filter of the speech wave before convex optimization. In addition, the step of modeling each tone period by means of a glottal wave and a vocal tract filter according to the Lu and Smith model, further comprises the steps of: obtaining a glottal wave derived by inverse filtering of said period of tone using said omnipolar vocal tract filter coefficients, said set of LF parameters adjusting to said derived glotal wave filtered by direct estimation and constrained nonlinear optimization.

The conversion stage preferably comprises, for each tone period of said test voice wave: obtaining a glotal vector to be converted, said glotal vector comprising an excitation force parameter, a set of R parameters and the energy of said estimate of suction noise; obtaining the vocal tract vector to be converted, said vocal tract vector comprising a set of spectral frequencies in line on the Bark scale; the application of said continuous probabilistic linear transformation of vocal tract function estimated during the training stage to obtain a vector of converted vocal tract parameters; the transformation of said glotal vector using said continuous probabilistic linear transformation function of the estimated glottal wave during the training stage thus obtaining a converted glotal vector comprising a set of converted parameters.

In particular, these steps of obtaining a glotal vector to be converted and a vocal tract vector to be converted further comprise the steps of: modeling each tone period by means of a glotal wave and a vocal tract filter according to the model of Lu and Smith, to obtain a set of LF parameters, said set of LF parameters comprises an excitation force parameter and a set of T parameters that model a glottal wave and a set of omnipolar vocal tract filter coefficients; converting said omnipolar vocal tract filter coefficients into in-line spectral frequencies on the Bark scale; converting said parameters T into parameters R; define a glotal vector to convert; and define a vocal tract vector to convert.

Preferably, the conversion step further comprises a post-filtering step of said converted vocal tract parameter vector.

The synthesis step, in which said converted voice wave is synthesized from said set of converted parameters, preferably comprises the steps of: interpolating the paths of said converted parameters of each tone period, thus obtaining a set of interpolated parameters comprising interpolated R parameters, interpolated energy and an interpolated vocal tract vector, converting said interpolated vocal tract vector into an omnipolar filter coefficient vector; converting said interpolated R parameters into interpolated T parameters; for each frame of said test voice wave, generating an excitation signal.

Preferably, the step of generating an excitation signal comprises, for each of said frames: if said frame is sound: from said interpolated parameters T and said excitation force parameter, generate a glottal wave; from said interpolated aspiration noise energy parameter generate interpolated aspiration noise; generating said sound excitation signal by adding said interpolated glotal wave and said interpolated aspiration noise. And, if said frame is not sound: generate said non-sound excitation signal from a Gaussian noise source.

In addition, the synthesis step further comprises: the generation of a synthetic contribution of each frame by filtering said excitation signal with said omnipolar filter coefficient vector, the multiplication of said synthetic contribution by means of a Hamming window, the superposition and the addition, to generate the converted voice signal.

The present invention also provides a method applicable to voice quality transformations, such as repair of tracheo-esophageal speech, comprising at least some of the steps of the aforementioned procedure.

Another object of the present invention is to provide a device comprising means for carrying out the aforementioned procedure.

Finally, a further object of the present invention is to provide computer program code means adapted to perform the steps of the aforementioned procedure when said program is executed in a computer, a digital signal processor, an array of field-programmable doors, a application-specific integrated circuit, a microprocessor, a microcontroller or any other form of programmable hardware.

The advantages of the proposed invention will be apparent in the following description.

Brief description of the drawings

To complete the description and to provide a better understanding of the invention, a set of drawings is provided. Said drawings form an integral part of the description and illustrate a preferred embodiment of the invention, which should not be construed as restricting the scope of the invention but, on the contrary, as an example of how the invention can be realized. The drawings comprise the following figures:

Figure 1 shows a conventional schematic diagram of the LP model.

Figure 2 shows a schematic diagram of the joint estimation analysis synthesis model (JEAS) of according to an embodiment of the present invention. Figure 3 shows a schematic diagram of the modeling of the glottal wave. Figure 4 shows a schematic diagram of the modeling of the derived glottal wave. Figure 5 shows typical LF pulses that correspond to the derived glotal and glotal waves. Figure 6 shows a conventional model of the voice source. Figure 7 shows an example of a joint estimate: a) speech period, b) speech spectrum and spectral cover

jointly estimated, c) inverse filtered residue and RK wave jointly estimated. Figure 8 shows a glotal wave derived RK. Figure 9 shows a schematic diagram of a conventional model of the spectral inclination. Figure 10 shows the effects of adaptive pre-emphasis. Figure 11 shows an example of LF adjustment in normal, chopped and forced phonations. Figure 12 shows a typical noise elimination result. Figure 13 shows parameters of the standard suction noise model. Figure 14 shows the modulation of Gaussian noise by an LF wave. Figure 15 shows a schematic diagram of a suction noise modeling approach according to

An embodiment of the present invention. Figure 16 shows a schematic diagram of the overlay-addition synthesis scheme employed. Figure 17 illustrates the new sampling of the frame size contour. Figure 18 shows the spectral covers of JEAS versus PSHM. Figure 19 shows the continuous probabilistic linear transformation of the LF glottal waves

Figure 20 shows the RLSF distortion ratios of the spectral shells of PSHM and converted JEAS.

Figure 21 shows the RLSD distortion ratios of the predicted residual (RP) and wave converted spectra glottal (GWC). Figure 22 shows the results of the ABX test. Figure 23 shows the result of the quality comparison test.

Detailed description of the invention

Definitions

In the context of the present invention, the term "approximately" and the subgroup terms (such as "approximate", "approximation", etc.) should be understood as indicative values or forms very close to those that accompany the aforementioned term. That is, a deviation from an exact value can be accepted within reasonable limits, since any person skilled in the art will understand that said deviation from the indicated values or forms is inevitable due to the inaccuracies of the measurements, etc. The same applies to the term "close."

In the context of the present invention, the following terms are defined as follows:

The term "tone period" means a segment of a voice wave that comprises a period of the fundamental frequency.

The term "frame" means a segment of a voice wave, which corresponds to the period of the tone in the sound parts and with a fixed amount of time in the non-sound parts. In a preferred embodiment of the present invention, which should not be construed as a limitation of the present invention, a frame corresponds to 10 milliseconds in the non-sound parts.

The expression "source data" refers to a collection of voice waves launched by a source speaker, while the expression "target data" refers to a collection of voice waves launched by a target speaker. In addition, the expression "parallel source and target data" refers to a collection of voice waves launched by both source and target speakers.

In this text, the term "comprises" and its derivatives (such as "understanding", etc.) should not be understood in an exclusive sense, that is, these terms should not be construed as excluding the possibility that what is described and defined may include additional elements or steps.

1. Summary of the Joint Estimation Analysis

Next, a speech modeling procedure for speech analysis, modification and synthesis is described. The model is called synthesis of joint estimation analysis (JEAS). Its greatest advantage is the automatic and simultaneous parameterization of the vocal tract and the voice source, which allows the manipulation not only of the spectral covers, but also of glottal characteristics. In addition, it also supports high quality tone and modifications in the time scale. Next, the voice source model used and the deconvolution technique of the source filter are described and mode analysis, synthesis and prosodic transformations are implemented.

1.1 Speech model

Figure 2 shows a schematic diagram of the JEAS model. It is based on a general representation of Source - Filter. It employs white Gaussian and white Gaussian amplitude modulated noise to modulate the components of turbulence and aspiration noise respectively, A differentiator for Lip Radiation and an omnipolar filter to represent the Vocal Tract. In addition, the Liljencrants - Fant (LF) model is adopted to better capture the characteristics of the derived glottal wave. Then, to estimate the different parameterizations of the components of the speech wave model, a technique of estimating the parameters of the union voice source and the vocal tract is applied based on Convex Optimization.

Then, the modeling of the voice source is explained.

Numerous parametric models of the glottal source have been proposed in the literature. Despite their differences, they all share many common characteristics and can be described by a small set of parameters. In most cases, the properties of linearity and time invariance are exploited in the representation of Source - Filter and assume the switching of the filters of the vocal tract and the radiation of the lips to combine the model of the source excitation and of the radiation of the lips in the parameterization of the derivative of the glottal wave as shown in Figure 4.

The present procedure adopts the well-known LF model, which is a four-parameter time domain model of a derived glotal wave cycle. Typical LF pulses that correspond to the derived glotal and glotal waves are shown in Figure 5. Mathematically, it can be described as

image 1

The model consists of two segments: the first characterizes the glottal wave derived from the instant of the glottal opening to the moment of main excitation Te, where the amplitude reaches the maximum negative value –Ee.

As shown in equation 1, the segment is a sinusoidal function that grows exponentially in amplitude, the frequency of the sine function being and determining α the rate of amplitude increase. E0 is a scale factor used to ensure that the signal is close to zero. The synchronization parameter image 1 Tp is related to the sinusoidal frequency through

image 1

5 and denotes the moment of maximum glottal flow. Ee is closely related to the force of the excitation of the source and the main determinant of the intensity of the speech signal. Its variation affects the total harmonic amplitudes, except for very low components that are more determined by the shape of the pulse.

The second segment models the closing or return phase from the main excitation Te to the moment of total closing Tc using an exponential function. The duration of the return phase is determined in this way by Tc-Te. He

The main parameter that characterizes this segment is Ta, which represents the “effective duration” of the return phase. This is defined by the duration from Te to the point where a tangent set at the start of the return phase crosses zero. ε-1 is the time constant of the exponential function and can be determined iteratively from Ta, Te and Tc through

image 1

15 To corresponds to the fundamental period. Generally Tc is made to coincide with the opening of the next impulse. This fact could suggest that the model does not take into account the closed phase of the glottal wave. However, for reasonably small values of Ta the exponential function will adjust very close to the zero line that supplies a closed phase without the need for additional control parameters.

Together with the excitation force Ee, the pulse LF can only be determined by the parameters T (Tp, Te, Ta,

20 Tc). These parameters can be easily identified from the estimated derived glottal wave. Therefore, the synthesis parameters are usually obtained first and then, from which the LF wave can be calculated directly, (E0, α; ωg; ε) are derived taking into account the following restrictions:

image 1

Another important set of parameters LF are the parameters R (Rg, Rk, Ra), which are normalized with respect to T0 and correlate with the most outstanding glottal phenomena, that is, the width of the glotal impulse and the asymmetry and abruptness of the closure

image 1

Rg is a normalized version of the frequency of the glotal formant Fg which is defined as the inverse of twice the duration of the opening phase Tp. Rk is the parameter of LF that captures the glotal asymmetry. It is defined as the ratio between the times of opening and closing of the branches of the glotal impulse, and the larger its value, the more symmetrical is the impulse. The relationship between Rg, Rk and the Opening Ratio OQ is: OQ = (1 + Rk) / (2 Rg). Thus, OQ correlates positively with Rk and correlates negatively with Rg. The Ra parameter corresponds to the effective “return time” Ta normalized by the fundamental period and captures the differences related to the spectral inclination.

Then, the procedure adopted for the deconvolution of the glottal source and the vocal tract filter is applied:

The objective of the Source - Filter deconvolution is to obtain estimates of the components of the glottal source and the vocal tract filter from the speech wave. There are two main approaches to deconvolution. Before the parametric models of the glottal wave were developed, Reverse Filtering (IF) was the most commonly used deconvolution procedure. It is based on the calculation of a transfer function of the vocal tract filter, whose inverse is used to obtain the estimate of the glottal wave that can then be parameterized.

A different approach includes modeling both the glottal source and the vocal tract filter, and development techniques to jointly estimate the parameters of the source and tract models from the speech source. The joint estimation procedures are completely automatic. This is an important condition that a mathematical model oriented to the analysis, synthesis and modification of the speech signal must satisfy. Due to the characteristics of the mathematical voice source and the descriptions of the vocal tract, such an approach is a complex nonlinear problem. For this reason, the use of the LP has been developed more widely than a simpler method to obtain a parameterization of Source - Direct and efficient filter of the voice signal. Its poor modeling of the voice source has not limited its application in the context of speech coding and efficiently represents the speech spectrum with a small number of parameters. However, its use in speech synthesis and transformation applications has been avoided. Advances in voice conversion and speech synthesis of the Hidden Marco Model (HMM) in recent years have emphasized the importance of redefined voice coding, and thus, the problem of joint estimation has gained renewed interest Automatic voice source and vocal tract filter parameters.

The procedure used to obtain the JEAS voice source and vocal tract model parameters from the speech wave follows the second deconvolution approach and is based on the joint estimation of the vocal tract filter and the glotal wave proposed by Lu and Smith (Proc. 1999 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Paltz, New York, Oct. 17-20, 1999).

1.2 Analysis within the Joint Estimate Analysis Synthesis Model

During the analysis, the sound and non-sound speech segments are processed differently due to their different source characteristics. While the sound source in the speech is represented by a combination of LF and suction noise models, white Gaussian noise is used to excite the vocal tract filter in the non-sound frames (see Figure 2). Its different modeling requires a preprocessing step in which the sound and non-sound voice sections and the glottal closing moments (GCI) of the voice segments are determined. Then, the voice source and vocal tract parameters are obtained through a Source - Joint filter estimate and a LF reparameterization in the sound sections (V) and through the standard autocorrelation LP and Gaussian noise energy that agree on the non-sound parts (U).

An algorithm is used, such as the well-known projected dynamic programming phase slope (DYPSA) algorithm for GCI estimation. The group delay function is used in combination with a phase slope projection procedure to determine GCI candidates, plus N-best dynamic programming to select the most likely candidates according to a cost function that takes the similarity into account of the wave, the tone deviation, the normalized energy and the deviation of the ideal phase slope.

The reproduction decision is made based on energy information, zero crossing and GCI. The sound segments are then processed in synchrony with the tone while the non-sound frames are periodically extracted. In a particular embodiment, they are extracted every 10 milliseconds.

The method used by the invention to obtain the parameters of the JEAS voice source and vocal tract models comprises the use of a voice source model simple enough to allow the deconvolution of the source filter to be formulated as a Convex Optimization problem. Then, the derived glotal wave obtained by inverse filtering (IF) with the estimated filter coefficients is reparameterized by adjusting the LF model.

5 The success of the technique relies on the supply of a restriction of the derived glottal wave when the vocal tract filter is estimated. Because of this, the glottal wave derived from the resulting IF is closer to true glottal excitation and its adjustment with an LF model is less susceptible to errors.

The joint estimation algorithm models the voice source using the well-known Rosenberg-Klatt (RK) model, which consists of a basic reproduction wave that describes the shape of the derived glottal wave and a filter of

10 low pass, with μ> 0, as shown in figure 6. The RK derivative of the glottal wave is given by

image2

image 1

where To corresponds to the tone period and nc represents the duration of the closed phase, which can also be expressed as

image 1

where OQ is the open quotient, that is, the fraction of the tone period in which the glottis is open. In addition, parameters a and b need to always be positive and maintain the following relationship,

image 1

to maintain a proper waveform.

Source - Filter deconvolution through convex optimization is done by minimizing the quadratic error between moderate and true derived glottal waves. The modeled derived glottal wave ĝ (n) corresponds to that of equation 7, while the true derived glottal wave g (n) is obtained through inverse filtering such as

image 1

where s (n) is the speech wave and αk are the omnipolar filter coefficients of the vocal tract. The error between the modeled and true derivative glottal waves e (n) can be calculated by subtracting equations (7) and

image3

previous expression and rewriting it in matrix form we have

image 1

where X = [α1… αp a b] T is the vector of parameters to be estimated so that the sum of the squares of the error E of the equation is minimized, that is to say

image 1

H. Lu et al. Demonstrated (Proc. 1999 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Paltz, New York, Oct. 17-20, 1999) that the simplicity of the RK glotal model ensures that this optimization is convex, that is, that it has only a minimum that corresponds to the optimal solution, and thus, can be solved efficiently through Quadratic Programming. A quadratic problem is defined as follows

image 1

Equation (12) can be solved using quadratic programming if it expands so that it has its shape, that is

image 1

defining

image 1

and ignoring the term STS, which is always positive, for minimization purposes. In addition, equation 9 imposes the following restrictions on equality and inequality

image 1

The derived quadratic program can be solved using a number of existing iterative numerical algorithms. In the developed implementation, the quadratic programming function of the MATLAB Optimization Toolbox has been used. The result of the minimization problem is the simultaneous estimation of the parameters of the RK model a and b and the omnipolar filter coefficients αk. Figure 7 shows an example of a joint estimate for a tone period.

The joint estimation process described assumes that the closed and open phases are defined, although in practice the parameter that defines the end of the closed phase and the beginning of the open phase is not known. Its optimal value is uniformly sampling the possible values of nc (empirically shown to vary between 0% and 60% of the To tone period), solving the quadratic problem in each sampled nc value and selecting the estimate given as Minimum error result.

As can be seen in Figure 8, the basic reproduction wave RK of equation (7) does not explicitly model the return phase of the derived glottal wave and changes abruptly at the moments of the glottal closure. For this reason, a low-pass filter is added to the basic model, with the purpose of reducing the abruptness of the glottal closure. In the frequency domain, the filter coefficient μ is responsible for controlling the inclination of the source spectrum.

To allow the formulation of the convex optimization problem, the spectral tilt filter is separated from the source model and incorporated into the vocal tract model by adding an extra pole to the omnipolar filter as shown in Figure 9. This implies that The vocal tract filter coefficients estimated using this formula also encode the spectral slope information of the voice source. As a result, derived glottal waves obtained using this approach fail to properly capture the variations of the return phase of the glottal source.

The present invention uses adaptive pre-emphasis to estimate and eliminate the contribution of the spectral inclination filter of the speech wave before convex optimization. For an LP and IF analysis to be applied to estimate and eliminate the spectral slope of the voice frames under analysis, the effect of adaptive pre-emphasis is illustrated in Figure 10: a) Speech spectrum and estimated spectral cover, b ) IF-derived glottal wave and LF wave adjusted, c) Glottal wave-derived spectrum and LF wave spectrum adjusted. The estimates of the vocal tract filter cover obtained in this way do not encode the spectral inclination characteristics of the source, which are reflected in the closing phase of the resulting derived glottal waves. This improves the adjustment of the return phase of the LF model and thus of the high frequencies of the glottal source.

The LF model is able to more accurately describe the wave of the glotal derivative than the RK model. However, its more complex nonlinear formulation does not meet the convexity condition and avoids its use in the algorithm for estimating joint parameters of the voice source and the vocal tract filter. On the contrary, the RK model is used during the Source-Filter deconvolution and the LF model is then used to reparameterize the derived glotal wave obtained by inverse filtering of the speech wave with the jointly estimated filter coefficients.

The adjustment of the LF model is carried out in two steps: first, the initial estimates of the T parameters of LF (Tp, Te, Ta, Tc) and the glotal excitation force Ee are obtained from the source wave IF voice of the time domain by conventional direct estimation procedures. Then, their values are redefined using the conventional constrained nonlinear optimization technique. The complete procedure is as follows.

First, the glottal excitation force Ee and its time index are located, finding the minimum of the derived glotal wave IF. Then, Tp and Tc are determined as the first zero crossings before and after Te respectively. Ta is estimated as Ta = (Tc -Te) 2/3. Tp and Ta are redefined using constrained nonlinear minimization. Since the initial estimates of Ee, Te and Tc are quite reliable, their values remain unchanged during optimization. Ta is restricted to vary between 0 and Tc -Te and Tp to b within ± 20% of its initial estimate. The return and opening phases are optimized separately and sequentially. In both cases, the minimization function is the sum of the quadratic error between the derived glotal wave IF and the adjusted estimate for the particular phase. Figure 11 shows an example of LF adjustment in normal, chopped and forced phonations.

Since the LF parameterization does not model the glotal aspiration noise, the stochastic component present in the derivative glotal wave IF is not captured during the IF adjustment. However, perceptually, the lack of aspiration noise results in an unnatural speech quality and thus, a methodology for its extraction and parameterization within the JEAS framework has been developed.

The elimination of the noise of the wavelet is used to extract the noise of glotal aspiration from the estimation of the derived glotal wave IF. In a preferred embodiment, the technique of eliminating the noise of the wavelet that is used is the Analysis of Ripple Packages, which has been found to obtain more reliable suction noise estimates compared to other techniques used to identify and separate the components. newspapers and aperiodic of quasi-periodic signals, such as frequency transformation analysis or periodic prediction. The Ripple Packet Analysis is preferably performed at level 4 with the 7th order Daubechies wavelet, using virtual thresholding and the threshold evaluation criterion of the Stein Unexpected Risk Estimate. Figure 12 shows a typical noise elimination result: a) the original IF derived glottal wave and with the noise removed, b) noise estimation.

Once an estimate of the suction noise has been extracted, it must be parameterized. Studies on aspiration noise have shown that it is synchronous with the glottal wave and is likely to present bursts of noise at the glottal closure and often also at the glottal opening. Most models neglect the nature of the glottal opening impulse and approach it as a Gaussian noise modulated in synchronous amplitude with the tone, with greater energy near the instants of glottal closure. The amplitude of the noise burst is usually modulated using Rectangular, Hanning or Hamming windows. Sometimes a spectral shaping filter is included to take into account the average spectral density of the suction noise and the high-pass filtering introduced by the switching of the vocal tract and radiation filters. However, some models also neglect the spectral conformation filter since it has been found to be not significantly important. These approximations of Gaussian noise modulated in synchronous amplitude in the tone require the determination of the following parameters illustrated in Figure 13 of the suction noise component:

-Minimal noise threshold (Nf): the minimum suction noise threshold;

-Noise pulse amplitude (NPa): the amplitude modulation index of the noise impulse

- Noise impulse position (NPp): the position of the center of the noise impulse window in the glottal period

-Noise pulse width (NPw): the width of the noise impulse window.

Unfortunately, the automatic calculation of the above parameters from the estimated suction noise components is problematic in many cases. To avoid these errors, a different approach is followed in the present invention and, in particular, in the JEAS implementation. Although the approach to the aspiration component is still the Gaussian noise modulated in synchronous amplitude with the tone, an alternative function is used that does not require the estimation of Nf, Npa, NPp and NPw to modulate its amplitude: the LF wave. In fact, the shape of the LF wave follows most of the outgoing amplitude modulation characteristics of the glotal aspiration noise, that is, the magnitude of its amplitude increases during the open phase and is maximal in the glottal closure. If stationary Gaussian noise is modulated with an LF wave, the resulting signal will present the two probable bursts of aspiration noise near the glottal opening and the glottal closure as shown in Figure 14 (a) Gaussian noise source, b) wave LF, c) modulated Gaussian noise). According to informal hearing tests, this approach is comparable to the previously described window-based modeling techniques.

Thus, the present invention, the estimation of the aspiration noise obtained for a particular tone period during the JEAS analysis is parameterized as follows. First, the Gaussian noise of unit variance close to zero is modulated with the LF wave already set for that tone period. Then, its energy is adjusted to match the estimate of the suction noise (ANE). Since it has been found informally that the use of a spectral conformation filter does not make a perceptual difference, it is not included in the paremetrization. Figure 15 shows a diagram of the suction noise modeling approach that has been used.

1.3 Synthesis within the Joint Estimate Analysis Synthesis model

The synthesis is carried out following the JEAS Model of Figure 2 and applying the parameters estimated during the analysis. In theory, each frame K of the speech wave, which corresponds to a tone period in sound segments and with a fixed segment (in a particular example, a fixed segment of 10 milliseconds) in non-sound parts, can be generated by filtering the estimated sound and non-sound excitation signal e (n) with the vocal tract filter vt for that particular frame

image 1

where p is the order of the filter and Nk is the number of samples in the frame. The excitation signal is constructed well by adding the adjusted estimates of the LF and suction noise, 1f (n) and one (n), or simply by generating a source of Gaussian noise, gn (n), in the sound segments (V) and non-sound (U), respectively.

image 1

5 in practice, since the JEAS analysis described is done independently for each frame, the continuity of the parameters estimated between adjacent frames is not guaranteed, particularly within the sound segments. As a result, preceptual distortions occur when the parameters change too sharply from frame to frame. To reduce this problem, the trajectories of the parameters of the glottal sound source and the vocal tract are smoothed before resynthesis.

10 With respect to the vocal tract, the jointly estimated filter coefficients (α1… αp) are first converted to in-line Spectral Frequencies (LSF) due to their better interpolation properties. Then, each set of LSF LSFP coefficients (lsf1… lsfP) is averaged with those of the previous and next frames to obtain a smoother vocal tract filter estimate for synthesis

fifteen

image 1

As for the glottal source, a similar approach is followed. First, the adjusted T parameters of LF (Tp, Te, Ta, Tc) become R parameters (Rg, Rk, Ra) that are more suitable for interpolation since they are normalized with

20 regarding the fundamental period. Again, to soften its trajectories, each set of R parameters is averaged with those of the previous and next frames. The ANE trajectories of the aspiration noise energy are smoothed in the same way

25 Once the source parameters (Rg, Rk, Ra, ANE) have been averaged, they are used to recalculate glottal waves derived from smoothed LF If (n) and estimates of modulated suction noise in amplitude an (n) to use as excitation filter e (n) for resynthesis.

image 1

30 To resynthesize the speech wave, the overlay-addition scheme of equation 21 is used

image 1

where K is the total number of frames Wk is the Hamming window so that

image4

and SCk is the synthetic contribution of length MkSC = Nk-1 + Nk generated by

image 1

5

10

fifteen

twenty

25

30

35

40 so that the K-th synthesis frame of the Nk samples is obtained as

image 1

Figure 16 shows a schematic diagram of the overlay-addition synthesis scheme employed.

1.4 Modification of tone and time scale

Due to the explicit and independent modeling of the fundamental period and the interpolation possibilities of the vocal tract and glotal source parameters used, tone and time scale modifications are easily implemented within the JEAS framework.

Both tone and time scale transformations are based on a parameter path interpolation approach, in which the first task involves calculating the number of frames required in a particular segment to achieve the desired modifications. Once the modified number of frames is calculated, the contours of the frame size, the excitation and the trajectories of the vocal tract parameters are again sampled in the modified number of frames using, for example, cubic adjuster interpolation. Since the JEAS model is synchronous in tone, the frame sizes correspond to the periods of tone in the sound segments while they are fixed in the non-sound segments. Due to their better interpolation characteristics, LSF coefficients and R parameters are used during tone and time scale transformations to respectively represent the parameters of the vocal tract and the glottal source, in addition to the energies of the aspiration ANE and of Gaussian GNE noise.

The time scale modification is carried out by increasing or decreasing the number of frames per segment and consequently interpolating the parameter tracks. For example, to increase the duration of a sound segment of frames by 25%, the modified number of frames is calculated as mf = f + 0.25f. Then, the contour of the period of the tone of the point f is again sampled in the new set of uniformly separated mf points, as shown in Figure 17. In this way, the contour of the fundamental period is preserved, that is, intonation while its variation slows down. It is necessary to apply the same repetition of sampling to each LSF coefficient, parameter R and ANE tracks, to synthesize the modified speech over time. Non-sound segments can also be scaled over time using the procedure described. In this case, the paths of the excitation parameters for the new sampling are the GNE energies of the Gaussian noise source.

The tone can be altered by simply multiplying the contour of the fundamental period by a scaling factor.

For example, if a given tone period contour of f frames T = {T1, T2,…, Tf} is multiplied by 0.5, the speech synthesized with the modified contour T = 0.5 T = {T'1 , T'2, ..., T'f} could be perceived as having twice the original fundamental frequency. However, its duration could also be perceived as half of the original. The scaling of the fundamental periods includes the modification of the frame sizes and, as a consequence, the durations of the segments. For this reason, it is also necessary to modify the number of frames in a segment when the tone is scaled if its duration is to be maintained. The modified number of mf frames in the scaled fundamental periods whose duration approximates the original can be calculated as

image 1

where T is the original mean fundamental period, T ’is the scaled average fundamental period. Once mf has been calculated, the contour of the scaled tone period T ', the LSF coefficients, the R parameters and the ANE trajectories must be sampled again in the new number of frames before resynthesizing the modified speech wave in the tone.

5

10

fifteen

twenty

25

30

35

40

2. Voice conversion

In this section, the use of the JEAS glottal source parameterization and the continuous probabilistic linear transformations for the voice conversion is explored and the resulting JEAS Voice Conversion Frame performance is compared with that of a conventional Sinusoidal VC system (H. Ye and S. Young, High Quality Voice Morphing in Proc. ICASSP, volume 1, p. 1-9-12, 2004), referenced as PSHM. The first section details the speech model and the characteristics transformation techniques used in the JEAS VC implementation. The objective measurement of its spectral cover and the conversion performance of the voice source and the subjective evaluation of the recognition capacity and the quality of the converted output are presented below.

2.1 JEAS Voice Conversion

The following describes the procedures for transforming the spectral sheath and the glottal wave used in JEAS voice conversion. Although the conversion of the spectral cover is done in a manner similar to the implementation of known sinusoidal voice conversion, the main advantage of the JEAS model, that is, the parameterization of the voice source, allows the characteristics of the source to be also transform to match the target. As it offers the potential to improve fidelity in the target's identity, this also avoids the need for conventional residual prediction procedures. In addition, since JEAS parameterization does not imply a division of magnitude and phase of the spectrum, defects due to converted magnitude and phase mismatches do not occur and, thus, the use of additional techniques, such as the prediction of phase.

2.1.1 Spectral Cover Conversion

The jointly estimated JEAS vocal tract filter coefficients (α1… αp) are converted into LSF parameters by Bark scaling for the transformation of the JEAS spectral shells. First, the linear frequency response of the jointly estimated vocal tract filter is calculated. This is sampled again according to the Bark scale, using, for example, the already known cubic adjuster interpolation technique. The distorted omnipolar filter coefficients are then calculated by applying, for example, the conventional Levinson-Durbin algorithm to the autocorrelation sequence of the distorted power spectrum, then the filter coefficients are transformed into LSF for conversion.

A continuous probabilistic linear transformation function is used to convert the LSF spectral shells. Gaussian Mixture Models (GMK) are used to describe the vector spaces of the glottal characteristics of the source and target, classify them into M classes and prepare specific linear transformations for the classes. A weighted sum of the linear transformations is then used to convert each vector x of characteristics of the glottal source.

image 1

where X is the extended feature vector X = [x ’; 1] ’y _m is the interpolation weight of the transformation matrix Wm, its value is given by the probability of the vector x belonging to the class Cm.

image 1

where αm, μm and Σm are the weights, means and variances of the GMM components respectively and N () represents the Normal Distribution. Wm transformation matrices are estimated using parallel training data from the source and target and a minimum quadratic error criterion.

After conversion, the new LSF parameters are transformed into omnipolar filter coefficients and are sampled again for the linear scale before synthesis. Since the use of linear transformations widens the formants of converted speech, a perceptual post filter is applied to narrow the bandwidths of the formants, increase the spectral valleys and sharpen the peaks of the formants.

Figure 18 illustrates the JEAS versus the PSHM spectral decks, and it can be seen that the PSHM decks capture the spectral inclination but in JEAS it is encoded by glottal waves instead. In addition, although both procedures are aimed at representing the most important formants, there are small differences in their amplitudes, frequencies and / or bandwidths.

2.1.2 Conversion of the Glotal Wave

Previous work on the conversion of the glottal wave has shown that quantification of the glottal parameters is possible and able to capture differences in the quality of the voice source. For example, Childers and collaborators (Glottal source modeling for voice conversion, Speech Communication, 16: 127-138, 1995) created codes of 32 voice source polynomial parameter inputs from phrases produced with different voice qualities and aimed at get the conversion between modal, broken, choppy, rough, falsetto, whispering and hoarse phonations. However, experiments involving transformations between more similar phonations, that is, different modal speakers or alternative conversion procedures have not yet been explored. Nor has the use of glotal parameterization of LF been investigated.

The approximation of the mutation of the glottal wave adopted in the JEAS voice conversion uses Continuous Probabilistic Linear Transformations to map glottal LF parameters of different male and female modal speakers, which are the types of speakers most commonly used in voice conversion applications. .

Continuous probabilistic linear transformations have been selected to be the most robust and efficient approximation found to convert spectral decks. The limitations of the code-based conversion procedures for roof transformations, that is the discontinuities caused by the use of a discrete number of code entries, can also be interpolated for the modification of glottal waves. Thus, the use of the continuous probabilistic model and the transformations is expected to also achieve better glottal conversions.

The feature vectors used to convert the characteristics of the glottal source are derived from the parameters of the JEAS model linked to the voice source of each tone period, that is, the glottal excitation force Ee and the T parameters (( Tp, Te, Ta, Tc) obtained from the LF adjustment procedure and the aspiration noise estimation (ANE) energy used to adjust the modulated Gaussian noise in synchronous amplitude in modeled tone. T-dependent T for the conversion, are transformed into parameters R (Rg, Rk, Ra), resulting in the vector of characteristics of dimension five (Ee, Rg, Rk, Ra, ANE) for the conversion of the glottal wave. As shown in Figure 19, the described glotal conversion approach is capable of bringing the contours of the parameters of the source characteristic vector closer to the target which, as a consequence, also produces more converted glottals more similar to the target. In particular, Figure 19 shows the linear transformation of the LF Glottal Waves: a) converted, source and target LF glottal waves; b) source, target and converted trajectories of the glottal characteristics vector parameters (Ee, Rg, Rk, Ra, ANE).

2.2 Experiment: comparison between a conventional sinusoidal voice conversion procedure and a JEAS Voice Conversion procedure

An experiment is described below, in which a conventional sinusoidal voice conversion procedure (H. Ye and S. Young, High Quality Voice Morphing in Proc. ICASSP, volume 1, p. 1-9-12, 2004) , called PSHM is compared to the performance of the JEAS procedure. Both procedures have been evaluated in a conversion task based on the VOICES database (A. Kain, High Resolution voice transformation, thesis PhD, Oregon Health and Science University, 2001). Specifically designed for voice conversion purposes, the corpus is composed of three examples of 50 phonetically rich phrases spoken by 10 speakers (5 men, 5 women), that is, a total of 150 words per speaker. Speech data was recorded using a "mimicry" approach that resulted in a natural temporal alignment between identical phrases produced by different speakers and without taking into account the prosodic references of the speaker's identity to some extent. For each phrase, the instants of the glottal closure derived from the laryngographic signals were also supplied and have been used for synchronous tone analysis of both PSHM and JEAS. Four different voice conversion experiments have been investigated: transformations of man in man (MM), man in woman (MF), woman in man (FM) and woman in woman (FF). The first 120 sentences are used for training and the remaining 30 to check each speaker pair conversion.

Throughout the conversion experiments, LSF spectral vectors of order 30 have been used to train 8 spectral cover transformations between each pair of source and target speakers using parallel VOICES training data. This number has been selected for being able to achieve small spectral distortion ratios at the same time as it is generalized for the test data. Source-target vector pairs were obtained by applying forced alignment to mark the subphonic boundaries and using Dynamic Time Distortion to further constrain their time alignment. For residual and phase prediction, GMM target and codes of 40 classes and inputs have been constructed. Finally, glotal wave conversions have also been carried out using 8 linear transformations for each pair of speakers. Objective and subjective evaluations have been used to compare the performance of the two procedures.

5

10

fifteen

twenty

25

30

35

2.2.1 Objective Evaluation 2.2.1.1 Spectral Cover Conversion

Since linear spectral sheath transformations are currently applied to LSF vectors, their conversion performance can be easily evaluated by comparing the distances of source, target and converted LSF vectors. If the distance between two LSF vectors lsf1 and lsf2 is defined as

image 1

The following RLSF distortion ratio can be used as a measure of objectives of how to close the source vectors that have become the target

image 1

where lsfsrc- (t), lsftgt (t) and lsfconv (t) are respectively the source, target and converted LSF vectors and the sum is calculated over the time-aligned test data, where L is the total number of vectors of test after alignment. Note that a 100% distortion ratio corresponds to the distortion between the source and the target.

The RLSF ratios for the PSHM and JEAS spectral cover conversions on the VOICES test set have been calculated. Figure 20 shows the results obtained. Although the differences are small, JEAS has been found to work slightly better than PSHM with 3% lower LSF distortion ratios in all conversion tasks. This could be due to the fact that JEAS spectral covers do not encode spectral inclination information, which reduces LSF variations caused by inclination differences resulting in more accurate mappings.

2.2.1.2 Voice Source Conversion

Similar measures of objective distortion can also be used to evaluate the conversion of the characteristics of the voice source, that is, Residual Prediction and Conversion of the Glotal Wave in PSHM and JEAS implementations, respectively.

The Residual Prediction reintroduces the spectral details of the target not captured by the conversion of the spectral envelope, resulting in converted spectra closest to the target. The Glottal Wave Conversion, on the other hand, maps representations of the time domain of the glottal waves which in the frequency domain results in a better concordance of the glottal formants and the spectral inclinations of the converted spectra. Although the procedures are different, their spectral effect is similar, that is, they are aimed at reducing the differences between the spectra of converted and target speech.

One way to assess whether the voice source conversion procedures achieve the desired effect is to measure the logarithmic spectral distances (LSD) between the converted and target spectra before and after the voice source conversion. If the RMS logarithmic spectral distance between two spectra is defined as

image 1

where {ampk} are harmonic amplitudes again sampled from the S spectrum at points K on the Bark frequency scale (K has been set at 100 points in this work). Then, an RLSD distortion ratio similar to RLSF can be used to compare the converted log-target spectral distances with and without voice source conversion.

image 1

In which Sconv (t) and Sorig (t) are the spectra converted respectively with and without voice source conversion and Stgt (t) is the spectrum of the target. Thus, a ratio of 100% corresponds to the distortion between the spectral spectra converted without transformation of the voice source and the target spectra.

Figure 21 illustrates the RLSD ratios calculated for Residual Prediction and Glottal Wave Conversion over the test set. The results show that both voice source conversion techniques reduce distortions between spectra of converted and target speech. The Residual Prediction works slightly better, mainly because the algorithm is designed to produce residues that minimize the logarithmic spectral distance represented in RLSD. In contrast, the glottal wave conversion is aimed at minimizing the error of conversion of the glottal parameters on the training data and not the logarithmic spectral distance. However, both procedures are successful in bringing the converted spectra near the target.

2.2.2 Subjective Evaluation

To significantly compare the PSHM and JEAS voice conversion systems, a hearing test was conducted to check their performance in terms of recognition ability and quality. 12 subjects took part in the perceptual study that consisted of two parts.

The first part was an ABX test in which subjects converted with PSHM (A), converted with JEAS (B) and target (X) were presented to the subjects and were asked to select sample A or B that they found most similar to target X in terms of speaker identity. The characteristics of the spectral covers and the voice source were transformed with the procedures described above for each system, that is, prediction of conversion, residue and spectral cover phase were used for PSHM transformations and conversion of spectral and wave cover. Gotal for JEAS transformations. In addition, the prosody of the target was used to synthesize the converted phrases to normalize the differences in tone, duration and energy between the source and target speakers for perceptual comparison. There were 10 words of each type of conversion (MM, MF, FM, FF). The order of the samples in terms of conversion type and conversion system was random. An informal hearing of the transformed words using the PSHM and JEAS conversion systems revealed that it was often very difficult to choose convincingly among the systems in terms of speaker identity. For this reason, the subjects were also allowed to select an option “WITHOUT CLEAR PREFERENCE” when they found the selection difficult or did not have a clear preference towards one of the A or B speech samples presented.

Figure 22 shows the results of the ABX test. In all types of conversion, samples converted with JEAS were preferred instead of those converted with PSHM, but the difference in preference varies depending on the type of conversion, being for example almost the same for FM transformations. However, the option “WITHOUT CLEAR PREFERENCE” (NSP) has been selected almost as often as words converted with JEAS in general, which reveals that the subjects found it really difficult to distinguish between conversion systems in terms of speaker identity . Since the most important speaker identification signals, ie spectral covers, are transformed using the same procedure in the two conversion implementations, both systems are expected to function the same in terms of speaker recognition capability. In addition, the results obtained show that the techniques of Residual Prediction and Conversion of the Glotal Wave are also comparable in terms of transformation of the perceptual identity of the speaker.

The second hearing test was aimed at determining which system produces speech with a higher quality. Pairs of speech sounds converted with PSHM and JEAS were presented to the subjects and asked to select the one they thought had a better speech quality. The results are illustrated in Figure 23. There is a clear preference for words converted using the JEAS procedure, selected 75.7% on average, which resulted from the clearly distinguishable quality difference between samples transformed with PSHM and JEAS . The sounds obtained after the conversion with PSHM had a "noisy" quality caused by the phase discontinuities that still exist despite the Phase Prediction. Comparatively, words converted with JEAS sound more fluid. This difference in quality is thought to also slightly offset the preference for conversion with JEAS in the ABX trial.

Among others, the method and the voice conversion device of the present invention is applicable to frames that require voice clarity transformations. As one such application, its use to repair the characteristics of the abnormal voice source of tracheo-esophageal speech can be mentioned.

The invention is obviously not limited to the specific embodiments described herein, but includes any variation that may be considered by any person skilled in the art (for example, with respect to component selection, configuration, etc.), within the general scope of the invention as defined in the appended claims.

Claims (12)

1. A method for converting a speech signal from a source speaker into a converted voice signal, comprising the steps of: -a training stage, in which: - given a training database of parallel source and target data, for each tone period of said training database:

-

 model each tone period by means of a glottal wave and a glottal tract filter according to the Lu and Smith model, to obtain a set of Liljencrants - Fant LF parameters, said set of LF parameters comprises an excitation force parameter Ee and a set of parameters T, Tp, Te, Ta, Tc that model a glottal wave and a set of omnipolar vocal tract filter coefficients (α1… αp);

-

 converting said parameters T, Tp, Te, Ta, Tc into parameters Rg, Rk, Ra;

-

 converting said omnipolar vocal tract filter coefficients (α1… αp) into in-line spectral frequencies on the Bark scale lsf1… lsfp;

-define a glotal vector G to convert;

-

 defining a vocal tract vector of LSF to be converted, said vocal tract vector of LSF comprises said spectral frequencies in line on the Bark scale lsf1… lsfp;

-

 apply the elimination of the noise of the wavelet to obtain an estimate of the noise of glotal aspiration;

-from the set of LSF vocal tract vectors obtained for each tone period of said training database, estimate a continuous probabilistic linear transformation function of the vocal tract using the minimum quadratic error criterion; The procedure is characterized in that said modeling step further comprises the steps of: - modeling said estimation of the aspiration noise by modulating the Gaussian noise of the unit variance close to zero with the aforementioned modeled glotal wave and adjusting its ANE energy to coincide with the said estimation of the aspiration noise; said glotal vector G to be converted comprises said excitation force parameter Ee, said parameters R, Rg, Rk, Ra and said ANE energy of the aspiration noise estimate, The procedure also includes:

-

 a conversion stage in which a given test speech wave is modeled and transformed into a set of parameters Ee´, Rg´, Rk, Ra´, ANE´, LSF´;

-

 a synthesis stage in which a converted speech wave is synthesized from said set of converted parameters Ee´, Rg´, Rk, Ra´, ANE´, LSF´.

2. The method according to claim 1, wherein said training step further comprises: - from the set of Glottal vectors G obtained for each tone period of said training database, estimate a function of continuous probabilistic linear transformation of the glottal wave using the minimum quadratic error criterion. 3. The method according to claim 1 or 2, wherein said step of modeling each tone period by means of a glottal wave and a vocal tract filter according to the Lu and Smith model, comprises the Steps of: -model the glottal wave using the Rosenberg model - Klatt; - using convex optimization to obtain a set of Rosenberg glottal parameters - Klatt and the coefficients of the omnipolar vocal tract filter α1… αp in which said step of using convex optimization comprises an adaptive pre-emphasis step to estimate and eliminate the contribution of spectral inclination filter of the speech wave before convex optimization. 4. The method according to claim 3, wherein said step of modeling each tone period by means of a glottal wave and a vocal tract filter according to the Lu and Smith model, further comprises the steps of: - obtaining a glotal wave derived by inverse filtering of said tone period using said omnipolar vocal tract filter coefficients α1… αp; - adjusting said set of LF parameters to said glotal wave derived from inverse filtering by direct estimation and constrained nonlinear optimization. 5. The method according to any preceding claim, wherein said conversion stage comprises, for each tone period of said test speech wave: - obtaining a glotal vector G to be converted, said glotal vector comprising an excitation force parameter Ee, a set of parameters R, Rg, Rk, Ra and the ANE energy of said aspiration noise estimate; - obtaining the LSF vocal tract vector to be converted, said LSF vocal tract vector comprises a set of in-line spectral frequencies on the Bark scale lsf1… lsfp; - applying said continuous probabilistic linear transformation function of the vocal tract estimated during the training stage to obtain a vector of converted vocal tract LSF parameters; -transform said glotal vector G using said continuous probabilistic linear transformation function of the glotal wave estimated during the training stage, thereby obtaining a converted glotal vector G 'comprising a set of converted parameters Ee´, Rg´, Rk, Ra´, ANE´, LSF´; 6. The method according to claim 5, wherein said steps of obtaining a glotal vector G to be converted and a vocal tract vector of LSF to be converted further comprise the steps of:

-

 model each tone period by means of a glottal wave and a vocal tract filter according to the Lu and Smith model, to obtain a set of parameters LF, said set of parameters LF comprising an excitation force parameter Ee and a set of parameters T, Tp, Te, Ta, Tc that model the glottal wave, and a set of omnipolar vocal tract filter coefficients α1… αp;

-

 converting said omnipolar vocal tract filter coefficients into in-line spectral frequencies on the Bark scale lsf1… lsfp;

-

 converting said parameters T into parameters R, Rg, Rk, Ra;

-

define a glotal vector G to convert;

-

Define an LSF vocal tract vector to convert.

7.

The method according to claim 5 or 6, wherein said conversion step further comprises a post-filtering step of said converted vector of LSF 'vocal tract parameters.

8.

The method according to any preceding claim, wherein said synthesis step, in which said converted speech wave is synthesized from said set of converted parameters Ee´, Rg´, Rk, Ra´, ANE´, LSF ´; Understand the steps of:

- interpolate the paths of said converted parameters Rg´, Rk, Ra´, ANE´, LSF´; of each tone period, thus obtaining a set of interpolated parameters Rg ", Rk", Ra ", ANE", LSF "; comprising the parameters Rg ", Rk", Ra ", interpolated energy (ANE) and an interpolated vocal tract vector of LSF";

-

converting said interpolated vocal tract vector of LSF "into a vector of omnipolar filter coefficients A";

-

converting said interpolated R parameters Rg ", Rk", Ra "into interpolated T parameters Tp", Te ", Ta", Tc ";

- for each frame of said test voice wave, generate an excitation signal ek (n), in which k indicates the kinth frame. 9. The method according to claim 8, wherein said step of generating an excitation signal comprises, for each of said frames: -if that frame is sound:

-

 from said interpolated parameters Tp ", Te", Ta ", Tc" and said excitation force parameter Ee

generate an interpolated glottal wave lfk (n);

-

 from said parameter of interpolated energy ANE ”, generate an interpolated suction noise ank (n);

-

 generating said sound excitation signal ek (n) by adding said interpolated glottal wave lfk (n) and said interpolated aspiration noise ank (n);

5 - if said frame is not sound: -generate said non-sound excitation signal ek (n) from a Gaussian noise source gnk (n). 10. The method according to claim 8 or 9 wherein said synthesis step further comprises: - generating a synthetic contribution of each frame by filtering said excitation signal ek (n) with said vector of 10 omnipolar filter coefficients A ”; - multiply this synthetic contribution by a Hamming window, superimpose and add, to generate the converted speech signal. 11. A method applicable to voice quality transformations, such as tracheoesophageal speech repair, comprising the procedural steps of any preceding claim. A device comprising means adapted to carry out the steps of the procedure of any preceding claim. 13. A computer program code medium adapted to perform the steps of the procedure of any claim 1-11, when said program is executed on a computer, a digital signal processor, an array of field-programmable gates, a specific integrated circuit for the application, a microprocessor, a 20 microcontroller or any other form of programmable hardware.