JPH01998A - How to normalize spectrograms - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed Description of the Invention] [Industrial Field of Application] This invention relates to a spectrogram normalization method, and in particular, it normalizes spectrograms between different speakers using vector quantization, and is useful for speaker-independent recognition. This paper relates to a spectrogram normalization method that can be applied to speaker adaptation and property conversion techniques.

[Prior art and problems to be solved by the invention] Automatic translation telephones use voice as input;
The voice is the voice of an unspecified speaker, and it is necessary to accurately recognize such voice of an unspecified speaker. One method for speaker-independent recognition is to normalize spectrograms between speakers of different speakers, but conventional methods of normalizing spectrograms between speakers of different speakers mainly involve normalization of vowel intervals. The only methods available were deterministic spectral frequency changes.

Therefore, a method of normalizing spectopigrams between different speakers using vector quantization may be considered. However,
In conventional vector quantization, the spectral distortion measure used in vector quantization has been improved in order to suppress increases in calculation amount and memory and improve recognition performance. and,
Composite spectral distortion measures that combine various features have been used, but this method mixes various types of features in the spectral distortion measures and uses the dependencies between them as constraint conditions to improve recognition performance. The meaning lies in mapping features into space. However, this method has the following problems.

■ In order for the dependencies between features to have statistical validity in the vector quantization codebook, a large number of learning samples and an enormous amount of calculation time are required.

■ In terms of codebook size, the codebook size required for each feature can be reduced by using the dependencies between features as a constraint. However, the overall codebook size is still the product of the codebook sizes required for each feature, resulting in a very large size and requiring a huge amount of memory.

■ When a vector quantization codebook is generated using a composite spectral distortion measure, the ability to reproduce the spectrum decreases due to the correlation between various features.

Therefore, the main purpose of this invention is to express spectra for each individual with finite vectors using vector quantization, and then to find the correspondence between the vectors between different speakers. An object of the present invention is to provide a normalization method for spectrograms that can be normalized.

[Means for solving the problem] The present invention is a spectrogram normalization method that digitizes speech, extracts a spectrogram as a feature of the speech, and normalizes the extracted spectrogram between different speakers. After vector quantizing speech, the vector quantization codebook is associated with different speakers, and the spectrogram is normalized based on this association. 1 [Operation] The spectrogram normalization method according to the present invention vector quantizes the speech, expresses the spectrogram with a finite vector for each individual, and then calculates the correspondence between the vectors between different speakers, thereby creating a codebook. Since the size is the sum of the codebook sizes required for each feature, the overall codebook size can be reduced.

[Embodiments of the Invention] Examples of the invention will be described in more detail below with reference to the drawings.

FIG. 1 is a schematic block diagram of a speech recognition device to which the present invention is applied.

In FIG. 1, the speech recognition device is composed of an amplifier 1, a low-pass filter 2, an A/D converter 3, and a processing device 4. The amplifier 1 is for amplifying an input audio signal, and the low-pass filter 2 is for removing aliasing noise from the amplified audio signal. The A/D converter 3 converts the audio signal into a 16-bit digital signal using a 12kHz sampling signal. The processing device 5 includes a computer 5, a magnetic disk 6, a terminal 7, and a printer 8. The computer 5 performs voice recognition based on the voice digital signal input from the A/D converter 3.

FIG. 2 is a flow diagram showing the overall flow from audio input to output of a normalized spectrogram in one embodiment of the present invention.

Next, the operation of one embodiment of the present invention will be described with reference to FIGS. 1 to 3. The input audio signal is amplified by the amplifier 1, and after aliasing noise is removed by the low-pass filter 2, the input audio signal is input to the A/D converter 3 in step SPl shown in FIG. 2 (abbreviated as SP in the figure). Converts the audio signal into a 16-bit digital signal.

In step SP2, the computer 5 of the processing device 4 extracts features of the audio converted into a digital signal. In this feature extraction, for example, linear analysis (LPG analysis)
This is done using methods such as

In step SP3, it is determined whether or not a codebook is to be generated. If a codebook is to be generated, in step SP4, based on the features of the extracted speech,
Codebook generation is performed. For example, the LBG algorithm is used to generate this codebook, and each feature is generated and stored in a separate codebook on the magnetic disk 6 in step SP5. Regarding the LBG algorithm, Linde and Buz
o, Gray; An algorithm for
Vector Quantization D
esLgn'IEEE C0M-28 (1980-0
1) is described in detail.

When performing quantization, it is determined in step SP3 that a codebook is not generated, and the speech features obtained in step SP2 described above are subjected to separate vector quantization with reference to a separate codebook in step SP6.

Then, in step SP7, it is determined whether or not the learning is of a transformation vector. If it is learning of a transformation vector, in step SP8, the code string for each feature generated by separate vector quantization is DP (Dynamic Programming:
dynamic programming) is matched. This learning standard pattern series is registered in advance on the magnetic disk 6 in step SP9. In step 5P10, a transformation vector is generated using a histogram of vector correspondences resulting from DP matching.

This conversion vector is registered in the magnetic disk 6 in step 5P11.

In step SP7, when it is determined that the transformation vector is not learning, that is, when it is determined that normalization is being performed, the code string for each feature generated by separate vector quantization is transferred to step 5P11 in step 5P12. The already stored transformation vectors are used to replace each frame, and a normalized spectrogram is generated and output.

Fig. 3 is a flow diagram for explaining the operation of spectrogram normalization using vector quantization, Fig. 4 is a flow diagram for explaining the operation of separate vector quantization, and Fig. 5 is a flow diagram for explaining the operation of spectrogram normalization using vector quantization. FIG. 6 is a flowchart for explaining a vector learning algorithm, FIG. 6 is a spectrogram normalization algorithm, and FIG. 7 is a flowchart for explaining a matching method.

Next, spectrogram normalization using vector quantization will be explained with reference to FIG.

Spectrogram normalization using vector quantization in this invention consists of two main functions. 1
The first is vector quantization in step 5P23. This vector quantization is separate vector quantization in which vector quantization is performed separately for each type of feature, and in step 5P22, separate codebooks are generated for each feature.

The second step is the conversion (normalization) of the spectrum in step 5P24, and in step 5P24, the vectors are associated by having the unknown speaker speak the qt word for learning. Here, we obtain a histogram of the correspondence obtained for all training words, use this as a weight, express the feature vector of the unknown speaker's codebook as a linear combination of the feature vectors of the standard speaker's codebook, and use this as a conversion code. It is stored as a book in step 5P25, and during normalization, the input spectrum is replaced with the conversion codebook for each input to normalize the spectrum.

Here, separate vector quantization will be explained in detail. In this invention, audio is divided into power and spectral information (
It is divided into two types of features (autocorrelation coefficient and LPC cepstrum coefficient), and vector quantization is performed separately on each of them. However, since the power is a scalar, it is non-uniform scalar quantization. In FIG. 4, in step 5P31, the audio signal converted to a 16-bit digital signal is subjected to LPG analysis using 14th order autocorrelation analysis, and the power, autocorrelation coefficient, and LPC, which are the characteristics of the input audio, are analyzed. Extract cepstral coefficients. In step 5P32, it is determined whether or not power codebook generation is being performed, and if power codebook generation is being performed, the power of the input voice is scalar quantized in step 5P33. In the scalar quantization, a power codebook is generated in step 5P33 using a non-uniform quantization method, and the generated power codebook is stored on the magnetic disk 6 in step 5P34.

When a power codebook is not being generated, that is, during quantization, the power codebook in step 5P34 is used, quantization is performed in step 5P35, and a code string related to power is output.

On the other hand, if it is determined in step 5P36 that the codebook is to be generated for LPC correlation coefficients and LPC cepstral coefficients, then in step 5P37, LBG
A codebook is generated based on the WLRR degree by the algorithm, and the generated codebook is stored on the magnetic disk 6 in step 5P38. Here, WL
The R scale is a scale that emphasizes the features of speech and shows high performance in word speech recognition, according to Murayama.

Shikano, “LPG spectrum matching scale scale with peak weighting” (IEICE paper) J64-A5
(198-05).

Note that when a codebook of LPG correlation coefficients and LPC cepstrum coefficients is not generated, that is, during quantization, vectors are generated in step 5P39 using the spectral codebook in step SP3g for the autocorrelation coefficients and LPC cepstrum coefficients of input speech. Performs quantization and outputs a code string related to spectral information.

Here, the spectral distortion measure used for codebook generation and quantization is as follows.

d −P/P' + P' /P-2...(1
)ower d -Σ (C(n)-C' (n)) (R(
n)-R' (n))spectrus...(2) where d is the distortion measure of the power term, power dSpOetrtJl is the spectral distortion measure, and R
(n) is the n-th autocorrelation coefficient of the codebook, R' (n) is the n-th autocorrelation coefficient of human power, and C(
n) is the n-th order LPC cepstral coefficient of the codebook, and C' (n) is the n-th order LPC cepstral coefficient of the human power.

Next, referring to FIG. 5, step 5 shown in FIG.
P24. The spectral normalization and conversion codebook generation in step 5P25 will be described in detail. First, when generating the conversion code book,
Have an unknown speaker say the learning words. This input voice is subjected to separate vector quantization in step 5P41 using the codebook already stored in step 5P42. In step 5P43, the quantized code string is used as the learning standard for the same words of standard speakers already stored in step 5P44.
The vectors are matched using the same learning word uttered by an unknown speaker and a standard speaker using D P 7 tsching using the ble 5plit method. Then, the correspondence is obtained for all the learning words and stored in the form of a histogram.

In step 5P45, the obtained histogram is used to express the feature vector of the unknown speaker as a weighted sum weighted in step 5P46 with the histogram of the association of feature vectors in the codebook of the standard speaker stored. This weighted sum can be expressed by the following formula.

a' (g) - Σb (k) Elk/i hn (k)
n k: Code number of the standard speaker's codebook Mouth: Code number of the unknown speaker's codebook a': Conversion vector from unknown speaker to standard speaker b (k): Characteristics of the standard speaker's codebook Vector h (k): histogram of the standard speaker's code for the unknown speaker's code n found by the correspondence by DP matching.Next, in step 5P48, the unknown speaker's two 1 b. Swap the Tsuku, step 5
P43. Repeat steps 5P45, 5P47 and 5P48. Repeat this process a certain number of times or until DP short distance convergence for all learning words is reached, step 5P47.
When it is determined that convergence has been achieved in , the final conversion vector from the unknown speaker to the standard speaker is determined.

Next, normalization of the spectrum will be explained with reference to FIG. In step 5P51, the input speech of the unknown speaker is subjected to separate vector quantization using the codebook. Here, the unknown speaker's codebook is
It is stored in advance in P52. Then, in step 5P53, the codebook of the unknown speaker is replaced based on the conversion vector from the unknown speaker to the standard speaker obtained in step 5P54, framewise spectrum replacement is performed, and a normalized spectrogram is output. .

Next, with reference to FIG. 7, a matching operation for determining correspondence will be described. Matching is Double
This is carried out using the 5-plit method. In step 5P61, a code string generated by vector quantizing power and spectrum separately by separate vector quantization is matched with a standard pattern stored as a code string.

As the standard pattern, in step 5P62, a standard pattern of power and spectrum coded by separate vector quantization is stored in advance. and,
In the matching in step 5P61, the distance between codes is determined by creating a distance matrix in advance in step 5P63 and performing this table search. In this way, the correspondence between the vector of the human voice and the standard pattern obtained by sequential matching with the standard pattern is output to the histogram generation section in step 5P64. Then, using the histogram obtained by the histogram generation unit as a weight, the unknown speaker's feature vector is expressed as a linear combination of the standard speaker's feature vectors, and is used as a conversion vector.

Next, the matching method will be explained in detail.

In conventional matching, both the input and the standard pattern are one feature string or code string, but in separate vector quantization, they are generally composed of a plurality of code strings.

In this invention, a method of matching two sequences, a power code sequence and a spectrum code sequence, will be exemplified and explained. There is a PWLR measure as a distance measure when both power and spectrum information are considered. This is shown by the following equation.

dPVLR-Σ(C(n)-C'(n))Q? (n)
-R'(n))+a(P/P'+P'/P-
2) , -(3) a = 0.01 In matching code strings using the conventional Double 5plit method, as mentioned above, all spaces are vector quantized and represented by a finite number of points. Then, the distances between all representative points are calculated in advance and stored in a distance matrix. Therefore, d, WLR(1,j) −oL(A(+),B(j))
DL(A(1), B(j)) one Σ(CK(n)-c, (n))(RK(n)-RL
(n))+ a-(PK/PL+ P, /PK-2)
A (i) is the code number of the i-th frame of the input audio B (j) is the code number of the j-th frame of the standard pattern DL (K, L) is the code number that represents the distance between L from the distance matrix in the code. L is the code number of A (j) and B (j).However, in separate vector quantization, there are two sequences, so the distance is determined as follows.

d[p][VLR]”J) =DL (A (+), B (j)
) Sp8eL 5pect 5pect+
a' DL, o, 8. (Apo, , (1), B
,,,e, (j)) where DL (A (1), B (j))
spectrum 5pect 5pect-Σ(
CK(n)-C,(n)>(RK(n)-R,(n))
DL (A (1), B (j))
power power power
r” PK me/PLl+PL me/PK, -2,
L is code 5 of A (i), B (j)
pcct 5pect ', L' in the numbers are A (i), B (j
) is the power code number.

This is obtained by separately encoding the first and second terms of the PWLR measure, calculating distances, and finding the sum. Using this local distance measure, the distance is determined by DP matching.

In the manner described above, a normalization method using vector quantization with very high performance can be achieved.

[Effects of the Invention] As described above, according to the present invention, a spectrogram is extracted after vector quantization of speech, and correspondence is made between different speakers using the vector quantization codebook.
Since the spectrogram is normalized based on this correspondence, the dependent term of each feature can be ignored, the number of learning samples can be reduced, and the amount of calculation can be reduced. However, by separating, a separate vector quantization system is constructed, which slightly increases the amount of calculation, but since there are few learning samples, the amount of calculation can be sufficiently reduced. In separate vector quantization, the codebook size is the sum of the codebook sizes required for each feature, so
The overall codebook size can be drastically reduced.

Moreover, since the dependent terms of each feature are ignored, optimal quantization can be performed within the features of the codebook, and therefore the spectrogram can be faithfully reproduced.

[Brief explanation of the drawing]

FIG. 1 is a schematic block diagram of a speech recognition device to which an embodiment of the present invention is applied. FIG. 2 is a flow diagram showing the overall processing flow from inputting audio to normalization. Third
The figure is a flow diagram for explaining the operation of spectrogram normalization using vector quantization. FIG. 4 is a flow diagram for explaining the operation of separate vector quantization. FIG. 5 is a flow diagram for explaining the transformation vector learning algorithm. FIG. 6 is a flow diagram showing an algorithm for spectrogram normalization. FIG. 7 is a flow diagram for explaining the matching operation. In the figure, 1 is an amplifier, 1 is a low-pass filter, 3 is an A/D converter, 4 is a processing device, and 5 is a computer. Patent applicant A.T.R. Automatic Translation Telephone Research Institute

Claims (5)

[Claims] (1) In the spectrogram normalization method, which digitizes speech, extracts a spectrogram as a feature of the speech, and normalizes the extracted spectrogram between speakers of different speakers, the speech is vector quantized, and then vector quantization is performed. A spectrogram normalization method that maps codebooks between different speakers and normalizes spectrograms based on this mapping. (2) The scope of the claim is that, as the method for making the correspondence between the different speakers, the correspondence between the vectors of the codebooks of the different speakers is obtained by learning certain learning words, and normalization is performed based on this. The spectrogram normalization method described in Section 1. (3) By creating a matching histogram using dynamic programming during the learning, and rewriting the unknown speaker's feature vector with a linear combination of the reference speaker's feature vectors using this as a weight, The spectrogram normalization method according to claim 2, wherein the spectrogram is normalized. (4) When matching the codebooks between the different speakers,
Claim 3, wherein the learning of correspondence is performed by constraining the coding path using the sum of inter-code distances of various features as the local distance of matching by dynamic programming. Spectrogram normalization method. (5) Performing separate vector quantization using two types of voice characteristics: power and autocorrelation coefficient;
By learning a certain number of learning words, a histogram of the correspondence is created, and normalization is performed by replacing the feature vectors of each codebook of the unknown speaker with a linear combination of the feature vectors of the reference speaker with the histogram as a weight. A spectrogram normalization method according to claim 3, wherein the spectrogram normalization method is performed.