JPH02220099A - Word voice recognition device - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a word speech recognition device that has improved recognition ability for an unspecified number of speakers.

[Conventional technology]

Conventionally, in the recognition of word sounds uttered by an unspecified number of speakers, methods have been used to model and absorb speech fluctuations using a probabilistic model, and to prepare standard patterns that correspond to the utterances of multiple speakers for each word. A method is used in which the standard pattern closest to the input audio is selected.
In either case, it is difficult to obtain high recognition performance for speakers who deviate from the standard.For this reason, methods of adapting the recognition device to the speaker have been attempted (for example, in the literature, Nakamura and Shikagari: Spectrogram normalization using separate vector quantization, Journal of the Acoustical Society of Japan, 44.8
．． In order to do this, it was necessary to utter a large number of predetermined words and sentences for learning each time the speaker changed, which was extremely inconvenient. Adaptation methods using unknown input speech to be recognized rather than predetermined words or sentences have been attempted (for example, in the literature, Sugiyama: Various methods for unsupervised speaker adaptation of vowels). Comparison of IEICE Transactions, 70-D, 5. P, 958.19
87), there was a problem that high adaptation performance could not be obtained.

This invention was made in view of the above, and its purpose is to use any voice of a speaker, calculate the correspondence between the voice of the speaker and a standard voice, and based on this, By automatically adapting the speaker's voice to a standard voice, or by adapting the standard voice to the voice of the speaker, high recognition performance can be obtained for the voice of an unspecified speaker. An object of the present invention is to provide a speech recognition device that does the following.

[Means to solve the problem]

In order to achieve the above object, the present invention provides a parameter vector calculation means for calculating a parameter vector indicating temporal changes in the frequency spectrum and power of a word audio signal; codebook creation means for clustering a large number of parameter vectors calculated from audio signals of l! and storing a plurality of representative vector values in a codebook; A word dictionary creation means for expressing the word "Umugi" as one or more time series of vector values stored in a codebook and storing the expression in a word dictionary, a large number of human parameter vectors calculated from input audio signals, and a code. means for hierarchically dividing the element vectors of the codebook into clusters; means for determining segmental movement direction vectors for adapting input parameter vectors to the codebook for each divided cluster; means for adapting all input parameter vectors using a weighted average of segmented movement direction vectors; means for calculating distances between these adapted input parameter vectors and each element of the codebook; The object of the present invention is to have a means for calculating the distance between the input speech and each recognition target vocabulary by dynamic programming or hidden Markov model using the distance and each word dictionary. Alternatively, instead of adapting the input parameter vector to the codebook, the codebook is adapted to the input parameters and the distance between the adapted codebook element and the input parameter vector is calculated.

The conventional technology includes means for clustering parameter vectors and codebooks using speech uttering arbitrary words whose utterance content is not limited, and adapting input speech parameter vectors or codebooks based on this clustering. Things are different.

〔Example〕

Hereinafter, the present invention will be explained in detail using the drawings.

FIG. 1 is a diagram showing a circuit block of a word speech recognition device according to the present invention.

In the figure, 1 is connected to, for example, a microphone,
This is an audio input terminal for inputting a word audio signal (hereinafter simply referred to as "audio signal") and supplying it to the next stage audio section detection circuit 2.

The voice section detection circuit 2 calculates the power for each set time (for example, 10 m5) and detects silence based on the power since the voice signal generally includes a silent part mainly consisting of noise and an actual voice part that is not. For example, a method of determining a part where the absolute value of the power for each set time exceeds a predetermined level is a sound part, and a method for determining a part where the absolute value of the power for each set time exceeds a predetermined level is a circuit that distinguishes between the part and the audio part. Various known methods can be used, such as a method in which if the level exceeds the level for a predetermined period of time, it is determined that this is a voice part. The voice section detection circuit 2 is a parameter vector calculation circuit 3
It is connected to the. Among the power values calculated by the speech section detection circuit 2, the values of the sections determined to be speech sections are input to the parameter vector calculation circuit 3.

The parameter vector calculation circuit 3 is the voice section detection circuit 2.
This is a circuit that converts the signal of the audio part detected in the above process into a parameter vector series that indicates temporal changes in frequency spectrum and power. For this conversion process, a known method is used, for example, a method of converting an audio signal into a linear predictive cepstrum (hereinafter simply referred to as r-cepstrum) time series. This is done by first converting the audio signal into a time series of linear prediction coefficients, and then converting this into a cepstrum.

Overview of the conversion process from audio signals to linear prediction coefficients (e.g., literature, Kimoto-Sai H: Estimation of audio spectral density and formant frequency using statistical methods, Journal of the Institute of Electronics and Communication Engineers,
53-A, 1. P, 35.1970) is as follows. Basically, it is first passed through a low-pass filter, then sampled and quantized, and then for a certain period of time (for example,
ms), a short section waveform is cut out, multiplied by a Hamming window, etc., and a correlation coefficient is calculated by calculating the sum of products. From the correlation coefficients, linear prediction coefficients can be easily extracted by solving algebraic equations through repeated arithmetic processing.

Details of the conversion process from linear prediction coefficients to cepstrum 11 (
For example, see the literature, Saito and Chunichi: Fundamentals of Speech Information Processing, Ohmsha, Chapter 7, P, 102, 1981), but it can be processed by calculating a recursive formula using linear prediction coefficients. The cepstrum obtained by this conversion process is
A codebook creation circuit 4 which is combined with the power input from the speech interval detection circuit 2 to form a parameter vector and which is connected to the output stage of the parameter vector calculation circuit 3.
is supplied to

The codebook creation circuit 4 clusters a large number of parameter vectors calculated from audio signals of a large number of words uttered by one or more standard speakers. This clustering is to group a large number of parameter vector sets into a predetermined number of representative vector value sets.

For example, 10 meters from 100 different words uttered by 4 speakers! Assuming that an 11-dimensional vector (10-dimensional cepstrum and power) is extracted for each word, and the average length of word speech is 500 s + s, a total of 5
0.times.4.times.100-20.000 types of 11-dimensional vectors are given. For example, to summarize this into 1,024 types of representative 11-dimensional vectors, a known method (Reference, !, Linde + A, Buzo and R
,M.Gray: An algorithm
lv for vector quanti-z
ation+ 111111 TraIIs,
Commun,, vol, C0M-28+
pp.

84-95.1980) can be used. In this method, vectors that are am are collectively represented by one average value, and when all the original 20.006 types of vectors are replaced with the closest one among the 1,024 types of representative values, , the representative value is determined so that the error due to replacement is minimized as a whole. The 1,024 kinds of 11-dimensional vector representative values determined in this manner are stored in the codebook storage unit 5 as codebook elements. The output terminal of the codebook storage section 5 and the output terminal of the parameter vector calculation circuit 3 are connected to a word dictionary creation circuit 6.

The word dictionary creation circuit 6 converts the parameter vector series calculated from the audio signals of all recognition target words uttered by one or more speakers into a time series of codebook elements stored in the codebook storage unit 5. It is converted into . This method calculates the distance between the parameter vector of each word, for example every 10×3, and all codebook elements, selects the codebook element with the smallest distance, and indicates the codebook element obtained in this way. This is done by converting it into a time series of numbers.

One or more number sequences for each word thus determined are stored in the word dictionary storage section 7 as a word dictionary.

The output terminal of the parameter vector calculation circuit 3 is connected to the learning vector storage section 8. Learning vector storage unit 8
is a large number of parameter vectors (hereinafter referred to as
It stores a set of parameters (referred to as "learning parameter vectors").

The output terminal of the codebook storage section 5 and the output terminal of the learning vector storage section 8 are connected to the hierarchical clustering circuit 9. The zero hierarchical clustering circuit 9 is stored in the codebook storage section 5. All codebook element vectors (hereinafter referred to as "codebook vectors") and all learning parameter vectors stored in the learning vector storage unit 8 are hierarchically clustered. In this method, first, the number of clusters is set to 1, and the average vector of all codebook vectors (hereinafter referred to as "codebook centroid") is calculated. At the same time, the average vector of all learning parameter vectors (hereinafter,
(referred to as the “learning speech centroid”). Both centroids are supplied to a segment movement vector calculation circuit IO connected to the floor 1 clustering circuit 9.

The segmented movement vector calculation circuit 10 is a circuit that calculates segmented movement vectors by subtracting the learning speech centroid from the codebook centroid.

The segmented movement vectors thus obtained are supplied to the learning speech adaptation circuit 11 together with the contents of the learning vector storage section 8.

The learning speech adaptation circuit 11 performs an adaptation process that brings the learning parameter vector closer to the codebook vector by adding a segmentation movement vector to each learning parameter vector. The learning parameter vectors subjected to the adaptation process are stored in the learning vector storage section 8 and then supplied to the hierarchical clustering circuit 9 again. At this time, the initial values of the learning parameter vectors stored in the learning vector storage section 8 are not deleted, and the vectors subjected to the adaptation process are stored separately.

The hierarchical clustering circuit 9 doubles the number of clusters, clusters all codebook vectors, and calculates the average vector of each cluster, that is, the codebook centroid.
Next, for each learning parameter vector, I! with all codebook centroids. The distance is calculated and associated with the codebook centroid with the smallest distance. For all learning parameter vectors, the average vector of vectors associated with the same codebook centroid, that is, the learning speech centroid is calculated. All codebook centroids and all corresponding learning speech centroids are fed to a partitioned movement vector calculation circuit 10 which is connected to a hierarchical clustering circuit 9.

The segmental movement vector calculation circuit 10 calculates the segmental movement vector corresponding to each centroid by subtracting the corresponding learning speech centroid from each codebook centroid. Each segmental movement vector obtained in this way is supplied to the learning speech adaptation circuit 11 together with the contents of the learning vector storage section 8.

The learning speech adaptation circuit 11 performs adaptation to bring the learning parameter vector closer to the codebook vector by adding a weighted average value of segmented movement vectors (hereinafter referred to as "adaptation vector") to each learning parameter vector. processing is performed.

The adaptation vector is calculated as follows.

Here, a is the adaptation vector to be added to the first learning parameter vector, p is the m-th segmental movement vector, and M is the total number of segmental movement vectors (number of clusters) %W
111 is a weighting coefficient, W1+e-1/l I
C! -u-1l (2)
It is calculated. Here, cl is the first learning parameter vector, U is the m-th learning speech centroid, and II is the calculation of the norm (size) of the vector. The learning parameter vectors subjected to the adaptation process are stored in the learning vector storage section 8 and then supplied to the hierarchical clustering circuit 9 again.

In the hierarchical clustering circuit 9, the number of clusters is doubled again, and the hierarchical clustering circuit 9, the segmented movement vector extraction circuit 10, and the learning speech adaptation circuit 11,
Adaptation processing of the learning parameter vector is performed in the same manner as above. This series of processing is repeated until the number of clusters reaches a certain predetermined number or the magnitude of the adaptation vector becomes smaller than a certain predetermined value. When the second iterative process is completed, the first ItlI(
i! I is supplied to the segmented movement vector calculation circuit 10,
The 0-segment movement vector is calculated as the difference between the adapted learning parameter vector of the final state and each initial value. 12.

Next, an unknown audio signal to be recognized is input to the audio input terminal 1. This audio signal is supplied to the audio section detection circuit 2, and the actual audio section is determined. The audio signal of the section determined to be a speech section is supplied to the parameter vector calculation circuit 3, where it is converted into a parameter vector series. The parameter vector series is input to the input adaptation circuit 12.
is supplied to The human adaptation circuit 12 performs an adaptation process that brings the parameter vector closer to the codebook vector by adding the adaptation vector to each vector in the parameter vector series. The adaptation vector is calculated as follows using the segmented movement vector already supplied to the circuit.

Here, bj is the adaptation vector to be added to the j-th vector in the parameter vector series, q, is the n-th segmental movement vector, N is the total number of segmental movement vectors (loss of learning parameter vectors), and Vj++ is the With the weighting factor, vjm-1/l l yj t, 11 l
Calculated by (4). Here, yj is the j-th parameter vector, and tl is the n-th learning parameter vector. The parameter vector sequence subjected to the adaptation process is supplied to the distance matrix calculation circuit 13.

All the codebook elements are supplied from the codebook storage section 5 in parallel to the distance matrix calculation circuit 13, and the distances between each parameter vector and all the codebook elements are calculated. These distance values are arranged in the form of a matrix whose rows and columns are the discretized time axis and codebook element number, respectively, and are sent to the DP calculation circuit 14.
sent to.

At the same time, the DP calculation circuit 14 is supplied with the word dictionary of all recognition target words stored in the word dictionary storage section 7, that is, the codebook series. The DP calculation circuit 14 calculates a spectral sequence of an input audio signal (hereinafter referred to as an "input cover pattern") and a spectral sequence expressed by a codebook sequence of each recognition target word search (hereinafter referred to as a "standard pattern"). The degree of similarity (distance II) is calculated. Even if the same speaker repeats the same word, the rate of speech changes both partially and completely each time, so in order to compare the two, it is necessary to make sure that the common sounds (phonemes) correspond. , it is necessary to appropriately non-linearly expand or contract one time axis to match the other time axis and compare the parameter vectors at corresponding points in time. It is already known that this operation can be performed by dynamic programming (DP) operation using a distance matrix and a word dictionary (Reference, Kanzai Furui: Large word speech recognition using onomatopoeic standard patterns). , Journal of the Institute of Electronics and Communication Engineers, 65-d, 8. P, 1041, 1982)
, use this.

The distance between the standard pattern and the input pattern at corresponding points in time was averaged over the entire speech interval when the time axes were associated using dynamic programming calculations so that the reconnaissance of the standard pattern and the input pattern was maximized. value (hereinafter referred to as “
(referred to as "total distance"). The total distance thus obtained is output to the recognition determination circuit 15 connected to the output stage of the DP calculation circuit 14.

! ! The identification determination circuit 15 determines which of the supplied total distances are:
The standard pattern with the smallest value, that is, the highest degree of similarity, is determined, the word indicated by this standard pattern is determined to be the word input from the audio input terminal 1, and the result is transmitted to the output terminal connected to the output stage. The recognition result is outputted via the recognition result output terminal 16.

Conventionally, a word to be used for adaptation is determined in advance, and the matched input pattern and standard pattern are matched on the time axis using dynamic programming. Adaptation was performed using the movement vector as the difference between the spectra of standard patterns, but in this example, the movement vector was calculated based on the difference between the centroids obtained by clustering the input speech and the codebook. Adaptation is performed using .

As a result, adaptation can be performed using an arbitrary number of word speech or short sentence speech, and it is much easier to achieve a large improvement in recognition accuracy than conventional techniques for an unspecified number of speakers. be able to.

According to this embodiment, a codebook and a word dictionary (codebook series of 411 classes for each word) created from the voices of four standard male speakers are accumulated, with 100 city names as recognition target words. When recognition was performed on the voices of 20 male speakers different from the speaker after adaptation using 10 arbitrary words, a recognition accuracy of 96.6% was obtained. When compared with the recognition accuracy of 95.1% without adaptation, it can be seen that a clear improvement effect can be obtained even though the adaptation process is performed using a very small number of arbitrary words.

In this example, the entire codebook was clustered, but
It is also possible to specify in advance to which standard speaker's speech the elements composing the codebook belong, and then extract only the codebook elements belonging to the standard speaker closest to the input speech signal and use them for clustering. This standard speaker selection can be performed by calculating the distance between the parameter vector calculated from the input audio signal and the codebook element belonging to each standard speaker, and based on these calculated distances. When word speech recognition is performed using this method under the same conditions as above,
We achieved a v2va accuracy of 97.2%.

It can be seen that an extremely large improvement effect can be obtained compared to the case where no adaptation is performed.

Furthermore, before creating a codebook using the voices of multiple standard speakers, the technique of this invention can be used to add other standard speakers' voices to the codebook created from the voices of one standard speaker. The speech signal may be adapted in advance. Using this method, a codebook created after adaptation between standard speakers is used, and this codebook is adapted to the input speech without selecting a standard speaker. 97. under the same conditions as above.
A recognition accuracy of 4% was achieved. The improvement effect of this method is also extremely large.

In the above embodiment, word speech different from the speech to be recognized is used for the adaptation process, but according to the present invention, any speech can be used as the speech for calculating the adaptation vector. It is also possible to temporarily store the voices of the speakers and use them for the adaptation process. In this case, the speaker does not need to specially utter the voices for adaptation, which reduces the burden on the speaker. .

In the above embodiment, the input parameter vector is adapted to the codebook, but the codebook may be adapted to the input parameter and the distance between the adapted codebook element and the input parameter vector may be calculated.

In this example, the linear prediction cepstrum was used as a parameter indicating the frequency spectrum of the voice, but linear prediction coefficients, formant frequencies, Percoll coefficients, log cross-sectional area ratios, number of zero crossings, etc. may also be used. Although dynamic programming was used to time-axis match the input pattern and the standard pattern, a hidden Markov model or the like may also be used.

〔Effect of the invention〕

As explained above, according to the word speech recognition device of the present invention, it is possible to adapt the speaker's speech signal to match the stored codebook using arbitrary speech uttered by the speaker. or the stored codebook can be adapted to match the speaker's speech signal, so a small number of word speech or the speech to be recognized can be used as is for adaptation, and high It has the advantage of being able to perform high-performance speech recognition.

[Brief explanation of the drawing]

FIG. 1 is a block diagram of a word speech recognition device showing an embodiment of the present invention. Patent applicant Nippon Telegraph and Telephone Corporation

Claims (4)

[Claims] (1) A parameter vector calculation means for calculating a parameter vector indicating temporal changes in the frequency spectrum and power of a word speech signal; codebook creation means for clustering parameter vectors and storing a plurality of representative vector values in a codebook; and one or more time series of vector values stored in the codebook for each recognition target vocabulary. means for creating a word dictionary and storing the expression in a word dictionary; means for hierarchically dividing a large number of input parameter vectors calculated from an input speech signal and the element vectors of the codebook into clusters; means for determining a piecewise movement direction vector for adapting said input parameter vector to said codebook for each cluster, and adapting all input parameter vectors as a weighted average of these piecewise movement direction vectors; a means for calculating distances between these adapted input parameter vectors and each element of the codebook; and a means for calculating distances between these adapted input parameter vectors and each element of the codebook; A word speech recognition device comprising means for calculating the distance between input speech and each recognition target vocabulary using a model. (2) a parameter vector calculation means for calculating a parameter vector indicating temporal changes in the frequency spectrum and power of a word speech signal; codebook creation means for clustering parameter vectors and storing a plurality of representative vector values in a codebook; and one or more time series of vector values stored in the codebook for each recognition target vocabulary. means for creating a word dictionary and storing the expression in a word dictionary; means for hierarchically dividing a large number of input parameter vectors calculated from an input speech signal and the element vectors of the codebook into clusters; means for determining piecewise movement direction vectors for adapting said codebook to said input parameter vector for each cluster, and adapting all said codebook elements as a weighted average of these piecewise movement direction vectors; means for adapting these adapted codebook elements;
means for calculating distances from input parameter vectors; means for calculating distances between input speech and each recognition target vocabulary by dynamic programming or hidden Markov models using these calculated distances and the word dictionary; A word speech recognition device consisting of. (3) means for displaying which element of the codebook belongs to which standard speaker; and means for calculating the distance between the input parameter vector calculated from the input audio signal and the codebook element of each standard speaker. and means for selecting a codebook of a standard speaker closest to the input audio signal based on these calculated distances, and the codebook selected in this way is used. 1. The word speech recognition device according to 1. (4) Before creating a codebook using multiple standard speakers,
Claim 1, characterized in that a codebook created from one standard speaker is used in advance to adapt parameter vectors calculated from speech signals of other standard speakers to the codebook of that one standard speaker. 3. The word speech recognition device according to 2 or 3.