JPH053596B2 - - Google Patents

Description

[Detailed description of the invention]

FIELD OF THE INVENTION The present invention relates to a speech recognition device for automatically recognizing the content of speech. 2. Description of the Related Art Recently, voice input devices have come into widespread use in fields such as sorting luggage and inquiring about bank deposits. This speech recognition device was developed, for example, by Joji Miwa et al. in ``Word speech recognition system using the outline form of the speech spectrum and its dynamic characteristics,'' Journal of the Acoustical Society of Japan.
34.3 (1978) is known. Hereinafter, a conventional speech recognition device will be explained with reference to FIG. First, the speech data of many speakers is divided into groups using clustering methods, etc., and standard patterns are created for each phoneme.Standard pattern groups 1, 2,
... is stored in the storage unit 59. For the sake of explanation, it is assumed here that the standard pattern group 1 in the storage section 59 is created for a male voice, and the standard pattern group 2 is stored for a female voice. Further, for the sake of simplicity, it is assumed that only the standard pattern groups 1 and 2 are prepared in the storage section 59. Input audio from the microphone 51 is sent to the A/D converter 52
After being converted by
Transferred to 4. On the other hand, the segmentation section 5
6 performs band power calculation, voice section detection, voiced/unvoiced determination, and consonant segmentation, and transfers the results to the main memory 57. Similarity calculation unit 55
calculates the degree of similarity with the standard pattern in the storage unit 59 using the output of the band filter group obtained by the feature extraction unit 54 as a parameter. First, calculate the similarity with the standard pattern group 1, and store the result in the main memory 5.
Transfer to 7. The same process is performed for the standard pattern group 2, and the main processor 58 adopts the phoneme corresponding to the standard pattern with the highest degree of similarity in the main memory 57 as the recognition result. Furthermore, a time series of phonemes is created using the results of the segmentation unit 56. Finally, word matching unit 6
1, word recognition is performed by comparing the word dictionary 52 with a single prepared confusion matrix to determine word similarity, and the main processor 58 selects the words with the highest word similarity. And the word is output to the output section 63. Problems to be Solved by the Invention However, the above configuration has the following problems. (1) Since the recognition tendency of phoneme recognition, substitution, addition, omission, etc. differs depending on the type of microphone, the content of the noise, and the type of speaker. Due to the deviation, the word recognition rate decreases. (2) It is necessary to always perform similarity calculations for multiple sets of standard patterns, which increases the amount of calculation, and as the number of standard patterns increases, they are often confused with each other during recognition, making it difficult to recognize phonemes. reduce the rate. The present invention solves the above problem, and uses unknown input speech to automatically select and use a standard phoneme pattern group and conflation matrix that are most suitable for the input environment and type of speaker. It is an object of the present invention to provide a speech recognition device that obtains a stable and high word recognition rate even when the type of speaker changes. Means for Solving the Problems The present invention includes a segmentation unit that divides input speech into phonemes, and a standard pattern group and confusion matrix created based on the same speech input environment or type of speaker. A storage section that prepares and stores a plurality of pairs, a similarity calculation section that calculates similarity using the phoneme standard pattern group, and a phoneme sequence created using the results of the segmentation section and similarity calculation section. a main processor that determines the center position of vowels and nasal sounds therein and controls the entire device; and a selection section that automatically selects a group of standard patterns most similar to the input speech using the results of the main processor. The above object is achieved by a speech recognition device configured mainly of a word matching unit that performs word recognition using a fusion matrix that is a pair of the selected standard pattern group. Function The present invention prepares in advance a standard pattern group and a conflation matrix created for the same voice input environment or type of speaker as a pair, and when the voice input environment or type of speaker changes. , automatically selects the phoneme standard pattern group and conflation matrix that are most suitable for the usage conditions, without increasing the amount of calculation in similarity calculation.
It is possible to avoid confusion during recognition and achieve a high phoneme recognition rate, and at the same time, it is possible to improve the word recognition rate. Embodiment The present invention allows the voice of unspecified speakers to be heard without being affected by the input environment such as the type of surrounding noise or the type of microphone, and without being affected by differences between speakers such as gender, age, etc. , to provide a speech recognition device that can stably recognize speech. For this purpose, a group of standard patterns for recognizing speech are created separately according to the input environment such as the type of noise and the type of microphone, and according to the type of speaker such as gender and age. Also, a conflation matrix is created from the voices of multiple speakers under the same conditions, and is prepared as a pair with the standard pattern group. What is Confusion Matrix (hereinafter abbreviated as CM)?
The rate of substitution between phonemes, the rate of dropout of each phoneme,
This represents the rate at which other phonemes are added between phonemes. Although it is unknown what kind of input environment or type of speaker the actual input voice is, by using this method, it is possible to
or automatically determine the type of speaker,
Stable recognition can be achieved by using the standard pattern group and CM that are most suitable for the input voice. In the following embodiment, a method for recognizing two types of voices, male and female voices, will be described for adult men and women. FIG. 1 shows a functional block diagram of a speech recognition device according to an embodiment of the present invention. First, the contents to be stored in the storage section 9 will be explained. First, the LPC cepstrum coefficient of male voice phoneme i is taken as a parameter, and its average value is set as M _i ⁽¹⁾ = (M _i1 ⁽¹⁾ , M _i2 ⁽¹⁾ , . . . , M _ip ⁽¹⁾ ). p is the number of parameters used. This is calculated for six vowels: |a|, |i|, |u|, |e|, |o|, and nasal sounds. Similarly, find 6 pieces for the female voice, making a total of 12 pieces. Let the co-fraction matrix common to these 12 phonemes be |R, and let the inverse matrix be 〓 ^-1 . If the (j, j′) element of 〓 ^-1 is ijj′, then the weighting coefficient for the j-th parameter is, for example, aij ⁽¹⁾ = 2 _p 〓 ^j ′ ⁼¹ rjj′Mij′ ^{(1 )} (1). Also, the distance di ⁽¹⁾ to the average value is determined by di ⁽¹⁾ =〓i ^(1)t 〓−1〓i ⁽¹⁾ (2). These aij ⁽¹⁾ and di ⁽¹⁾ are obtained for each phoneme and stored in the standard pattern group 1 in the storage section 9. At the same time, in this standard pattern group 1, standard patterns of other phonemes (consonants, semi-vowels, etc.) for male voices are also allowed to coexist. Similarly, standard pattern group 2 stores various standard patterns for female voices. Furthermore, the storage section 9 stores conflation matrices CM ₁ , CM ₂ , . . . that are paired with each of the above standard pattern groups.
..., store CM _o . In this case, the conflation matrix only needs to be created from the voices of multiple speakers under the same conditions as the phoneme standard pattern group.
Of course, the storage location may be stored in another location. Next, the operation of the similarity calculation section 5 will be explained. When unknown input audio is input from microphone 1,
A/D converter 2 performs A/D conversion, and signal processing circuit 3
After performing pre-emphasis and window processing, the feature extraction unit 4 extracts the LPC cepstral coefficients Cj (j=1,
2, ..., p) is found. The similarity calculation unit 5 uses this
Similarity calculation is performed using Cj and the standard pattern in the storage section 9 transferred through the selection section 10. The similarity li ⁽¹⁾ of standard pattern group 1 to phoneme i is determined by li ⁽¹⁾ = _p 〓 ^j=1 aij ⁽¹⁾ Cj−di ⁽¹⁾ (3). For the standard pattern group 2, li ⁽²⁾ = _p 〓 ^j=1 aij ⁽²⁾ Cj−di ⁽²⁾ (4) is obtained and transferred to the main memory 7. Next, the operation of the selection section 10 and its relationship with the main processor 8 will be described. The main processor 8 is
With reference to the results of the similarity calculation unit 5 and the results of the segmentation unit 6 transferred to the main memory 7,
Find the positions of vowels and nasal sounds, and determine the center position among them. Furthermore, all similarities of vowels and nasals at that position (6 for each standard pattern group,
(12 patterns in total for standard pattern groups 1 and 2) The one with the highest degree of similarity is selected. If the selected standard pattern belongs to standard pattern group 1, the number of times it was selected N ⁽¹⁾
Increase by 1. If it belongs to standard pattern group 2, the number of times
Increase N ⁽²⁾ by 1. Repeat this operation for each vowel/nasal center position and count up N ⁽¹⁾ or N ⁽²⁾ . Reliability Re is calculated for each input word using N ⁽¹⁾ and N ⁽²⁾ using the following formula. Re ⁽¹⁾ =N ⁽¹⁾ /N ⁽²⁾ (5) Re ⁽²⁾ =N ⁽²⁾ /N ⁽⁴⁾ (6) This Re ⁽¹⁾ or Re ⁽²⁾ is a predetermined threshold When the threshold value is exceeded, the standard pattern group that exceeds the threshold value is decided to be used from now on, and learning is terminated. As long as the threshold is not exceeded, N ⁽¹⁾ and N ⁽²⁾ are compared and the main processor is instructed to use the larger standard pattern group. The word matching unit 11 is instructed to use the CM in a pairing relationship with the simultaneously selected standard pattern group for word matching. The word matching method will be explained. Let each item of the word dictionary 12 be 〓, let the phoneme sequence created by the main processor 8 based on the result of the segmentation unit 6 be 〓, and let the similarity between the two be S.
Represented by (〓, 〓). S (〓, 〓) is calculated by the following formula, assuming that dropouts and additions do not occur consecutively for three or more phonemes in the phoneme sequence, and that dropouts and additions do not occur consecutively. S(〓,〓) =g(I+1, J+1)/(I+J) (7) g(i, j)=2l(i, j) +max{L, La, Laa, Lo, Loo} (8) L= g(i-1, j-1) La=g(i-1, j-2)+la(j-1) La=g(i-1, j-2)+la(j-1) Laa=g( i-1, j-3) + laa (j-2) + laa (j
-1) Lo=g(i-2, j-1)+lo(j-1) Lo=g(i-2, j-1)+lo(j-1) Loo=g(i-3, j-1 )−loo(i−2)+loo(i
−1)(9) Here, I is the number of phonemes of 〓, J is the number of phonemes of 〓, l
(i, j) is the likelihood function of the i-th phoneme of 〓 and the j-th phoneme of 〓, la(j), lo(i) are the likelihood functions of adding and dropping one phoneme, laa(j), loo( i) represents the likelihood function of addition and omission of two consecutive phonemes. Obtained S(〓, maximum similarity among 〓〓
is output to the output unit 13 via the main processor 8 as a word recognition result. Therefore, the content of the CM is the above l(i, j), la(j), lo
(i), laa(j), and loo(i), and their contents can be categorized and prepared according to the input environment and type of speaker to determine the error tendency of phoneme recognition that meets the usage conditions. It is possible to perform word matching that reflects the FIG. 2 shows the flow of processing according to this embodiment. After the first voice is input (judgment A), acoustic analysis such as pre-emphasis, window calculation, and calculation of LPC cepstral coefficients is performed (STUTSUPRO). Next, segmentation and similarity calculation (1) are performed (step 1), and the pattern with the maximum similarity among all the standard pattern groups at the center position of the vowel/nasal sound is selected (step 2). With this, N ⁽¹⁾ and N ⁽²⁾ are obtained, and the reliability is calculated using equations (5) and (6) (step 4). If the reliability is greater than or equal to the threshold (determination), it is determined to use the standard pattern group that exceeds the threshold. Also, it is decided to use the CM prepared in pair with this standard pattern group (step). If it is below the threshold, compare N ⁽¹⁾ and N ⁽²⁾ above, and use the larger standard pattern group and the CM prepared for its pair. Next, phoneme recognition is performed using the specified standard pattern group (step), word recognition is performed using the specified commercial (step), the results are output (step), and the process returns to the beginning. When the next voice is input, after acoustic analysis (STEP PRO), it is checked whether the selection has been completed (JUDGE), and if not, the same process as for the first voice is repeated. If so, perform segmentation and similarity calculation (2) using the already selected standard pattern group (step), perform phoneme recognition (step), and then perform segmentation and similarity calculation (2) using the already selected standard pattern group. Performs word recognition (Stepply) and outputs the result (Stepnu). In this example, the standard pattern groups are for male and female voices, but the standard pattern groups are created based on the voice input environment (type of external noise, type of microphone, etc.) rather than the type of speaker. This method is more effective if the fusion matrix is prepared in pairs with a group of standard patterns in the same input environment.
Furthermore, it is natural that the number of storage units does not need to be one, and that the storage unit may be divided into a plurality of units. Also, when there are three or more standard pattern groups, selection can be done in the same way, and N ⁽¹⁾ and N ^{(2) in equations (5) and (6)}
If we add N ⁽³⁾ , N ⁽⁴⁾ ... and calculate the ratio using the two largest Ns, the reliability Re
can be found. In this way, in this method, multiple sets of standard patterns are prepared depending on the voice input environment and the type of speaker, and a conflation matrix is prepared in pairs with the standard pattern group under the same conditions. The degree of similarity at the position near the center of the vowel/nasal sound is determined, and the degree of confidence for selection is determined using the number of times the standard pattern group to which the standard pattern with the highest degree of similarity belongs is used, and when the degree of confidence exceeds the threshold, It is characterized by completing the selection work, and can be realized without requiring particularly complex calculations or processing. Regarding the voice recognition device of this example, 100 adult men and women
Using the first 10 words out of 212 words,
Table 1 shows the results of the cumulative evaluation of the number of words required for selection based on the number of people.

[Table] In other words, if four words are used, up to 98 out of 100 people can correctly select the standard pattern group. One person who made a mistake mistook a female voice for a man's voice, but the nature of this speaker's voice is between that of a man's voice and that of a woman's voice, so even if he makes a mistake, the rate of vowel/nasal sound recognition will be extremely small. In this way, by using this method, it is possible to select the optimal standard pattern group with high accuracy. Table 2 shows the results of evaluating and comparing the average recognition rate of the five vowels and nasal sounds on a frame-by-frame basis for 20 men and women. The frame recognition rate is shown in %, and the variation in recognition rate is shown in parentheses as standard deviation. Compared to the conventional method, which does not distinguish between men and women, the recognition rate of the present method with the distinction of men and women is greatly improved and the variation is reduced, demonstrating the effectiveness of the present method. Further, by selecting the standard pattern group, a paired CM can be selected, and a high word recognition rate can be obtained by using the selected CM. Table 2 shows a comparison of word recognition rates when commercials were created for both men and women and when commercials were created for both men and women. It was evaluated using 212 words written by 40 people.

[Table] In other words, by using the commercial that best matches the input environment or type of speaker, the average recognition rate improves, and the variation in word recognition rate among speakers decreases.
Stable results are obtained. Effects of the Invention As described above, the present invention separately prepares standard pattern groups depending on the input environment and type of speaker, simultaneously creates commercials under the same conditions, and prepares them in pairs with the standard pattern group. By providing a function that automatically selects the most suitable standard pattern group and CM for the unknown input voice using unknown input voice, it is possible to select the most suitable standard pattern group and CM without putting a burden on the speaker. can be used to recognize speech,
This method has the advantage that it is possible to obtain a stable and high word recognition rate for various input environments and types of speakers.

[Brief explanation of the drawing]

FIG. 1 is a functional block diagram of a speech recognition device according to an embodiment of the present invention, FIG. 2 is a flowchart showing a processing procedure of the present invention, and FIG. 3 is a functional block diagram of a conventional speech recognition device. 5...Similarity calculation unit, 8...Main processor, 9...Storage unit, 10...Selection unit, 11...Word matching unit, 12...Word dictionary, 55...Similarity calculation unit, 59... . . . Storage section, 60 . . . Confusion matrix, 61 . . . Word matching section.

Claims (1)

[Scope of Claims] 1. A feature extraction unit that obtains feature parameters from input speech, a segmentation unit that divides input speech into phonemes, and a plurality of pairs of phoneme standard pattern groups and confusion matrices that are stored. Do 1
one or more storage units, a similarity calculation unit that calculates the similarity of the feature parameters using the phoneme standard pattern group, and a phoneme sequence using the results of the segmentation unit and similarity calculation unit. a main processor that determines the center position of the vowels and nasals in the voice and controls the entire device; and automatically selects a group of standard patterns that are most similar to the input voice using the processing results of the main processor. a selection section, a word dictionary section written in a phoneme sequence, and a word matching section that calculates the degree of similarity between the content of the word dictionary section using a confusion matrix that is a pair of the selected standard pattern group. A voice recognition device comprising at least the following. 2. Selection of a standard pattern group in the selection section;
Calculate the total number of times a standard pattern with the maximum similarity is obtained for each standard pattern group, and terminate when the ratio or difference between the maximum number and the second largest number exceeds a predetermined threshold. A speech recognition device according to claim 1, characterized in that: 3. Claim 1, characterized in that the pair of the standard pattern group and the confusion matrix in the storage section are created under the same voice input environment or the same type of gender and age range. Speech recognition device described in section.