JPH0566599B2 - - Google Patents

Description

[Detailed description of the invention]

INDUSTRIAL APPLICATION FIELD The present invention relates to a speech recognition device that recognizes continuously uttered words and phrases in units of speech segments such as syllables. 2. Description of the Related Art If voice, which is the most natural means of generating information for humans, could be used as an input means for a human-machine system, the effect would be enormous. Conventionally, speech recognition devices based on a specific speaker registration method have been put into practical use. That is, a speaker who intends to use a recognition device converts all the words to be recognized into a series of feature vectors using his/her own voice and registers them as standard patterns in a word dictionary, and then uses the voice uttered during recognition. is similarly converted into a series of feature vectors, which word in the word dictionary is closest is calculated according to predetermined criteria, and the most similar word is taken as the recognition result. . However, this method is good when the number of recognized words is small, but as the number of words increases to hundreds or thousands of words, the following three problems become impossible to ignore. (1) The burden on speakers during registration increases significantly. (2) The time required to calculate the similarity or distance between the voice uttered and the standard pattern during recognition increases significantly, and the response speed of the recognition device becomes slow. (3) The memory required for the word dictionary becomes very large. As a method to avoid the above-mentioned drawbacks, there is a method in which the unit of recognition is a consonant + vowel or a monosyllable of a vowel (hereinafter expressed as CV and V, respectively. C means a consonant and V means a vowel). That is, monosyllables are registered as a series of feature vectors as a standard pattern, and input speech converted into a series of feature vectors during recognition is converted into a series of monosyllables by matching with the monosyllable standard pattern. It is something to do. In Japanese, the number of monosyllables is at most
There are 101 types, and monosyllables correspond to kana characters, so this method converts (recognizes) any Japanese word or sentence into a monosyllable string.
This will solve all of the problems (1) to (3) above. However, problems in this case include articulatory combination and segmentation. Articulatory coupling is a phenomenon in which when syllables are uttered in succession, each syllable is influenced by the syllables before and after it, and the spectral structure changes depending on the syllables connected before and after it. Segmentation is the process of dividing continuously uttered speech into monosyllable units, but a definitive method for doing this reliably has not yet been found. In order to solve these two problems, the current practice is to separate each single syllable and pronounce it, and some devices are in practical use. However, uttering monosyllables discretely is unnatural and puts stress on the speaker. Problems to be Solved by the Invention The present invention avoids the uncertainty of segmentation for the continuously uttered speech, and also
It is an object of the present invention to provide a continuous speech recognition device capable of recognizing continuously uttered words or phrases. Means for Solving the Problems The present invention is defined as a feature extraction means for converting an input audio signal obtained by continuously uttering words, phrases, etc. into a series of feature vectors, and vowels, consonants, or combinations thereof. standard pattern storage means for storing a series of feature vectors corresponding to each of the speech segments in association with the speech segment name;
determined by an arbitrary or predetermined various combinations of a unit boundary candidate detection means for detecting unit boundaries with respect to an input pattern, each of the standard patterns, and the unit boundary candidates detected from the input pattern. a segment matching means for calculating the distance (similarity) between the two by matching with a subinterval (first subinterval), and representing each word, phrase, etc. to be recognized as a sequence of the phonetic segment names; words
A word/phrase dictionary that stores phrases, etc., and distances ( similarity), the first sub-interval group included in the second sub-interval is such that adjacent sections are continuous so as to correspond to the sequence of segment names specified by the word/phrase dictionary. By optimally determining as follows, the sum of the distances (similarities) corresponding to the start point and end point of each first subinterval and the segment name of that subinterval is set to the minimum (maximum), and the obtained minimum value ( a word/phrase matching means having a function of outputting a maximum value (maximum value) as the distance of the second partial interval to each word/phrase, and optimally determining the second partial interval group so that adjacent intervals are continuous. According to the second
A sequence in which the sum of the distances (similarities) corresponding to the start and end points of each subinterval and the word/phrase name in that subinterval is the minimum (maximum), and the word/phrase string at that time is determined as the recognition result. This is a continuous speech recognition device equipped with word/phrase determination means. Effect The present invention has the above-described configuration, converts an input speech signal obtained by continuously uttering words, phrases, etc. into a series of feature vectors, and converts the input speech signal into a series of feature vectors into speech elements defined as vowels, consonants, or combinations thereof. A standard pattern in which a series of feature vectors corresponding to each of The distance (similarity) between the two is calculated by matching with the subinterval (first subinterval) determined by various combinations, and each word, phrase, etc. to be recognized is identified by the phoneme name. Distance (similarity) between each word or phrase to be recognized, such as a word or phrase expressed as a sequence, and various partial intervals (second partial intervals) of the arbitrary or predetermined segment boundary candidates of the input pattern. Optimize the first subinterval group included in the second subinterval so that adjacent sections are continuous, so as to correspond to the sequence of segment names specified by the word/clause. By determining, the sum of the distances (similarities) corresponding to the start point and end point of each first subinterval and the segment name of that subinterval is the minimum (maximum), and the obtained minimum value (maximum value) is By determining the distance of the second sub-interval to each word/phrase and optimally determining the second sub-interval group so that adjacent sections are continuous, the starting point and end point of each second sub-interval and their The sum of the distances (similarities) corresponding to the word/phrase names in the partial section is set as the minimum (maximum), and the word/phrase string at that time is determined as the recognition result. Examples Hereinafter, the word "word" will also represent the word "bunsetsu." Further, "similarity" will be explained using "distance" as a representative. That is, a small distance means a large degree of similarity. First, DP matching, which is the basis of the present invention, will be described. FIG. 2 is a lattice graph explaining DP matching when recognizing discrete words. That is, input pattern A = a ₁ , a ₂ ... a _i ... a _I and standard pattern B ⁿ =
A case is shown in which the distance to b ⁿ ₁ , b ⁿ ₂ ... b ⁿ _j ... b ⁿ _j n is calculated. The horizontal axis shows the input pattern, the vertical axis shows the standard pattern, and 1 is a curve showing the correspondence between the feature vectors of both. DP matching minimizes the weight average along this path of the distance d ⁿ (i, _j ) between a _i and b ⁿ j that are matched by this path by optimally determining this path, and This value is used as the distance between the two, and this calculation is performed efficiently. d ⁿ (i, j) is, for example, d ⁿ
It can be expressed as (i, j)=|a _i −b ⁿ _j |, etc. In this case, when determining route 1, constraint conditions for route selection are provided. Figure b is
This is an example of the route constraint conditions. That is, the point (i,
The points before reaching point (i, j) are point (i+1, j+2), point (i+1, j+1), and point (i+2, j+1), and the route to point (i, j) is limited to the route shown in the figure. Ru. The numbers shown on the routes in the figure indicate the weighting coefficients when that route is selected. When constraining paths as in this example, on the lattice graph in Figure A, the sum of weights for a path connecting arbitrary lattice points is constant regardless of how it is selected, and the sum of weights between both points is constant. Equal to the length of the input pattern. Therefore, in this case, there is no need to average the sum of d ⁿ (i, j) along the path using a weighted sum, and the sum itself can be used as the distance between the input pattern and the standard pattern. The specific calculation is performed by solving the following recurrence formula. That is, D ⁿ (i, j)=minD ⁿ (i+1, j+2)+d ⁿ (i, j
) D ⁿ (i+1, j+1)+d ⁿ (i, j) D ⁿ (i+1, j+1)+d ⁿ (i, j) D ⁿ (i+2, j+1)+d ⁿ (i+1, j)+d ⁿ (i, j
)...(1) as i=I, I-1,..., 2, 1, i=J ⁿ , J ⁿ −
Initial value D ⁿ (I, J ⁿ )=d ⁿ for 1,...,2,1
(I, J ⁿ ) and let D ⁿ (1, 1) be the distance between the two. By selecting the route constraint conditions as shown in FIG. 2B, the routes that can actually be selected are limited to those within the hatched area in FIG. This means that pattern A and pattern
When B ⁿ is for the same word, it should not deviate that much, and when it is for different words, there is no risk of making an unreasonable correspondence and making the distance value of both patterns unduly small. This is consistent with the purpose of doing so. Figures 3 and 4 show that by DP matching,
FIG. 3 is a diagram illustrating the principle of the present invention when performing continuous word recognition. Figure 3 shows partial patterns of the input pattern whose ending point is the k-th syllable boundary and whose starting point is the range described below, and V, CV, VV, VCV (V is a vowel, C
is a consonant) and other syllables (speech elements) standard patterns.
It is a diagram illustrating the state of DP matching, and is a grid graph in which the horizontal axis is the input pattern and the vertical axis is the standard pattern. 4 is a straight line with j=1, n ₁ and n ₂ each represent an example of a standard speech unit pattern, and the number of frames of unit n is J ⁿ . Now, let us consider the case of matching the partial pattern of the input with the standard pattern of segment _n1 . At this time,
Applying the path constraint shown in Figure 1b, if the k-th segment boundary candidate is St(k) (k = 0 is the beginning of a word), then
For the starting point of matching of point (St(k), J ⁿ¹ ),
The matching range is the range surrounded by straight lines 5, 6, and 4, and the segment boundary candidate points between points 9 and 10 are
If k′, then according to the calculation of recurrence formula (1), the segment cumulative matching distance D _n1 ⁽ k ^′ : k) is D ⁿ¹
It is given by (k′:k)=D ⁿ¹ (St(k′), 1). Here, point 9 is the intersection of straight line 5 and straight line 4, point 10 is the intersection of straight line 6 and straight line 4, and straight line 5 has a slope of 1/
2. The straight line 6 has a slope of 2. In this case, in matching the partial pattern of the input pattern whose end point is the k-th unit boundary candidate point and the standard pattern R ⁿ¹ , the range of the starting point k' is between points 9 and 10, and gradually According to Formula 1, j=J ⁿ , J ⁿ −1,…,
1, i′=max{St(k)−2(j ⁿ −
j), 1}, {St(k)−2(J ⁿ −j)+1,1},…,
Regarding max{St(k)−[(J ⁿ −j+1)/2], 1}
By sequentially calculating D ⁿ¹ (i′, j), D ⁿ¹ (k′:k)=D ⁿ¹ for k′ between points 9 and 10
(St(k′), 1) can be found at the same time. Here, max
{x, y} means the larger value of x, y,
[x] indicates the largest integer not exceeding x. Also j
The range of i' at =j is as above because D ⁿ (i', j) is not defined at i'0. Similarly, for the standard pattern R ⁿ² , the intersection point 7 of the straight line 2 with a slope of 1/2 passing through the point (St(k), J ⁿ² ) and the straight line 4, and the point (St(k), J ⁿ² ) For k ^′ in the range of intersection 8 between straight line 3 with slope 2 and straight line 4,
(k′:k)=D ⁿ² (St(k′), 1) is found. In each input boundary candidate frame St(k), n=1, 2,
..., N in this manner to find D ⁿ (k':k). FIG. 4 explains a method for performing calculations similar to those for the segment standard pattern when the standard pattern is a word. In other words, it explains the state of DP matching between a standard pattern ^w for a word w and a partial pattern of an input pattern whose end point is the input St(k)th frame and whose start point is the range described below, and the horizontal axis represents the input pattern. , is a grid graph with a standard pattern on the vertical axis. 11 is the straight line of j′=1,
^w = R ^s(w,1) , R ^s(w,2) , R ^s(w,3) are word standard patterns ^w
An example is shown. Here, j' is the frame number assigned from the first frame to the last frame of the word standard pattern ^w , and s(w,) is the number representing the name of the th phoneme segment of the word w. In this example, The word w consists of a series of three segment names, and the standard pattern ^w of the word w is the corresponding three segment standard patterns R ^s(w,1) , R ^s(w,2) , R ^{s(w ,3)} . In this case too, Figure 2b
Applying the constraint, the number of frames for word w is J ^w = J ^s(w,1) + J ^s(w,2) , +J ^s(w,3) , and the point (St(k
),
With respect to the matching starting point of J ^w ), the matching range is the range surrounded by straight lines 12, 13, and 11, and the boundary candidate number between points 14 and 15 is
If k′, then according to calculation similar to recurrence formula (1),
The cumulative word matching distance ^w (k':k) between the subpatterns of the input pattern k' to k and ^w is determined. That is, the recurrence formula in this case is ^w (i', j') = min ^w (i'+1, j'+2) + ^w (
i', j') ^w (i'+1, j'+1)+ ^w (i', j') ^w (i'+1, j'+1)+ ^w (i', j') ^w (i'+2, j′+1)+ ^w (i′+1,j′)+ ^w
(i′, j′)…(2) Initial value ^w (St(k), ^w )= ^w (St(k), ^w )
Then, ^w (i′, j′) is j′= ^w , ^w −1,…,
By sequentially calculating i′ for each of 2 and 1 within the range of straight line 12 to 13, ^w (k′:k)=
It can be obtained as ^w (St(k'), 1). Here, ^w (i′, j′) is the intervector distance between the feature vector a _i ′ of the input pattern of the i′th frame of input and the feature vector ^w _j ′ of the j′th frame ^of the standard pattern w of word w. The same definition as d ⁿ (i, j) above is used: ^w (i', j')=|a _i '−b ⁿ _j '|. Also,
Straight line 12 passes through the point (St(k), ^w ) with a slope of 1/2, straight line 13 passes through the point (St(k), ^w ) with a slope of 2, and the point 14 is the line between straight lines 12 and 11. , point 15 is the intersection of straight lines 13 and 11. Then w∧=arg
Calculate minw [ ^w (k′:k)]. argminx
in[f(x)] means x that minimizes f(x). To recognize continuously uttered words, k=1, 2,
..., K, perform the above calculations, optimally divide the input pattern in terms of number, position, etc., minimize the minimum cumulative word matching distance for each divided partial interval, and then It is sufficient to use the words obtained for each section as the recognition results for each section, but as the number of words increases, it becomes difficult to obtain the cumulative word matching distance ^w (k':k) using the above method. The amount of calculation becomes enormous. Therefore, the present invention is characterized in that the amount of calculation is significantly reduced by using the segment cumulative matching distance to obtain the word cumulative matching distance. That is, in this example, ^w
The matching of the last segment standard pattern R ^{s (w, 3)} and the input pattern is performed on the area surrounded by straight lines 12, 13, and 16, and as a result ^w (St (k'), 3)
has already been determined as the segment cumulative matching distance D ^s(w,3) (k':k) on the straight line 16 in the portion sandwiched by the straight lines 12 and 13. k' is a segment boundary candidate number included in the portion. Straight line 16 is j′= ^w −J ^s(w,3) +1. From the final frame of word standard pattern ^w ,
Matching up to the penultimate segment R ^s(w,2) is performed on the area surrounded by straight lines 12, 13, and 17, and as a result, ^w (k′,2) is
This is required for the part sandwiched by straight lines 12 and 13, and according to the principles of dynamic programming, ^w (k', 2) = min k'' [ ^w (k'', 3) +D ^s(w,2) (k′:k″)]. The segment cumulative matching distance D ^s(w,2) (k′:
k'') and the intermediate cumulative matching distance ^w (k', 3) have already been found.Here, the straight line 17
is j′= ^w −J ^s(w,3) −J ^s(w,2) +1, and k′ is the segment boundary of the input pattern included in the part of straight line 17 sandwiched by straight lines 12 and 13. This is the candidate number. Also
For example, when k' is a point 20 on the straight line 17, k'' is a point on the straight line 16, and is the part that passes through the point 20 and is sandwiched by the straight line 18 that is parallel to the straight line 13 and the straight line 19 that is parallel to the straight line 12. is the segment boundary candidate number of the common part of the part sandwiched by straight lines 12 and 13. Similarly, when k' is point 23, k'' is a point on straight line 16, and
The part that passes through point 23 and is sandwiched by straight lines 21 and 22 that are parallel to straight lines 13 and 12, respectively, and the straight lines 12 and 13
This is the segment boundary candidate number of the common part of the parts sandwiched by. This means that when the path constraint conditions are set as shown in Figure 2b, the matching path from point (St(k), ^w ) to point 20 is a straight line 12, 13, 18, 19.
The point (St
This means that the matching path from (k), ^w ) to point 23 is limited to the inside of the parallelogram surrounded by straight lines 12, 13, 21, and 22. Similarly, from the last frame of the word standard pattern, 3 from the end
Fragment up to the th elemental fragment (in this example, the first elemental fragment of word w)
Matching up to R ^s(w,1) is straight lines 11, 12, 13
The result ^{is w}
(k', 1) is found for the part from point 14 to point 15, and this also follows the principle of dynamic programming as ^w (k', 1)=min k'' [ ^w (k ″, 2) + D ^s(w,1) (k′:k″)], and the segment cumulative matching distance D ^s(w,1) (k′:
k'') and the cumulative matching distance ^w (k'', 2) in the middle of a word have already been determined. In the above manner, the unitary cumulative matching distance is calculated in advance, and from this, the word cumulative matching distance ^w (k′:k) is calculated by ^w (k′:
k) = ^w (k', 1).
If each word w is represented by a combination of its component standard patterns and then directly matched with the input pattern, matching calculations for the number of words are required in each frame.
According to the method of the present invention, it is only necessary to match the number of elements in each frame of input, so when recognizing a large vocabulary of several thousand words, the amount of calculation is much smaller, and the equivalent It's something that gets results. Once the word cumulative matching distance ^w (k′:k″) is found,
Assuming that the St(k)th frame is the final frame,
The optimal word string from the first frame to the St(k)th frame can be found using the following recurrence formula based on the principle of dynamic programming. That is, D~(k) is the cumulative matching distance between the input partial pattern from the first frame to the St(k)th frame and the feature vector series for the optimal word string for it, and B~((k) is the last 1 from tail word
If the final boundary candidate number of the previous word, N~(k), is the last word name, initial conditions D~(o)=0, B~(o)=0, and D~(k)=min k ′, w [D~(k')+D~ ^w (k':k)] B~(k)=k^'B~(k)=k^' N~(k)=w^ (k^' , w^ is k′ that satisfies the above formula,
w)…(3) is given. If the above calculation is performed for k=1, 2, . . . , K, the recognition result is obtained as follows. Last word: N~(K) Second to last word: N~(B(K)) Third to last word: N~(B(B(K)))... First word: N~( B~(B~(...(B~(K))...
))) When B~(B~(...(B~(K))...)))=0, the process ends. FIG. 5 is a flowchart for determining the upper word string from N~(K) and B~(K). The above is an example of finding the optimal solution when the number of words is unknown. However, when the number of words is known, the method of the present invention can be easily used by changing equation (3) even when using automaton control. When the number of words is known, X is the number of words, and D ~ _x (k) is the standard that optimally connects the subpatterns from the first frame to the St(k)th frame of the input pattern and x word standard patterns. Cumulative matching distance with the pattern,
B~ _x (k) is the back pointer for the D~ _x (k), N~
_x
If (k) is the last word for the above D~ _x (k), then the formula
The recurrence formula (3) is given by the initial conditions D~ _p (O)=0, B~ _p (O
)
= 0, D~ _x (k)=min i', w [D~ _x-1 (k') + ^w (k':k)] B~ _x (k)=k^' B~ _x (k) =k^' N~ _x (k)=w^ (k^', w^ are k' that satisfy the above formula
, w)…(4). If equation (4) is calculated for k=1, 2, . . . , K, the recognition result is obtained as follows. Last word: N~ _x (K) Second to last word: N~ _x-1 (B~ _x (K)) Third last word: N~ _x-2 (B~ _x-1 (B _x
(K))) … First word: N~ ₁ (B~ ₂ (B~ ₃ ) (…(B~ _x (K)
)
…))) B~ ₁ (B~ ₂ (B~ ₃ (…(B~ _x (K))…))) =
It becomes 0 and ends. FIG. 6 is a flowchart for determining the upper word string from N~ _x (k) and B~ _x (k). In case of automaton control, it is as follows. The difference from normal automaton recognition problems is that
The frame number representing time is also included as a variable, and the output is not just acceptance or rejection, but the degree of acceptability (cumulative distance). Let D~ _q (k) be the minimum cumulative distance among all word strings assuming that it ends at the input St(k) frame in state q, and N~ _q (k) be the word string corresponding to D~ _q (k). The last word name of B~ _q (k) is the starting point position of N~ _q (k) minus 1 (N~ _q (
k), the long end frame of the previous word (i.e., the back pointer), and the state name that satisfies D~ _q (k), that is, Δ, by the state transition of Q~ _q (k) to q, is the state transition rule. When Δ
When (Q~ _q (k), N~ _q (k)) = q, a solution by automaton control can be obtained by solving the following recurrence formula. That is, by replacing x in equation (4) with state q, the recurrence equation for finding D~ _q (k) becomes as follows. Initial conditions D~ _p (O) = 0, B~ _p (O) = 0, D~ _q (k) = min k', w, p [D~ _p (k') + ^w (k': k) ], q=Δ(p,n)
...(5) for q = 1, 2, ..., |s|-1 (s
is a finite set of states q), k′, w,
When p is k^', w^, p^, N~ _q (k)=w^, B~ _q (k)=k^', Q~ _q (k)=p^. If this calculation is performed until k=K, words can be found in reverse order starting from the last word as follows. That is, k=K, q=minq _f D~ _qf ∈F (F is the set of final states) w^=N~ _q (k) B~ _q If (k)≠0, then k=B~ _q (k ), q=Q~ _q (k), and if B~ _q (k)=0, the process ends. FIG. 7 is a flowchart. FIG. 1 shows an embodiment of the present invention. This embodiment is an example in which the number of words is unknown. A case will be explained in which VCV syllables, CV syllables, etc. are used as speech segments. In this case, the syllable boundary is assumed to be the center of the vowel stationary part. 100 is an audio signal terminal. Reference numeral 101 denotes a feature extraction unit, which is composed of a filter bank and the like, and converts an input audio signal into a series of feature vectors a ₁ , a ₂ , . . . , a _I . Reference numeral 116 is a vowel standard pattern storage unit that stores standard patterns for each vowel. A vowel recognition unit 117 compares each frame of the input pattern with each vowel standard pattern in the vowel standard pattern storage unit 116, and performs vowel recognition by regarding each frame as a vowel. This can be done, for example, by determining the distance between each frame of the input and each vowel standard pattern. A vowel center detection unit 118 detects the center of each vowel part of the input pattern from the output vowel sequence of the vowel recognition unit 117. for example,
When the same vowel is consecutive, the center of the same vowel is set as the vowel center of that vowel. Reference numeral 119 detects silent intervals from the input pattern and roughly classifies consonants. To detect a silent section, the power is determined from the input pattern, and if the value is below a predetermined threshold, it is determined that there is no sound, and if it is above a predetermined threshold, it is determined that there is sound. For consonant classification, consonant parts are detected and roughly identified as fricative, plosive, etc., using well-known methods such as spectral polarization. 120 is a feature series storage unit, which includes a vowel center detection unit 118
It stores the vowel series centered on vowels obtained in the above, and the series of silences, consonants, etc. obtained by the silent section detection/consonant classification section 119. Reference numeral 102 denotes a segment standard pattern storage unit which stores a series of feature vectors corresponding to each of CV and VCV as a standard pattern. Reference numeral 103 denotes a segment matching unit, which matches the input pattern and the segment standard pattern.
Perform DP matching. In this case, for example, if we consider the case where processing is performed at the center of the kth vowel, k
If the recognition result of the vowel in the center of the th vowel is V(k), then the frame of the input pattern for k′k−1 is
The partial pattern from St(k') to St(k), the preceding vowel is V(k'), the following vowel is V(k), and the k'th consonant is stored in the feature sequence storage unit 120. DP matching is performed with the VCV syllable standard pattern R ⁿ that satisfies the consonant size division result between the vowel center of and the kth vowel center, and the unitary cumulative matching distance D ⁿ (k′:k) is calculated. Here, it is sufficient to consider only k' existing on the base of the triangle explained in FIG. 3 and determined for each syllable standard pattern. However, for k=1, 2,...,k, max{St(k)−2
(J ⁿ −1), 1}St(k′)max{St(k)−[(J ⁿ −
j
+1)/2], 1}, n is n that satisfies the above condition. Also, since D ⁿ (k':k) is not defined in St(k')0, the range of St(k') is
When using the constraint conditions for the path, the range is as shown here. Reference numeral 104 denotes a part that stores the segment cumulative matching distance D ⁿ (k':k) calculated by the segment matching section. Reference numeral 105 is a word dictionary in which each word w to be recognized is expressed as a series of phoneme names. Reference numeral 121 is a candidate word determination unit, which is a word dictionary 1.
05 is a word to be matched or not, it is compared with the memory contents of the feature series storage unit 120, and candidate words are preliminarily selected.
Now, when detecting the center of a vowel, assuming that there is an insertion but no omission, and that two insertions do not occur in succession, using the matching path shown in Figure 8a,
This method selects candidate words by matching feature series. In other words, if a segment is a VCV syllable, the characteristics of the +1st syllable of a word in the word dictionary are
If it is included in the features from the k'th vowel center to the k'+1 or k'+2th vowel center of the input pattern, the distance between the two is dd(k',) = 0, and if it is not included, dd (k′,)=1, recurrence formula DD(k′,)=minDD(k′+1,+1)+dd(k′,
) DD (k'+2, +1) + dd (k',) ...(6) Repeat the initial value DD (k, L ^w ) = 0 for = L ^w -1, L ^w -2, ..., 1, 0. Calculate and in the range k−2L ^w k′k−L ^w
Determine whether the value of DD(k′:o) is 0 or not,
If DD (k′, o) = 0, the word is a candidate word, and DD is selected as a target for word matching.
If (k', o)≠0, the word is considered to be a non-candidate word and is omitted from the word matching target. That is, in recurrence formula (6), the normalization coefficient of the path of DP matching is equal to the number of syllables of the standard pattern, and matching is performed with free endpoints on the input side. To explain this graphically, as an example for a three-syllable word, the matching range is the straight line 122 with a slope of 1/2 and the straight line 1 with a slope of 2 in Figure 8b.
This is the area sandwiched by 23. However, in the figure, the horizontal axis is the vowel center number sequence of the input pattern, and the vertical axis is the syllable standard pattern sequence of the word to be matched, and by expanding or contracting the time axis, they all become the same interval. It is depicted as follows. In this figure, if there is a feature sequence with a possibility of word w among the feature sequences whose starting point is something between k-6 and k-3 and ending with k, then DD(k', o) =0
If k′ exists and there is no possibility, then DD(k′,
k′ with o)=0 does not exist. here,
L ^w is the number of prime pieces of word w. 106 is a word matching unit which, for each candidate word w selected by the candidate word determination unit 121, calculates a partial pattern from the k'th vowel center to the kth vowel center of the input pattern and a word standard pattern ^w =
Matching with R ^s(w,1) , R ^s(w,2) , ...R ^s(w,Lw) is performed based on the segment cumulative matching distance, and word cumulative matching distance ^w (k′:k ) is the part that calculates. In the case of this embodiment, an example is illustrated in FIG. Similar to Figure 8, this is drawn by expanding and contracting each axis so that the length between the vowel centers of the input pattern and the length of the standard pattern are the same, and matching with the three-syllable word. This is the case.
The correspondence between the vowel center number k of the input pattern and the syllable number of the standard pattern is shown in Figure 9a, so
The matching path between the input pattern and the standard pattern passes through the point p=(k, 3) and is limited to the range sandwiched by straight lines 125 and 126 with a slope of 1/2 and a slope of 1. In this case, line segment A is the part sandwiched by straight line 125 and straight line 126 where = 1, and line segment B is the straight line = 2.
The part sandwiched between the straight line 125 and the straight line 126,
If r is a point on line segment A and q is a point on line segment B, then
The minimum cumulative matching distance from point p to point r is when the sum of the minimum cumulative matching distance from point p to point q on line segment B and the minimum cumulative matching distance from point q to point r is minimized with respect to point q. can be the minimum value of
In this case, the minimum cumulative matching distance from point p to point q and from point q to point r have already been found from the above explanation, so the minimum cumulative matching distance from point p to point r is ^w (k −3,1)=min min k″=k−2,k−1 [ ^w (k″,2)+D ^s(w,2) (k−
3:k)]. By sequentially repeating this operation for = L ^w -1, L ^w -2, ..., 1, 0 with ^w (k, 3) = 0, the vowel centers k-6 to k-3 of the input pattern are the starting point, and k The matching distance between the partial pattern and word w with end point ^{is w}
It is found as (k':k)= ^w (k', 1). however,
k-6k'k-3. In general, the range of k′ for calculating ^w (k′,) at = is max
{k−2(L ^w −), o}k′max{k−(L ^w −
, o}. Here, ^w (k′:k) is k′
-1 is not defined, so the range of k' is as shown here. Reference numeral 107 is a word matching result storage section that stores the word cumulative matching distance ^w (k':k). Reference numeral 108 denotes a terminal cumulative distance calculating section which calculates, according to recurrence formula 3, from the contents of the word matching result storage section 107 and the contents of the terminal cumulative distance storage section 108.
Calculate D~(k), N~(k), and B~(k). The terminal cumulative distance storage section 109 stores the terminal cumulative distance D~(k) calculated by the terminal cumulative distance calculation section 108 until it is no longer needed. This D~(k) is used in the calculation of recurrence formula 3 in the terminal cumulative distance calculation section 108. Reference numeral 110 is a back pointer storage unit that stores the back pointers B to (k) calculated by the end cumulative distance calculation unit 108. Reference numeral 111 denotes a last word storage unit that stores the last word at the k-th vowel center determined by the end cumulative distance storage unit 109. Reference numeral 112 denotes a speech interval detection unit, which determines the speech interval based on the magnitude of the input signal, etc. When the speech interval detection unit 112 detects that voice input has started, the vowel center counting unit 113 detects the vowel. Start counting for each center. The above processing was for the k vowel center, and the count value of the vowel center counting section 113 sets this k. Therefore, the same processing as described above is performed every time the vowel center advances by one. The vowel center counting unit 113 starts counting when a speech interval is detected,
It is reset when the voice section ends. In the last word storage section 111 and the back pointer storage section 100, N~(k) and B~(k) are stored for k=1, 2, . . . , K. Segmentation part 11
4 issues an instruction to the back pointer storage section 110 to read a predetermined back pointer. That is, the segmentation section 114 is k
When a value of B to (k) is issued to the back pointer storage section 110, the back pointers B to (k) are read out from the back pointer storage section 110. When the segmentation unit 114 receives the values B to (k) from the back pointer storage unit 100, it issues the same values to the back pointer storage unit 110. Therefore, when the speech interval detection section 112 detects the end of speech input, the final value K of the vowel center counting section 113 is supplied to the segmentation section 114.
first issues a value K to the back pointer storage unit 110. Thereafter, according to the operations described above, the back pointer storage units 110B(K), B(B(K)),
..., O outputs are obtained sequentially. These values are the end frame of the second to last word, the third end frame of the last word, the fourth frame of the same word, etc. Since N~(k) is a word that ends in frame k, , if this value is directly supplied to the last word storage unit 111, recognition results will be obtained in the reverse order starting from the last word. To reverse this order (to the normal order), convert the order by using the output of the back pointer storage unit 110,
It is sufficient to perform this on the output of the last word storage section 111. FIG. 10 expresses the operation of the above embodiment as a program, and this may be followed when implementing it in software. In addition, in Figure 10,

[Table] The notation means that B is performed while condition A is satisfied. Also,

[Table] The notation means that B is performed until condition A is satisfied. Step 200 is a part for initializing the cumulative distance D~(k), the back pointer B~(k), D ⁿ (k-1:k), and D ⁿ (k-2:k). Step 201 shows processing at the center of the k-th vowel, which can be broadly divided into unitary cumulative matching distance D ⁿ
(k':k) part 202, word cumulative matching distance ^w (k':k) part 203, end cumulative distance D~(k), end back pointer B~(k), last word N~ It is divided into a part 219 for calculating (k). Step 202 is a part for calculating the segment cumulative matching distance D ⁿ (k':k) for n=1, 2, . . . , N, and corresponds to the operation performed in FIG. 1 103. Steps 204 and 205 are parts that give initial values for the calculation in step 206, Step 209 is a part that gives initial values for the calculation in step 211, Step 210 is a part that calculates the distance between vectors d ⁿ (i', j), 211 calculates the intermediate result D ⁿ (i', j) of the segment cumulative matching distance at the grid point (i, j). This embodiment shows the case of the path constraint condition shown in FIG. 2b. Step 207 replaces D ⁿ (k', 1) with D ⁿ (k':k) as the segment cumulative matching distance. This D ⁿ (k':k) is stored in the segment matching result storage section 109. Steps 204 to 207 are executed only when n satisfies the matching condition. That is, n
The preceding vowel of is Vf(n), the following vowel of n is Vr(n),
When the vowel recognition result centered on the vowel is V(k), V
(k-1)orV(k-2) = Vf(n), V(k) = Vr(n) and features such as consonants and silence between k-1 and k or k-2 and k are the standard pattern R Executed only when there is a possibility of matching ⁿ features. Step 207′ is max(St(k)−2(J ⁿ −1), 1}St(k′)max
It is executed only when {St(k)−[J ⁿ /2], 1}. Step 203 calculates the cumulative word matching distance ^w (k':k) for w = 1, 2, ..., W, the cumulative distance D ~ ^w (k) when w is the last word, and the back pointer B ~ ^w (k). )
This is the part that calculates , and corresponds to the operation performed by the candidate word determination section 121 and word matching section 106 in FIG. Step 203′ calculates DD(k′,
o), and step 203″ is the part where DD
When (k′, o)=0, ^w (k′:k)=(k′,
o), when DD (k′, o)≠0, ^w (k′:k)=
∞. Step 213 is a part that performs initialization when performing the calculation in step 214. Step 214 starts from the final segment s(w, L ^w ) of the segment series corresponding to the word w and calculates a series of standard patterns R ^s(w,Lw) , R corresponding to the first segment to the th segment. ^s(w,Lw-1) ,…
R ^{s(w, )} and subpatterns ^a St(k), ^a St of the input pattern
(k)-1, ^a St(k)-2, . . . , ^a St(k') is the part where the cumulative matching distance is calculated from the segment cumulative matching distance that has already been determined. However, in the recurrence formula of step 216,
s(w,) is the preceding vowel V(k″) and the following CV is the VCV syllable equal to the first syllable of w. Step 217.
217' is a part that substitutes the cumulative matching distance ^w (k', 1) or ∞ into the word cumulative matching distance ^w (k':k). This ^w (k':k) is stored in the word matching result storage section 107 in FIG. step
218 is the cumulative distance D~ ^w when w is the last word
(k) is the part for calculating the back pointer B~ ^w (k). Step 219 is the terminal cumulative distance calculation section 1 in FIG.
This corresponds to the operation performed in step 08, and is the part that solves recurrence formula 3 to obtain D~(k), N~(k), and B~(k). Steps 220 and 221 are determination processes for obtaining recognized word strings from B~(k) and N~(k) for k=1, 2, ..., K obtained in step 201, and are as shown in FIG. The back pointer storage section 110, the segmentation section 114, and the last word storage section 110 perform processing corresponding to the operations performed in the last word storage section 110. Effects of the Invention As described above, the present invention uses speech segments such as CV and CCV syllables as the unit of recognition, so no matter how many words there are, the standard pattern only needs to be registered with this speech segment. Since the word dictionary is represented as a series of these segment names, it requires much less memory than storing it as a series of feature vectors.
Most of the matching is spent on matching each of the above-mentioned fragments, and no matter how much the number of words increases, the amount of calculation increases only slightly. In addition, before performing DP matching, the standard elemental patterns to be matched among the above-mentioned elements are limited by using reliable information obtained regarding the vowel center, its recognition results, consonants, silence, etc. In addition, the words to be matched can be limited, and the amount of calculation can be extremely reduced. Furthermore, even if segmentation is slightly incorrect, DP matching will result in correct segmentation and recognition as a result of optimization, avoiding false recognition due to incomplete segmentation. . From the above, according to the present invention, it is possible to recognize continuously uttered words with high accuracy,
This is a highly practical device.

[Brief explanation of the drawing]

FIG. 1 is a diagram showing an embodiment of the present invention, and FIG. 2 is a diagram showing an embodiment of the present invention.
Figures 3 and 4 are diagrams explaining the principle of DP matching, Figures 5 and 6 are diagrams explaining the principle of the present invention.
7 and 7 are diagrams illustrating a part of the operation of the recognition method when the present invention is applied when the number of words is unknown, when the number of words is known, and when automaton control is used, respectively. The figure is a diagram explaining the principle of the main part of the embodiment of the present invention, and FIG. 10 is a diagram expressing the principle of the present invention in terms of software. 100...Speech signal input terminal, 101...Feature extraction section, 102...Speech segment standard pattern storage section, 103...Segment matching section, 104...Segment matching result storage section 105...Word dictionary, 1
06...Word matching unit, 107...Word matching result storage unit, 108...Terminal cumulative distance calculation unit, 109...Terminal cumulative distance storage unit, 110...
Back pointer storage section, 111...Last word storage section, 112...Speech section detection section, 113...Frame number counting section, 114...Segmentation section, 115...Recognition result output terminal, 116...Vowel Standard pattern storage section, 117...Vowel recognition section,
118... Vowel center detection section, 119... Silent interval detection/consonant classification section, 120... Feature series storage section, 121... Candidate word determination section.

Claims (1)

[Claims] 1. Feature extraction means that converts an input speech signal obtained by continuously uttering words, phrases, etc. into a series of feature vectors, and a means for extracting features corresponding to each speech element defined as a vowel, consonant, or a combination thereof. standard pattern storage means for storing a series of feature vectors in association with the phoneme segment names; segment boundary candidate detection means for detecting segment boundaries with respect to the input pattern; and each of the standard patterns and the input pattern. A element that calculates the distance (similarity) between the two by matching with a partial interval (first partial interval) determined by various arbitrary or predetermined combinations of the element boundary candidates detected from the element boundary candidate. a one-sided matching means; a word/phrase dictionary that stores words/phrases expressing each word/phrase, etc. to be recognized as a series of phoneme segment names; and each word/phrase to be recognized and the input. The distance (similarity) between the arbitrary or predetermined segment boundary candidates of the pattern and various subintervals (second subintervals) is calculated based on the sequence of segment names specified by the word/phrase dictionary. Correspondingly, by optimally determining the first sub-interval group included in the second sub-interval so that adjacent sections are continuous, the starting point and end point of each of the first sub-intervals and their portions can be determined. A function that minimizes (maximizes) the sum of distances (similarities) corresponding to the segment names of an interval and outputs the obtained minimum value (maximum value) as the distance of the second subinterval to each word/clause. By optimally determining the second partial interval group so that adjacent intervals are continuous, the second partial interval group is
A sequence in which the sum of the distances (similarities) corresponding to the start and end points of each subinterval and the word/phrase name in that subinterval is the minimum (maximum), and the word/phrase string at that time is determined as the recognition result. 1. A continuous speech recognition device comprising a word/phrase determination means.