JPH11259086A - Voice recognition method and voice recognition device - Google Patents

Abstract

(57) [Problem] To provide a speech recognition device having a high-performance speaker adaptation function by speaker adaptation of a phoneme standard pattern with a small amount of speaker adaptation learning speech data. SOLUTION: An acoustic analysis unit 101 converts an input speech into an acoustic parameter sequence, and stores a phoneme standard pattern for unspecified speakers stored in a phoneme standard pattern storage unit 103 for unspecified speakers and a phoneme similarity calculation. The collation is performed by the unit 102 and converted into a phoneme similarity time series. Pattern matching unit 104
Then, the phoneme similarity time series is time-matched with a word standard pattern as a feature parameter, and the adaptive learning pattern extraction unit 1
In step 08, a speaker adaptation learning sample pattern is extracted from the vicinity of the input speech frame corresponding to the high phoneme similarity frame in the word standard pattern, and the phoneme standard pattern adaptation unit 109 performs speaker learning of the phoneme standard pattern, and It is used at the time of recognition as a standard phoneme standard pattern.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech recognition method and apparatus having a speaker adaptation function.

[0002]

2. Description of the Related Art When a different speaker utters a voice, even if the same vocabulary is uttered, the characteristics of articulatory organs composed of the larynx, tongue, lips, etc. are different for each individual. Acoustic parameters obtained as a result of acoustic analysis of the signal have subtle differences depending on the characteristics of the articulator for each person. This is called the personality of the audio signal. In many speech recognition devices, as a feature parameter for identification,
Acoustic parameters are used that are susceptible to articulatory characteristics. The standard pattern of the speech recognition device for unspecified speakers is composed of average values of acoustic parameters learned from speech signals uttered by an unspecified number of speakers. For different speakers, there is a problem that the recognition performance deteriorates. Therefore, conventionally, in order to cope with such a problem based on the individuality of a speaker, an unspecified speaker standard pattern is adapted to a value suitable for the speaker from a voice signal uttered by the speaker.
A method has been adopted in which a speech recognition apparatus is provided with a speaker adaptation function to maintain recognition performance. As a speaker adaptation method for preventing a decrease in the recognition rate based on the individuality of the speaker, a method described in Japanese Patent Application Laid-Open No. H8-22296 is known. FIG. 3 shows a configuration of an example of a conventional method for coping with such a decrease in the recognition rate based on the individuality of a speaker by performing speaker adaptive learning of a standard pattern.

[0003] In the conventional method, an input speech is converted into an acoustic parameter series in an acoustic analysis unit 301.
As the acoustic parameters, for example, LPC cepstrum coefficients, LPC mel cepstrum coefficients, etc., which are described in Nakagawa, "Speech Recognition by Stochastic Model", IEICE (1988), are used. At the time of speaker adaptation, the word dictionary 303 sends, to the unspecified speaker's speech unit standard pattern storage unit 304, speech segment information indicating what speech segment the word corresponding to the speaker learning input speech is composed of. . Here, as a unit of the speech piece, a phoneme bigram considering the following phoneme environment, a phoneme trigram considering the preceding and succeeding phoneme environments, and the like can be considered. In the unspecified speaker's speech unit standard pattern storage unit 304, based on the speech unit information sent from the word dictionary 303, the unspecified speaker's word standard pattern is created by connecting speech unit standard patterns composed of acoustic parameters. And the pattern matching unit 30
2 to the speech unit standard pattern adaptation unit 305. In the pattern matching unit 302, the time series of the acoustic parameter time series obtained from the input voice and the unspecified speaker word standard pattern are time-matched, and the time correspondence result is calculated. As a method of time matching in the pattern matching unit 302, for example, DP
Matching, HMM (Hidden Markov M)
model) is used. At the time of speaker adaptation, the pattern matching unit 302 sends the audio parameter time series of the input speech, the time correspondence result of the word standard pattern for unspecified speakers, and the audio parameter time series of the input speech to the speech piece standard pattern adaptation unit 305. Sent. The speech piece standard pattern adaptation unit 305 adapts the word standard pattern for a specific speaker by a method such as linear combination of the acoustic parameter time series of the input speech and the word standard pattern. The word standard pattern after the adaptation is divided into speech unit units in the speech unit standard pattern adaptation unit, and the speech unit standard pattern storage unit 3 for the specific speaker is used.
06. At the time of recognition, a specific speaker word standard pattern is formed for all the words in the word dictionary 303 using the specific speaker voice segment standard pattern stored in the specific speaker voice segment standard pattern storage unit 306. Then, the pattern matching unit 302 performs a time collation calculation with the acoustic parameter time series of the input speech, and outputs the dictionary word having the highest collation score as the final recognition result.

[0004]

However, in a conventional method for adapting a speaker for a specific speaker by time collation of a speech unit standard pattern represented by acoustic parameters with a time series of acoustic parameters of input speech. If the vocabulary of the word dictionary to be recognized is large and the type of speech vocal standard pattern necessary to compose the word standard pattern is large, for all the vocabularies, the speaker adaptation learning is performed for the specific speaker. In order to prepare a speech unit standard pattern, the speaker needs to utter a large amount of training sample speech for speaker adaptation learning, and utters enough to learn all the speech unit standard patterns. That puts a heavy burden on the speaker. Also, when a speaker performs vocalization for huge speaker adaptation learning, if a large amount of memory is required for storing a speech unit standard pattern, an additional speech unit standard pattern for a specific speaker is required. A large amount of memory is required, and the amount of memory required for the speech recognition device itself becomes large. further,
In the conventional method, when a speech unit standard pattern other than utterance is adapted from a limited small amount of speaker adaptation learning speech data, for example, the Institute of Electronics, Information and Communication Engineers SP92-16 (1
In such a case, a device such as VFS (Vector Field Smoothing) described in U.S. Pat.

An object of the present invention is to solve the above-mentioned problem, and to provide a high-performance speech recognition apparatus capable of realizing efficient speaker adaptation with a small amount of memory and with a small amount of speaker adaptation training data. With the goal.

[0006]

A speech recognition method according to the present invention obtains m feature parameter sequences for each analysis time for an input speech, performs matching with n types of phoneme standard patterns for each analysis time, In the speech recognition method for recognizing a word by matching n similarities obtained for each analysis time with a word standard pattern composed of a time series of phoneme similarities created in advance, the word standard pattern A frame having a high phoneme similarity is extracted from the input speech, and an input speech frame corresponding to a frame having a high phoneme similarity of the word standard pattern is obtained from the result of the time correspondence obtained by matching the input voice and the word standard pattern. The feature parameters of the input speech of the frame are extracted as phoneme standard pattern training data, and the phoneme standard pattern and the phoneme standard pattern learning are extracted. Integrating the chromatography data is intended to create a new phoneme standard patterns.

Thus, with a small amount of memory,
It is possible to provide a high-performance speech recognition device that can realize efficient speaker adaptation even with a small number of speaker adaptation learning data.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention according to claim 1 of the present invention provides a speech recognition system in which a phoneme similarity obtained by collating an input speech with a phoneme standard pattern for an unspecified speaker is used as a feature parameter for recognition. In the method, speaker adaptation learning is performed on the phoneme standard pattern, and the phoneme similarity obtained by matching the phoneme standard pattern with the acoustic parameter time series is used as a feature parameter at the time of word pattern matching. Performing speaker adaptation learning has the effect that efficient speaker adaptation is possible even with a small amount of speaker adaptation learning data.

According to a second aspect of the present invention, m feature parameter sequences are obtained for an input voice at each analysis time, and matching is performed with n types of phoneme standard patterns at each analysis time. In the speech recognition method for recognizing a word by matching the n similarities and a previously created word standard pattern composed of a time series of phoneme similarities, the phoneme similarity is calculated from the word standard pattern. A high frame is extracted, and an input voice frame corresponding to a frame having a high phoneme similarity of the word standard pattern is obtained from a time correspondence result obtained by matching the input voice with the word standard pattern. The feature parameters are extracted as phoneme standard pattern learning data, the phoneme standard pattern and the phoneme standard pattern learning data are integrated, and a new Intended to create a phoneme standard pattern, even in the case of small amounts speaker adaptive training for the voice data,
Alternatively, even if a large amount of memory is not available for realizing the speaker adaptation function, the phoneme similarity obtained by matching the phoneme standard pattern with the acoustic parameter time series is used as a feature parameter for word pattern matching. By performing the speaker adaptation learning of the phoneme standard pattern, there is an effect that efficient speaker adaptation can be performed with a small amount of memory and with a small amount of speaker adaptation learning data.

According to a third aspect of the present invention, m feature parameter sequences are obtained for each analysis time with respect to an input voice, matching is performed with n kinds of phoneme standard patterns at each analysis time, and each feature time is obtained for each analysis time. In the speech recognition device that recognizes a word by matching the n similarities and a previously created word standard pattern composed of a time series of phoneme similarities, the phoneme similarity is calculated from the word standard pattern. A high phoneme similarity frame information extracting means for extracting a high frame, and an input speech frame corresponding to a frame having a high phoneme similarity of a word standard pattern are obtained from a time correspondence result obtained by matching the input speech with the word standard pattern. Adaptive learning pattern extracting means for extracting feature parameters of the input speech of this frame as phoneme standard pattern learning data; It integrates the quasi-pattern and the phoneme standard pattern learning data, and includes phoneme standard pattern adaptation means for creating a new phoneme standard pattern, even when the speaker adaptation learning speech data is small, or The phoneme standard is used even when a large amount of memory is not available to implement the speaker adaptation function, or when the configuration of the speech recognizer cannot be complicated to implement the speaker adaptation function. By using the phoneme similarity obtained by matching the pattern with the acoustic parameter time series as a feature parameter at the time of word pattern matching and performing speaker adaptation learning of phoneme standard patterns, a small amount of memory can be used and a small amount of speaker adaptation learning can be performed. It has the effect that efficient speaker adaptation is possible even with data.

Hereinafter, embodiments of the present invention will be specifically described with reference to the drawings. (Embodiment) FIG. 1 is a block diagram showing a speech recognition apparatus according to an embodiment of the present invention. In FIG. 1, 10
Reference numeral 1 denotes an acoustic analysis unit that converts an input voice into acoustic parameters for each analysis time, and 102 collates the acoustic parameters obtained by the acoustic analysis unit 101 with a phoneme standard pattern for unspecified speakers prepared for each phoneme type. Phoneme similarity calculator,
103 is a phoneme standard pattern storage for an unspecified speaker that stores the obtained acoustic parameter time series, 104 is a pattern matching unit that performs time matching with a word standard pattern using a phoneme similarity time series as a feature parameter, and 105 is a word A dictionary 106 is a speech unit standard pattern storage unit for an unspecified speaker connecting a speech unit standard pattern composed of phoneme similarities learned in advance based on the speech unit information sent from the word dictionary 106, and 107 is a phoneme similarity A high phoneme similarity frame information extraction unit 108 for extracting a frame having a high phoneme similarity to any one of the phonemes from a word standard pattern composed of a time series, and an input 108 corresponding to the high phoneme similarity frame in the word standard pattern An adaptive learning pattern extraction unit for extracting a speaker adaptive learning sample pattern from the vicinity of a voice frame; Phoneme standard pattern adaptation unit for performing a turn speaker learning, 110 is a speaker-for the phoneme standard pattern storage section for storing phoneme standard patterns for a specific speaker.

The operation of the speech recognition apparatus configured as described above will be described below. First, the case of speaker adaptation will be described.

The acoustic analysis unit 101 converts an input voice into an analysis time (called a frame, in this embodiment, one frame = 10 ms).
For each ec), a 10-dimensional LPC cepstrum sequence, a time difference in power, and a normalized residual are converted into a total of 12-dimensional acoustic parameters.

The phoneme similarity calculation unit 102 includes a sound analysis unit 1
01 is compared with the phoneme standard pattern for unspecified speakers prepared for each phoneme type, which is stored in advance in the phoneme standard pattern storage unit 103 for unspecified speakers. Here, phonemes are defined as ｛a, o, u, i, e, z, s, hv, hu,
p, t, k, c, b, d, N, j, w, yv, yu,
A 24 phoneme classification of m, n, ng, r｝ is used. In addition, each phoneme standard pattern accurately detects a feature frame defined for each phoneme (temporal position at which the feature of the phoneme is well expressed) by visual observation and obtains a time pattern of acoustic parameters centered on the feature frame. Create using In the present embodiment, as the time pattern, the neighborhood 1 of the feature frame is used.
A total of 120-dimensional (12-dimensional acoustic parameters × 10 temporal patterns) patterns are formed by the acoustic parameters for 0 frames, and a phoneme standard pattern is learned in advance from the utterance data of an unspecified number of speakers.
The phoneme similarity is obtained using a Mahalanobis distance in which a covariance matrix is shared.

The phoneme similarity time series obtained as a result of comparing the acoustic parameter time series of the input speech with the phoneme standard pattern for an unspecified speaker is the dimension of the phoneme type (24 dimensions in this embodiment).
And sent to the pattern matching unit 104 as a feature parameter during word pattern matching.

The word dictionary 105 stores speech segment information indicating what speech segment the word corresponding to the input speech for speaker learning is composed of, based on a phoneme similarity time series, a speech segment for an unspecified speaker. The standard pattern is sent to the unspecified speaker voice segment standard pattern storage unit 106. Here, in the present embodiment, there are 536 types of speech piece units, which are CV / VC units in consideration of the preceding and succeeding phoneme environments.

Speech fragment standard pattern storage unit 1 for unspecified speakers
In step 06, based on the speech piece information sent from the word dictionary 105, a speech piece standard pattern consisting of phoneme similarities learned in advance is connected, and the word standard pattern is combined with the pattern matching section 104 and the high phoneme similarity frame information extraction section. 1
Send to 07.

In the pattern matching unit 104, the time series of the phoneme similarity obtained from the input speech and the word standard pattern are time-matched by DP matching, and a time correspondence result is calculated.

The high phoneme similarity frame information extraction unit 107 extracts a frame having a high phoneme similarity to any one of the phonemes from a word standard pattern composed of a phoneme similarity time series, and determines which phoneme is high in which frame. It is sent to the adaptive learning pattern extraction unit 108 as high phoneme similarity frame information indicating whether or not it has phoneme similarity. In the present embodiment, the high phoneme similarity frame information extraction unit 107
All frames in which one of the phoneme similarities to the phonemes was higher than the threshold value Th were defined as high phoneme similarity frames. The optimum value of the threshold value Th was obtained by an experiment.

In the adaptive learning pattern extraction unit 108,
The input speech frame corresponding to the high phoneme similarity frame in the word standard pattern is calculated from the high phoneme similarity frame information in the word standard pattern and the time correspondence result between the input voice and the word standard pattern, and the input near this frame is calculated. The acoustic parameter pattern of the voice is extracted as a speaker adaptation learning sample pattern, and sent to the phoneme standard pattern adaptation unit 109.

The phoneme standard pattern adapting unit 109 stores the speaker adaptive learning sample pattern from the adaptive learning pattern extracting unit 108 and the phoneme standard pattern storage unit 10 for unspecified speakers.
The phoneme standard pattern for specific speaker is calculated from the phoneme standard pattern for unspecified speaker from No. 3 and stored in the phoneme standard pattern storage unit for specific speaker 110. In this embodiment,
The average value (Equation 1) of the phoneme standard pattern for a specific speaker is

[0022]

(Equation 1)

The average of the phoneme standard pattern for unspecified speakers before adaptation (Equation 2);

[0024]

(Equation 2)

As a linear combination of the speaker adaptive learning sample pattern values (Equation 3),

[0026]

(Equation 3)

(Equation 4) is calculated.

[0028]

(Equation 4)

Here, α is a mixing ratio of a linear combination, and an optimum value by an experiment is used.

Next, the processing at the time of recognition will be described. The acoustic analysis unit 101 converts the input voice into one every analysis time (called a frame, one frame = 10 msec in this embodiment).
0-dimensional LPC cepstrum sequence, power time difference,
The normalized residuals are converted into a total of 12-dimensional acoustic parameters. The phoneme similarity calculation unit 102 uses the acoustic parameter time series obtained by the acoustic analysis unit 101 and the phoneme standard pattern for a specific speaker stored in the phoneme standard pattern storage unit for a specific speaker 110 to calculate the phoneme similarity. Calculate the series.

The pattern matching unit 104 performs a matching calculation with the word standard pattern from the unspecified speaker voice unit standard pattern storage unit 106, and outputs a dictionary word having the highest matching score as a final recognition result.

FIG. 2 is a conceptual diagram of a phoneme similarity time series showing the effect of speaker adaptation in the present embodiment.
MA (Zama) ", the temporal change of the phoneme similarity of the phoneme / a /. The upper part a) shows the time change of the word standard pattern created by connecting the unspecified speaker speech piece standard patterns, and the dotted line in the lower part b) shows the input speech and the phoneme standard pattern / a / for the unspecified speaker. The phoneme similarity time series obtained by collation, the solid line in the lower part b) is a phoneme similarity time series obtained by collating the input voice with the specific speaker phoneme standard pattern / a / after adaptation. The frame (Equation 5) which is one of the high phoneme similarity frames in the word standard pattern has a high phoneme similarity.

[0033]

(Equation 5)

The input frame (equation 6) corresponding to (equation 5) by DP matching is a phoneme similarity time series obtained by collating the input speech with the phoneme standard pattern for unspecified speakers / a / The similarity is smaller than 5).

[0035]

(Equation 6)

On the other hand, in the phoneme similarity sequence obtained by collation with the phoneme standard pattern / a / for a specific speaker, which has undergone speaker learning based on the present embodiment, the input frame (equation 7) corresponding to (equation 5) ) Has a high phoneme similarity as in (Equation 5),
The speaker adaptation effect according to the present embodiment can be confirmed.

[0037]

(Equation 7)

Using the configuration of the present embodiment, an experiment for recognizing voice data of 11 people who uttered 100 words was performed.
The speaker adaptation learning of the phoneme standard pattern was performed using 20 words different from the evaluation utterance data. The average recognition rate of 11 persons was 93.2% before performing the speaker adaptation based on the present embodiment, whereas the average recognition rate after the speaker adaptation based on the present embodiment was 93.2%.
The recognition rate was improved to 6.1%, and the error rate was improved by about 40%.

In the present embodiment, a time pattern of acoustic parameters is used for calculating a phoneme similarity which is a feature parameter at the time of word matching, and a Mahalanobis distance with a common covariance matrix is used as a distance measure. But HM
The phoneme similarity can also be calculated from the phoneme model composed of M.

In this embodiment, the recognition target is
Although a word is used, it can be used when recognizing a continuous utterance.

[0041]

According to the present invention, the speaker adaptation learning of the phoneme standard pattern is performed by using the phoneme similarity obtained by collating the phoneme standard pattern with the acoustic parameter time series as a feature parameter at the time of word pattern matching. Accordingly, an effect is obtained that a high-performance speech recognition device that can realize efficient speaker adaptation with a small amount of memory and a small amount of speaker adaptation learning data can be provided.

[Brief description of the drawings]

FIG. 1 is a block diagram of a speech recognition apparatus according to an embodiment of the present invention.

FIG. 2 is a conceptual diagram showing a phoneme similarity time series in the embodiment of the present invention.

FIG. 3 is a block diagram showing a conventional speech recognition apparatus.

[Explanation of symbols]

 Reference Signs List 101 acoustic analysis unit 102 phoneme similarity calculation unit 103 phoneme standard pattern storage unit for unspecified speaker 104 pattern matching unit 105 word dictionary 106 speech unit standard pattern storage unit for unspecified speaker 107 high phoneme similarity frame information extraction unit 108 Adaptive learning pattern extraction unit 109 Phoneme standard pattern adaptation unit 110 Phoneme standard pattern storage unit for specific speaker

Claims (3)

[Claims] 1. A speech recognition method in which a phoneme similarity obtained by matching an input speech with a phoneme standard pattern for an unspecified speaker is used as a feature parameter of recognition. A voice recognition method characterized by performing. 2. An m-number of feature parameter sequences are obtained for each analysis time with respect to an input voice, matching is performed with n types of phoneme standard patterns at each analysis time, and n is obtained for each analysis time.
In a speech recognition method for recognizing a word by matching a word standard pattern composed of a time series of phoneme similarities prepared in advance and a phoneme similarity, a frame having a high phoneme similarity from the word standard pattern is used. Is extracted, and an input speech frame corresponding to a frame having a high phoneme similarity of the word standard pattern is obtained from the time correspondence result obtained by matching the input speech with the word standard pattern, and the characteristic parameters of the input speech of this frame are obtained. Is extracted as phoneme standard pattern learning data, and the phoneme standard pattern and the phoneme standard pattern learning data are integrated to create a new phoneme standard pattern. 3. An m-number of feature parameter sequences are obtained for each analysis time with respect to an input voice, matching is performed with n kinds of phoneme standard patterns at each analysis time, and n is obtained for each analysis time.
In a speech recognition apparatus for recognizing a word by matching a word standard pattern composed of a time series of phoneme similarities prepared in advance and a phoneme similarity, a frame having a high phoneme similarity from the word standard pattern A high-phoneme-similarity frame information extracting means for extracting the input speech frame corresponding to a frame having a high phoneme-similarity of the word standard pattern from the result of the time correspondence obtained by matching the input speech with the word standard pattern. Adaptive learning pattern extraction means for extracting feature parameters of the input speech of the frame as phoneme standard pattern learning data, and a phoneme standard for integrating the phoneme standard pattern and the phoneme standard pattern learning data to create a new phoneme standard pattern A speech recognition device comprising: a pattern adaptation unit.