JPH03157697A - Word recognizing system - Google Patents

Abstract

PURPOSE:To obtain the recognizing system which extremely lessens the deterioration in a recognition rate with lapse of time and allows easy real time processing by comparing a recognition output with the threshold and executing additional learning when the output is below the threshold. CONSTITUTION:An input voice is processed in a voice preprocessing section 12 and is supplied to a hierarchical neutral network 13 and is stored into a memory section 15 so as to be used for additional learning. The voice is recognized in the network 13 and is outputted via a deciding section 14. This deciding section 14 decides a registered word and outputs this word when the recognition output is higher than the threshold. The deciding part outputs the additional learning signal and corrects the conversion function and weight coefft. of the network 13 via a control section 16 when the output is below the additional learning threshold higher than the threshold. The word recognizing system which can use the hierarchical neutral network by the automatic learning processing, extremely lessens the deterioration of the recognition rate with lapse of time and allows the easy real time processing is thus obtd.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a word recognition system suitable for voice operation of home appliances.

[Prior art] A conventional word recognition system is disclosed in Japanese Patent Application Laid-Open No. 63-229496.
According to the following procedure as described in .

■Extract features related to words contained in input speech.

■Calculate the distance to the dictionary extracted in advance using the same method as in (■) above.

■ Based on the calculation result of (■) above, determine which word the input voice is registered in the dictionary.

■Inform the input speaker of the recognized word. The input speaker checks whether there is any misrecognition by the word recognition system.

■If misrecognition is found through the check in (■) above, the feature amount of the word that was misrecognized this time is additionally registered as a new standard pattern by manual input by the input speaker.

As a result, for one registered word, additionally registered standard patterns coexist in addition to already registered standard patterns.

[Section M to be invented or solved] However, in the above conventional word recognition system, the following ■
There are ~■ problems.

■The recognition rate deteriorates as time passes from the time the standard pattern was created.For example, after 3 months have passed, the recognition rate deteriorates to 1.
It deteriorates from 00.0% to 85.0%.

■Real-time processing is difficult. In other words, in order to ensure a recognition rate above a certain level, it is necessary to use complex features;
In order to extract complex feature quantities, a complex processing device is required, and the processing time is also large.

- In order to prevent recurrence of misrecognition, memory capacity for the standard pattern is required each time a standard pattern is additionally registered for a registered word related to misrecognition.

■As a result of additionally registering standard patterns in the above (■), the number of patterns to be calculated for distance during recognition increases, and the recognition processing time increases.

An object of the present invention is to obtain a word recognition system that exhibits extremely little deterioration in recognition rate over time and that can be easily processed in real time.

In addition, the present invention does not require an increase in memory capacity or an increase in recognition processing time when additionally learning registered words related to unrecognizability to prevent recurrence of unrecognizability, and can easily be performed in real time. The purpose is to obtain a word recognition system that can process.

[Means for Solving the Problems] The present invention according to claim 1 is a word recognition system using a neural network, in which a word recognition threshold value is set for an output value of an output unit corresponding to a registered word. and a threshold for additional learning, and on the condition that the above output value is greater than the threshold for word recognition, the current input voice word is determined to be a registered word, and the above output value is determined as a registered word. Additional learning of the neural network is performed using the current input audio data, provided that it is greater than the threshold and smaller than the additional learning threshold.

The present invention according to claim 2 is a word recognition system using a neural network, in which a word recognition threshold is set for an output value of an output unit corresponding to a registered word, and the output value is On the condition that the output value is greater than the word recognition threshold, the current input audio word is determined to be a registered word, and on the condition that the output value is smaller than the word recognition threshold, the current input audio word is determined as a registered word. What has not been recognized is displayed, and in response to this display, the neural network performs additional learning using the current input word data taught by the current speaker and the current input voice data. .

The present invention according to claim 3 is a word recognition system using a neural network, which sets a word recognition threshold value and an additional learning threshold value to an output value of an output unit corresponding to a registered word. (A) The output value is smaller than the word recognition threshold, and (A) the output value is smaller than the word recognition threshold. Under the condition, it is displayed that the current input voice word could not be recognized, and in response to this display, the neural network is constructed using the current input word data taught by the current speaker and the current input voice data. Perform additional learning, and (B) perform additional learning of the neural network using the current input audio data, on the condition that the above output value is greater than the word recognition threshold and smaller than the additional learning threshold. This is what I decided to do.

The present invention according to claim 4 provides, as inputs to the neural network, (1) temporal changes in the frequency characteristics of the voice, (2) average linear prediction coefficients of the voice, (2) average PARCOR coefficients of the voice, and (2) the average PARCOR coefficient of the voice. Average frequency characteristics and pitch frequency, ■Average frequency characteristics of high-frequency emphasized audio waveforms,
and (1) one or more of the average frequency characteristics of voice is used.

According to a fifth aspect of the present invention, the neural network is a hierarchical neural network.

[Function] According to the present invention as set forth in claim 1, the following (1) to (5)
It has the function and effect of

(1) The deterioration of the recognition rate over time is extremely small. This is confirmed by the experimental results described below, and this is because the neural network has a structure that is not easily affected by fluctuations due to differences in speech timing. This is presumed to be because it is possible.

(2) In addition to the word recognition threshold, an additional learning threshold is provided for the output value of the output unit corresponding to the registered word, which constitutes the neural network. In other words, even if the output value is greater than the word recognition threshold and the input speech word can be determined as a registered word, the output value is greater than the 1i35 recognition threshold. If the additional training threshold is not exceeded, additional training is performed on the neural network using the current input audio data. This allows the neural network to be constantly updated before the recognition rate of the neural network deteriorates. As a result, it is possible to construct a word recognition system in which the deterioration of the recognition rate over time is extremely small.

(3) In principle, in a neural network, the calculation processing of the entire network is simple and quick.

(4) In principle, each unit constituting a neural network operates independently, and parallel arithmetic processing is possible. Therefore, calculation processing is quick.

(5) With the above (3) and (4), the word recognition system can be easily processed in real time without using a complicated processing device.

Further, according to the present invention as set forth in claim 2, the above (1)
In addition to the effects (3) to (5), there is the effect (6) below.

(6) When the current input voice word is unrecognizable by the neural network, the current input word data is taught with the help of the input speaker, and additional learning is performed on the neural network, thereby preventing recurrence of unrecognizability. . At this time, additional learning is performed by modifying the conversion function and weight of each unit of the 221 ral network, so there is no need to increase memory capacity or recognition processing time. .

According to the present invention according to claim 3, the above (1) to (6)
It has the function and effect of

According to the present invention according to claim 4, the above (1) to (6)
In addition to the above effects, there is the following effect (7).

(7) Since one or more of the elements ① to ② described in claim 4 are used as input to the neural network, the preprocessing for obtaining the input is different from conventional complex feature extraction. Therefore, the process is simple, and the time required for this preprocessing is short.

According to the present invention according to claim 5, the above (1) to (7)
In addition to the effects described above, there is the effect described in (8) below.

(8) Regarding hierarchical neural networks, a simple learning algorithm (pack propagation) as described later has been established, and it is possible to easily create a neural network that can achieve a high recognition rate.

[Example] Fig. 1 is a schematic diagram showing an example of a word recognition system to which the present invention is applied, Fig. 2 is a schematic diagram showing an example of a speech processing unit and a neural network, and Fig. 3 is a schematic diagram showing an input speech. Figure 4 is a schematic diagram showing the output of the bandpass filter,
Figure 5 is a schematic diagram showing a neural network, Figure 6 is a schematic diagram showing a hierarchical neural network, and Figure 7 is a schematic diagram showing a hierarchical neural network.
The figure is a schematic diagram showing the structure of the unit.

Prior to describing specific embodiments of the present invention, the configuration of the neural network and the learning algorithm will be described.

U) Neural networks can be roughly divided into two types based on their structure: the hierarchical network shown in FIG. 5(A) and the interconnected network shown in FIG. 5(B).
Although it may be configured using either of the two networks, the hierarchical network is more useful because it has a simple learning algorithm as described below and has been established.

(2) Network Structure A hierarchical network has a hierarchical structure consisting of an input layer, an intermediate layer, and an output layer, as shown in FIG.

Each layer is composed of one or more units. The connections are made in the forward direction from the input layer to the middle layer to the output layer, and there are no connections within each layer.

(3) Structure of the unit The unit is a model of a neuron in the brain and has a simple structure as shown in FIG. It receives input from other units, sums it up, transforms it using a certain rule (conversion function), and outputs the result. Each connection with another unit is given a variable weight that represents the strength of the connection.

(4) Learning (pack propagation) Network learning means bringing the actual output closer to the target value (desired output). Learn by making changes.

Further, as a learning algorithm, for example, Rumel
hart, D.E.', McCelland,
J, L, and thePDP Re5ea
rch Group, PARALLEL DI
STRIBUTED PROCESSING, the
MIT Press, 1986. Pack propagation as described in .

Hereinafter, specific examples of the present invention will be described.

As shown in FIG. 1, the word recognition system 10 includes a voice input section 11, a voice processing section 12, and a neural network 13.
, determination unit 14, memory unit 15, network control unit 16
, a device control section 17, a display control section 18, a display section 19, and a teaching section 20.

(1) Input the registered voice to the voice input section 11.

At this time, learn words such as "shoumei", "air conditioner",
The five words are "curtain,""television," and "door."

Furthermore, input words are assumed to be five single words: "shoumei", "air conditioner", "curtain", "television", and "door".

(2) The audio processing unit 12 performs simple preprocessing on the input audio in (1) above.

The preprocessing results are transferred to the neural network 13 for the current word recognition, and are also transferred to the memory unit 15 in preparation for the possibility of additional learning.

(3) The neural network 13 performs the following learning operation (2) and the following evaluation operation (2).

■Learning' Set the target output value of the output unit corresponding to the Q-recorded word to (1
), the input speech of a specific speaker characterized by the target output value of the uncorresponding output unit is subjected to preprocessing by the speech processing unit 12, and the preprocessing result is input to the neural network 13. Then, the conversion function and weight of each unit of the neural network 13 are corrected so that the output value of the neural network 13 (the output value of each output unit constituting the output layer) approaches the target value.

This learning operation is repeated, for example, 1000 times.

■Evaluation This time, preprocessing is performed on the input speech of the speaker, and the preprocessed speech is input to the neural network 13 to obtain the output value of the neural network.

Then, the output value (X) of the output unit corresponding to each registered word of the neural network 13 is transferred to the determination unit 14.

(4) The determination unit 14 sets threshold values θ1 and θ2 (θ1>θ2) for the output value (X) of the neural network 13.
will be established.

Ol is a threshold for additional learning, and 02 is a threshold for word recognition.

The determination unit 14 performs the following determination operations (1) to (2) using the above threshold value.

(2) Under the condition that [X>θ2], the determining unit 14 determines the current input voice word as a registered word, and outputs this registered word determination signal to the device control unit 17.

■ [X<02] On the condition that
Output to 8.

(2) Only when determining registered words in (2) above, the determination unit 14 further performs the following processes (a) and (b).

(a) On the condition that [X<θ1], the determination unit 14 outputs an additional learning execution signal to the network control unit 16 in order to perform additional learning of the neural network 13 using the current input audio data. .

(b) [X>01] When this is the case, the determination unit 14 does nothing.

(5) The device control unit 17 controls the device using the registered word determination signal based on the determination result of the above-mentioned (2) by the determination unit 14.

This device is, for example, a lighting fixture, and performs lighting control based on the registered word determination signal.

(6) The network control unit 16 determines to perform additional learning of the neural network 13 based on the additional learning execution signal based on the determination result of the above-mentioned (2) by the determining unit 14. At this time, the network control unit 16 controls the memory unit 1
5, extract the current input audio data, re-input this input audio data to the neural network 13, and calculate the output value of the neural network 13 for this input (X
) is closer to the target value (1) for the registered word in (3) (3) above, which corresponds to the currently recognized input word. The network control unit 16 repeats this additional learning operation, for example, 1000 times.

(7) The display control unit 18 drives the display unit 19 based on the unrecognized signal based on the determination result of the above-mentioned (■) by the determination unit 14 to display that the current input voice word has not been recognized, and to display the unrecognized word. make the person aware.

At the same time, the display control section 18 drives the display section 19 to sequentially display all registered words (the five words mentioned above). In contrast,
The present speaker manually operates the teaching unit 20 to identify the input gt word that he or she has input this time and is unrecognizable from the displayed word data on the display unit 19, and transfers this input word data to the display control unit 18. Send to.

The display control unit 18 determines the input word data transmitted from the teaching unit 20 and outputs an additional learning execution signal for the unrecognized word to the network control unit 16.

(8) The network control unit 16 determines to perform additional learning of the neural network 13 based on the additional learning execution signal based on the control result of the above (7) by the display control unit 18. At this time, the network control unit 16 transfers the current human voice data to the neural network 1 based on the current input word data taught by the current speaker and the current input voice data retrieved from the memory unit 15.
3, and the output value (X) of the neural network 13 for this input is set to the target value (
The conversion function and weight of each unit of the neural network 13 are modified so as to approach 1). The network control unit 16 repeats this additional learning operation, for example, 1000 times.

Hereinafter, as shown in FIG. 2, a specific example will be described in which a hierarchical neural network 13 is used and temporal changes in the average frequency characteristics of audio within a certain period of time are used as input to the neural network 13. do.

Note that the audio processing section 12 includes a low-pass filter 21, a band-pass filter 22, and an averaging circuit 23, as shown in FIG.
It is composed of the combination of

(2) The high-frequency components of the audio signal of the input audio are cut by the low-pass filter 21. Then, this input audio is temporally equally divided into four blocks as shown in FIG.

■As shown in Fig. 2, the audio waveform is passed through a band pass filter 22 of multiple (n) channels, and the sound waveform is passed through the band pass filter 22 of multiple (n) channels, and the waveform is filtered at each block, that is, at each fixed time interval, as shown in Figs. 4 (A) to (D). Obtain frequency characteristics.

At this time, the output signal of the bandpass filter 22 is averaged by the averaging circuit 23 for each block, that is, for a certain period of time.

Through the above preprocessing, the "temporal change in the average frequency characteristics of audio within a certain period of time" was obtained.

The output of the averaging circuit 23 is transferred directly to the neural network 13, or indirectly transferred to the neural network 13 via the memory section 15.

■The neural network 13 is composed of a three-layer hierarchical neural network. Input Ji131 is 4Xn corresponding to 4 blocks of preprocessing and n channels.
It is composed of units. The output layer 32 is composed of 5 units corresponding to the number of registered words (5 words) described above.

The target value of the output 132 is (lO, O, 0, 0) for the output unit corresponding to the registered word "Shoumei", "
For the output unit corresponding to "air conditioner" (0, 1
, 00, 0), for the output unit corresponding to "Curtain" (0, 0, 1, 0, 0), for the output unit corresponding to "r TV" (0, 0, 0, 1゜0)
, for the output unit corresponding to "door" (0,0
,0,0,1).

Experiment (1) When input was made by a specific speaker (one person) who was the speaker during learning, the recognition rate was 100%.

(2) When input by unspecified speakers (20 people), the recognition rate was 92.0% without the above-mentioned additional learning.

(3) When input by unspecified speakers (20 people), the recognition rate was 99.0% with the above-mentioned additional learning.

Next, the operation of the above embodiment will be explained.

(1) The deterioration of the recognition rate over time is extremely small. This is confirmed by the experimental results described later, and the neural network 13 has a structure that is not easily affected by fluctuations due to differences in the timing of speech. It is presumed that this is because it is possible.

(2) In addition to the word recognition threshold, an additional learning threshold is provided for the output value of the output unit corresponding to the registered word constituting the neural network 13. That is,
Even if the above output value is larger than the word recognition threshold and the input speech word can be determined as a registered word, additional learning is performed if the output value is larger than the word recognition threshold. If the input audio data does not exceed the threshold value, the neural network 13 performs additional learning using the current input audio data. As a result, the neural network 13 is constantly updated before the recognition wt rate of the neural network 13 deteriorates, and as a result, a word recognition system can be constructed in which the recognition rate deteriorates very little over time.

(3) In principle, the neural network 13 has simple and quick arithmetic processing for the entire network.

(4) In principle, the neural network 13 operates independently of each of its constituent units, and is capable of parallel arithmetic processing. Therefore, calculation processing is quick.

(5) According to (3) to (4) above, the word recognition system 1
0 can be easily processed in real time without using a complicated processing device.

(6) When the current input speech word is unrecognizable in the neural network 13, the current input word data is taught with the help of the input speaker, and the neural network 13 performs additional learning to prevent the recurrence of unrecognizability. It can be prevented. At this time, since the additional learning is performed by modifying the conversion function and weight of each unit of the neural network 13, there is no need to increase memory capacity or recognition processing time.

(7) As an input to the neural network 13, “
Because we use "temporal changes in the frequency characteristics of audio," the preprocessing to obtain input is more complex than conventional feature extraction.
It is simple and the time required for this preprocessing is short.

At this time, as an input to the above neural network,
Furthermore, since the "temporal change in the average frequency characteristic within a certain period of time" is used, the processing in the neural network 13 is simple, and the time required for this processing is shorter.

(8) Since the hierarchical neural network 13 is used, a simple learning algorithm that has already been established (
Pack propagation) can be used to achieve high recognition rates.

In the implementation of the present invention, as inputs to the neural network, ■temporal changes in the frequency characteristics of audio, ■average linear prediction coefficients of audio, ■average PARCOR coefficients of audio, ■average audio frequency characteristics and pitch frequency, ■Average frequency characteristics of high-frequency emphasized audio waveforms,
and ■ one or more of the average frequency characteristics of voice can be used.

Then, just as the element (■) above was further used as "temporal change in the average frequency characteristics within a certain period of time", the element (■) above is also used as "the average linear prediction coefficient within a certain period of time". "Temporal change in
``Temporal change in the average PARCOR coefficient within a certain time period of audio'', the above element (■) is ``the average frequency characteristic and temporal change in pitch frequency within a certain time period of audio'', and the above element (■) is: It can be used as a "temporal change in the average frequency characteristics within a certain period of time of a voice waveform that has been subjected to high-frequency emphasis."

Incidentally, the linear prediction coefficient of (2) above is defined as follows.

That is, it is known that -m has a high proximity correlation between sample values (χ) of audio waveforms.

Therefore, assume that the following linear prediction is possible.

Linear predicted value χt=-Σα1χt-1 ...(1
)Linear prediction error ε(=χ ahead−χ(...(2) Here, χ is the sample value of the audio waveform at time t, (αm)
(1=1...., p): (9th order) linear prediction coefficient Now, in the implementation of the present invention, the linear prediction error ε, 2
The linear prediction coefficient (a,) is determined so that the root mean value is minimized.

Specifically, find (ε)2 and calculate the time average as (ε,
)2, θ(εt)”/aa (=O, i=
], 2. ..., p, from the following equation (
al) is required.

ΣQ 1vll−Jl = L j= 1 +
2 + . . . p .

That is, [k,] (n=1..., p) is (9th order) P
When AR (:OR coefficient (partial autocorrelation coefficient)), PAR
COR coefficient k. , l is the forward residual εt due to linear prediction
l) Backward residual εL-+n+11 (defined as the normalized correlation coefficient between b' by the following equation).

ε%f1. ε, −(..,, (bl ... (4) Here, 8t(f)=χe−′t αlχt−1(α,
): forward prediction coefficient, ε1−(river, ll=χ(−(river edge j・χt−J・(β
C): Backward prediction coefficient Also, the pitch frequency of the voice mentioned in (2) above is the reciprocal of the repetition period (pitch period) of the vocal cord wave.

Furthermore, since the pitch frequency, which is a basic parameter of the vocal cords that differs among individuals, was added as an input to the neural network, it is possible to improve the recognition rate of speakers, especially between adults/dwarfs and male/female. .

Furthermore, the above-mentioned high frequency enhancement (2) is to compensate for the average slope of the spectrum of the audio waveform to prevent concentration of energy in the low frequency range. However, the average slope of the spectrum of the speech waveform is common to all speakers and is unrelated to the speaker's recognition.

However, when inputting an audio waveform that has not been compensated for the average slope of the spectrum to a neural network as is, the neural network will extract the feature of the average slope of the spectrum during learning, and the speech will be distorted. It takes time to extract the peaks and valleys of the spectrum necessary for human recognition. On the other hand, when high-frequency emphasis is applied to the input to a neural network, it is possible to compensate for the average slope of the spectrum that is common to all speakers and is unrelated to recognition, but that affects learning, which speeds up the learning process. becomes faster.

[Effects of the Invention] As described above, according to the present invention, it is possible to obtain a word recognition system in which the deterioration of the recognition rate over time is extremely small and which can easily perform real-time processing.

Further, according to the present invention, when additionally learning registered words related to unrecognizability to prevent the recurrence of unrecognizability, it is possible to easily implement it without requiring an increase in memory capacity or an increase in recognition processing time. A word recognition system capable of time processing can be obtained.

[Brief explanation of the drawing]

FIG. 1 is a schematic diagram showing an example of a word recognition system to which the present invention is applied, FIG. 2 is a schematic diagram showing an example of a speech processing unit and a neural network, FIG. 3 is a schematic diagram showing input speech, and FIG. The figure is a schematic diagram showing the output of a bandpass filter.
Figure 5 is a schematic diagram showing a neural network, Figure 6 is a schematic diagram showing a hierarchical neural network, and Figure 7 is a schematic diagram showing a hierarchical neural network.
The figure is a schematic diagram showing the structure of the unit. DESCRIPTION OF SYMBOLS 10...Speaker recognition system, 11...Speech input unit, 12...Speech processing unit, 13...Neural network, 14...Determination unit, 15...Memory unit, 16... Network control unit, 17... Equipment control unit, 18... Display control unit, 19... Display unit, 2o... Teaching unit.

Claims (5)

[Claims] (1) A word recognition system using a neural network, in which a word recognition threshold and an additional learning threshold are set for the output value of an output unit corresponding to a registered word, and the output value is The current input voice word is determined to be a registered word on the condition that it is greater than the word recognition threshold,
A word recognition system that performs additional learning of a neural network using current input voice data on the condition that the output value is greater than a word recognition threshold and smaller than an additional learning threshold. (2) A word recognition system using a neural network, in which a word recognition threshold is set for the output value of an output unit corresponding to a registered word, and the output value is greater than the word recognition threshold. On the condition that , a word recognition system that performs additional learning of a neural network using the current input word data taught by the current speaker and the current input voice data for this display. (3) A word recognition system using a neural network, in which a word recognition threshold and an additional learning threshold are set for the output value of the output unit corresponding to the registered word, and the output value is The current input voice word is determined to be a registered word on the condition that it is greater than the word recognition threshold,
(A) On the condition that the above output value is smaller than the threshold for word recognition, it is displayed that the current input speech word could not be recognized, and in response to this display, the current input voice word taught by the current speaker is displayed. Additional learning of the neural network is performed using the input word data of
) A word recognition system that performs additional learning of a neural network using the current input voice data, provided that the output value is greater than a word recognition threshold and smaller than an additional learning threshold. (4) As inputs to the neural network, [1] Temporal changes in the frequency characteristics of speech, [2] Average linear prediction coefficients of speech, [3] Average PARCOR coefficients of speech, [4] speech using one or more of the following: average frequency characteristics and pitch frequency; [5] average frequency characteristics of high-frequency emphasized audio waveform; and [6] average frequency characteristics of audio. The word recognition system according to any one of claims 1 to 3. (5) The word recognition system according to any one of claims 1 to 4, wherein the neural network is a hierarchical neural network.