JPH01204172A - Dynamic neural network having learning mechanism - Google Patents

Abstract

PURPOSE:To realize a learning method which performs recognizing actions with use of voice patterns different in length by using a learning mechanism with teachers which displays a pattern to be shown on a network after normalizing said pattern into the fixed duration length and minimizes the error of an output layer. CONSTITUTION:A time base matching part 30 decides the key pattern of each category in a group of learning data and secures the matching between the time base of another learning data and that of the key pattern to normalize the time length of all learning data. A correction value calculating part 30 calculates the correction value of a coupling coefficient by means of the learning data received from the part 30 and the coupling coefficient stored in an inter- unit coupling coefficient memory part 50 and displays a teacher signal to be sent to an output layer in correspondence. Then the learning with teachers is carried out to decide a coupling coefficient so that the difference is reduced between the teacher signal of the output layer and the actual output value. Thus it is possible to learn the variance of a feature parameter for each utterance of the voice data with no diversity caused by the variance of the uttering time length.

Description

DETAILED DESCRIPTION OF THE INVENTION (Field of Industrial Application) The present invention relates to a dynamic neural network having a pattern learning mechanism used for recognizing time-series patterns such as speech.

(Prior Art) A neural network is an information processing system devised based on the fact that the biological nervous system is an information processing system composed of neurons with relatively simple operating characteristics and numerous connections between them. The model has processing units t7 (hereinafter abbreviated as units) corresponding to neurons and inter-unit connections connecting them. By changing the coefficients of this inter-unit coupling, the system performs various information processing operations.

This neural network model is expected to be effective as an information processing system, especially for pattern recognition processing of images and sounds, etc. For details, please refer to "Nikkei Electronics" magazine, No. 427, page 115. (Published August 10, 2016) ``Using Neural Nets for Pattern Recognition, Signal Processing, and Knowledge Processing.'' (Hereinafter referred to as Document 1.) According to Document 1, a neural network has a hierarchical structure called an input layer, a middle layer, and an output layer, as shown in Figure 2, and each layer consists of multiple units. It is configured. Further, inter-unit coupling is allowed only between adjacent layers, and inter-unit coupling within a layer is prohibited. During recognition, the network is given input data as the activation level of each unit in the input layer, transmits information sequentially to adjacent intermediate layers through inter-unit connections, and finally reaches the output layer. In this way, the response result of the network to the input data is obtained as a pattern of the activity levels of the units in the output layer.

A method called supervised learning is used to determine connections between units so that the network performs specified operations. In other words, a pattern to be learned is presented to the input layer, a corresponding teacher signal to be output is presented to the output layer, and the combination is made to reduce the difference between the teacher signal and the actual output value in the output layer. Determine the coefficients. In the case of a neural network configured as described above, this output error minimization learning is called pack propagation learning, and its detailed algorithm can be found in Reference 1.

(Problem to be solved by the invention) If such a neural network can be used for speech recognition, it may be possible to absorb the diversity of speech patterns through learning and achieve good recognition performance. In order to actually use the above neural network for speech recognition, there are several problems that must be solved.

First, speech patterns of the same category (for example, words) have different durations each time they are uttered or for each speaker, so speech patterns of different lengths are presented to the input layer of the same neural network. It is necessary to devise ways to do so.

Second, when speech patterns of different lengths can be presented as input to a neural network, a learning method must be established to determine connections between units so that the network performs the expected recognition operations.

In order to present speech patterns of different lengths to a neural network having an input layer that can input feature parameter time series with a fixed time length, the time of the input layer is adjusted so that the output of the output layer is maximized during recognition. A supervised system that associates the axes with the input audio time series and determines the inter-unit coupling coefficient, normalizes the presented pattern to a fixed duration length and presents it to the network to minimize the error in the output layer. This paper attempts to provide a dynamic neural network with a learning mechanism.

(Means for Solving the Problems) The present invention is a neural network that recognizes time-series patterns such as speech, and has a hierarchical structure consisting of an input/output layer and a plurality of intermediate layers. The middle layer has a time-series structure corresponding to the time axis, and during recognition, dynamic programming is used to match the time axis of the input time-series pattern with the time axis of the input layer so that the output of the neural network is maximized. In a dynamic neural network where the recognition result is the output of the output layer at that time, when learning the coupling coefficient between units between each layer, a constant time length that is the same as the time length of the input layer is used. is normalized by linear expansion/contraction and presented to the input layer, and the corresponding teaching signal to be output is presented to the output layer, and the coupling coefficient is set so as to reduce the difference between the teaching signal and the actual output value in the output layer. It is characterized by having a mechanism that performs supervised learning to determine.

(Function) For the sake of simplicity, the present invention will be described in detail using a three-layer structure model with one intermediate layer. Needless to say, the present invention can be similarly applied to cases where there are two or more intermediate layers.

The input layer of the model is composed of J×P units so that it can receive a time series (length J) of P-dimensional feature vectors. The output value of this input crab unit is y”
'J(P) (j=1~J, p・1~P). In general, the length J of the time axis of the input layer and the input time series pattern at(p>(i・1~1.p・1~P
) are different, so the time axis of the input time series must be subjected to some expansion/contraction transformation to make it equal to the length J.

The correspondence relationship on the plane (i, j) formed by the time axis j of the input layer and the time axis i of the input time series pattern is expressed by the following equation.

c(k) ・(i(kn, j(k)),
(k4~K) = 111 However, c(1) ・ (1, 1), c(k) = <
1. J) i(k)≧1(k-1>, j(k)≧j(k
-1) ...+21Using this relationship, the output value y(1) of the input crab unit. +(p) (J・1~J
, p=l~P> is y(1ゝjt k+(p)
” az k+(+)),
=i3). That is, the input unit aligns the time axis and transmits the input data as is to the next layer.

The middle layer is composed of JXM units (called hidden units), and the input value to each unit is
(j・1~J, m・1~M) is the output value y + 1) j(p) of the input crab unit and the coupling coefficient between the input crab unit and the hidden unit βOt(m, p), β'J(I
IP) is given as follows.

X J (Peng): Σ ku β j ku Lp) y
j(k)(p)+β jD+1 <jp)3""j+to -t+<p) azb-t+(P)) ・
...4) In this way, the j(k)th hidden unit is a neural network that restricts the connections between units so that it only receives information from the 1(k)th and 1(k-1)th units in the input layer.・The structure of the network will be called the time-series structure. When the data itself has a time-series structure, such as a speech pattern, the structure of such a network requires fewer connections between units than a completely connected network (connecting all incoming units and all hidden units). Since a model can be constructed by combining, the amount of calculation during recognition and learning can be significantly reduced. 4) The response of the hidden unit to the input given by equation is as follows.

y (2'>(m)・f < x (2't(
m)-θ (2),ku■)) ...(9
f (x>=1z' (1+e-X)
= 161 where θ(21j(1) is the hidden unit (
j, m) has a threshold value. As is clear from Equation 6, the hidden unit functions as a kind of threshold logic.

The output layer is composed of N units corresponding to N categories to be recognized. The input value x ''(n>(n=l~N) to the n-th output unit is the output value y''J(1) of the hidden unit and the coupling coefficient α''(j , i+) is given as follows.

The response relationship between the input and output of the output unit is the same as Equation 2.

y(3'(n)" f (x (3)(n)-θ(3J
n)) ,, -U where θ(3'(n) is the equivalent value of output unit n.

The output value of the network obtained in this way y(31(n)
depends on the correspondence relationship (C(k)) between the time axis of the input time series and the time axis of the input crabnit layer given by Equation 1. The final recognition result by the network for category n is obtained as an optimized (maximized) output value On with respect to (c (k>)).

Here, since equation 8 is a monotone function, equation (9) is +β1. (Cao)
It is the same even if it is replaced with al, -口))1 .--(1o). Here, the sum of the component p of the feature vector in fN is omitted.

(10) Expression in parentheses is 7 (c(k), c(k-1))= (-)
=・kull), then equation (10) becomes o, =max[Σγ(c(k), c(k-1>)]
-(12) lcfkll ktl , and it can be seen that this optimization can be solved using the well-known dynamic programming method. That is, γ(c(k),
The cumulative sum of c(k-1)) is set as g(k), and the following recurrence formula is calculated to obtain o,=g(k).

g<k)=max[7(c(k>,c(k-1))+
g(k-1)] =-(13)c(k-11) Next, the inter-unit coupling coefficient (β0.(■, p), which is a parameter of the neural network model.

β1. (--p), α″(j, m) and threshold value (θ(21,
<II) A learning method for determining lθ(3)<n)) will be explained.

Let the set of time series of feature vectors used for learning category n be A"'q (a 'q, 1(p)). Here, q is a subscript that distinguishes multiple time series patterns within the same category. i is a subscript that represents the time axis of the time series, and p is a subscript that represents the component of the feature vector at each time.The range of each subscript is n = 1 to N, q -1 to Q'', i = 1 to I
', p = 1 ~ P ・= < 14) In order to present this data A (n + 9) to the network, the time series length I
Q must be normalized to the length J of the time axis of the input layer of the network. Since the model parameters are not optimized during learning, it is difficult to use dynamic programming as during recognition.

Therefore, for learning, we need a set of data of category n A3''''
, (q・1~Q″), the representative time series pattern ALfilqo is selected, and the other data A(II
) q (q≠qo) is associated with the time axis of the representative pattern by DP matching. The method is shown below. Representative pattern A+I11. Let the time of o be j(j・1~
J) Let i (i・1~■) be the time axis of data A"' (q≠qo) for which you want to perform time axis matching (normalization). At this time, perform DP matching of the two patterns. A correspondence relationship (distortion relationship) i=i r j) between the time axes of the two patterns can be obtained.

DP matching and distortion functions are explained in detail in "Nikkei Electronics" magazine, No. 329, page 171 (published on November 7, 1982) (hereinafter referred to as Document 2). Due to this distortion function i<j), the time axis j of the representative pattern has a frame with the time axis 1=i(j) of the learning data.
It can be seen that it is sufficient to associate the vectors a'Q, li). This distortion function takes the form 1=j(i) depending on how the local path used for DP matching is restricted, and there may be multiple frame vectors corresponding to a certain j. Even in such cases, similar time axis correspondence can be achieved by averaging the corresponding frame vectors.

As a result, the time length IQ, which varies from data to data, is normalized to a constant length I''. The time length J of the input layer of the network is set equal to this IQO.

Here, various methods can be considered to select the representative pattern of category n, but for example, the cumulative distance d (A, 0. Aq> is the distance between patterns) by DP matching between patterns in the set of patterns of category n. Let Qo be such that the quantity Δ given by the following equation is minimized. This q can be easily determined by the brute force method that calculates Δ assuming all 9.1 to Q″ as Qo. .

It is also possible to make any one pattern representative of this pond.

In this way, the input learning data with the time axis normalized to length J is A(fi' q= (α″q, +(p) ) (i・
1 to J). In addition, the learning data of other categories normalized to the same length J are B(″'−・(b −r, +
(p) ) (r・1~R) (hereinafter, this B will be referred to as anti-learning data).

At this time, the output value of the network for the qth learning data is y+s+,,(n), and the desired output value is ZQ(n
)(・1.0), the output value of the nth unit for the rth unlearning data is y(3'r(n), and the desired output value is z, (n)(・0.0), then the output is The error E of the output value in the crab knit layer is E=1/2Σ[3',<n)-z q(n)
]q=1 +1/2Σ [3', (n)-z r(n
)] −・−(16) r=1. This error amount E is determined by the inter-unit coupling coefficient (β0'(m, p)), which must be determined by learning.
β′>(m, p), α”(j, m)) and threshold value (θ(21
, (■).

θ”(n)), so these parameters can be determined to minimize E as the evaluation function.Also, the threshold value of the unit is determined by hypothetically considering a unit that always outputs 1. If you think of it as a coupling coefficient with a unit, you can learn it in the same way as inter-unit coupling.Then, the inter-unit coupling coefficient that connects the unit i of the adjacent 2nd layer, the nth layer, and the unit j of the n+1th layer is ω''' 1, and if we use the differential coefficient of E with respect to ωn and J, we will definitely have E (t+1)≦E(t) ・
...(18). Here, t is an integer value representing the step of iterative learning, and ε is a constant that determines the degree of correction. In the end, ωfl is used to reduce E. Parameter learning is achieved by repeatedly modifying the parameters. Here ω
Unit coupling coefficient (β0J(m,
P) β'J(1, P), α''(j, l), θ(21j(
1), θ” (nN may be associated with each other as follows, for example.

As a result of analytical calculation, the differential coefficient of E is found to be as shown in the following equation.

Here, δ″+1ゝ1.Q is the error converted to the input value of unit i in the n+1th layer when the qth learning (or anti-learning) data is presented to the input layer, and y(n)I, 9 is the output value of unit 1-j of the n-th layer for the q-th learning data. δ(n l , , q can be calculated using the following recurrence formula.

δ 3”ゝ31.. ・Σ δ (o++1.,
, Qω 'ml (df/dx), = x'''l where f (x) is the input/output response function of the unit given by Equation 6, x311 is the human input value to unit i in the nth layer, and Zl is the input/output response function of the unit given by Equation 6. The value that unit i of the N layer (output layer) should take is 1.0 during learning and 0.0 during anti-learning.Equation 21
This learning method is called a back-propagation learning method because the calculation to find the converted error amount δ for each unit proceeds from the output layer to the input layer based on For details, see Reference 1).

After all, if we start from a model in which arbitrary initial values are given to the inter-unit coupling coefficients, present multiple learning/unlearning data, and perform the above-mentioned iterative correction learning for each uni/sotho coupling, the output layer We can obtain a set of uni-sotho connections that minimize the error in .

(Example) The time axis correspondence on the (i, j) plane for calculating the recurrence formula of Equation 13 is as follows as a rule (relative positional relationship between c(k) and c(k-1)). The present invention will be described in detail when the rules as shown in FIG. 3 are used. In the case of Fig. 3, c(k)=(i,
j), then c(k-1) is (i-1, j),
Only three points (i-1, j-1) and (i-1, j-2> are possible. In the case of rules, such a correspondence determines the output of the neural neural network (12 ), (1
3) The formula can be written as follows.

γ” (i, j) ・Σ α (j, Cao) f
(β i < m)at+β.

+! 1 for l (m) at-1)
−...(23) g ” <i, j> = 7
”(i,j)+max[g'(i-1,j).

g' (i-1, j-1), g fl (i-1, j
-2)] ...(24) Figure 1 is (22)
FIG. 24 is a block diagram showing an embodiment of the present invention based on formulas 24) to 24). The analysis unit 10 analyzes the input audio waveform data and converts it into a time series of feature vectors.
The data is stored in the pattern buffer section 20. The pattern bass section 20 stores learning time series data during a learning operation, and stores analysis data of unknown utterances during a recognition operation. The following changeover switch switches between learning operation and recognition operation.

The time axis matching unit 30 determines a representative pattern for each category in the learning data group based on Equation 15, performs DP matching on the time axes of other learning data to the representative pattern, and matches all the learning data. Standardize the time length to length J. The correction amount calculation unit 40 uses the learning data sent from the time axis alignment unit 30 and the coupling coefficients stored in the inter-unit coupling coefficient storage unit 50 to calculate the coupling coefficient ω″1. A correction amount Δω″'IJ is calculated and sent to the coupling coefficient correction unit 60. The coupling coefficient correction unit 60 adds the correction amount Δω″ to the coupling coefficient stored in the inter-unit coupling coefficient storage unit 50 and writes it back. Correction amount calculation unit 4
0 repeats this correction operation until the correction amount Δω''IJ for all coupling coefficients becomes smaller than a predetermined threshold, or until the number of correct ICs exceeds a predetermined number.

The lattice point calculation section 70 uses the unknown utterance data sent from the pattern buffer section 20 and the coupling coefficient stored in the inter-unit coupling coefficient storage section 50 to calculate the lattice point data γ'(i, j) based on Equation 23. (i・1~I, j・1~J, n・
1 to N). The calculated grid point data is stored in the grid point storage section 80. The recurrence formula calculation section 90 uses the grid point data stored in the grid point storage section 80 to perform recurrence formula calculation based on equation 24, and stores the cumulative value g''(1, J) in the working storage section 100. The working storage unit 100 is used to store g'(i, j> even during recurrence formula calculation. The recognition determining unit 110 stores the cumulative value g'<1° stored in the working storage unit 100. J), the value of n that gives the maximum cumulative value is output as the recognition result.

(Effects of the Invention) As described above, according to the present invention, the time axis that allows the fluctuation of the utterance time length of unknown speech data to be normalized by dynamic programming and input into the neural network during the recognition operation. It is possible to provide a neural network with a normalization ability of

In this way, the neural network of the present invention has the ability to normalize the time axis during the recognition operation, so during the learning operation, it is possible to reduce the variation in the feature parameters for each utterance of the audio data by using a small number of learning data (the diversity due to the variation in the utterance duration). It is possible to provide a good recognition device by learning using the following information.

[Brief explanation of the drawing]

Figure 1 is a block diagram showing an embodiment of the present invention, Figure 2 is a diagram showing the hierarchical structure of a neural network, and Figure 3 is a diagram showing the time axis on the (i, n plane) for calculating recurrence formulas. The matching section is a diagram showing an example of rules. In the figure, 10 is an analysis section, 20 is a pattern buffer section, 30 is a time axis matching section, 40 is a modification amount calculation section, and 50 is a coupling coefficient between units 1 to 1. storage unit, 60 is a coupling coefficient correction unit, 7
0 is a lattice point calculation section, 80 is a lattice point storage section, 90 is a recurrence formula calculation section, 100 is a working storage section, and 110 is a recognition determination section.

Claims (3)

[Claims] (1) Neural technology that recognizes time-series patterns such as voices, etc.
The network has a hierarchical structure consisting of input/output layers and multiple intermediate layers, and the input layer and intermediate layers have a time-series structure corresponding to the time axis. During recognition, input is input using dynamic programming. In a dynamic neural network, the time axis of the time series pattern is associated with the time axis of the input layer so that the output of the neural network is maximized, and the output of the output layer at that time is the recognition result.
When learning the inter-unit coupling coefficient between each layer, the training time series pattern is normalized to a constant duration length that is the same as the time length of the input layer. The trained time series pattern is presented to the input layer, and the corresponding teacher signal to be output is presented to the output layer to reduce the difference between the teacher signal and the actual output value in the output layer. A dynamic neural network that has a mechanism for supervised learning to determine coupling coefficients. (2) The dynamic neural network according to claim (1), wherein the variation in the duration of the learning time-series pattern is normalized by DP matching to a representative pattern. (3) The dynamic neural network according to claim (1), wherein the supervised learning of the inter-unit coupling coefficients is realized by a backpropagation learning method.