JPH08123779A - Method for learning pattern - Google Patents

Abstract

PURPOSE: To speed up learning by calculating a difference between a neural network output value in a category and a teacher signal and changing load correction efficiency while estimating learning skillness from a code change of a difference from the square error sum of a succeeding learning cycle. CONSTITUTION: Grating dividing data (f) is inputted to a neural network recognizing part 7, teacher data yj is set up, a neuron output value oj is calculated, and the square error sum of all pattern inputs is calculated from Ep(K)=p ΣjΣ|yj (K)-O<-> [M]j(K)|<2> /2<ε1 . Then following calculation is executed based upon a code change in the variable ΔEp =Ep (K+I)-Ep (K) of the square error sum (provided that I=1, 2), namely if ΔEp >0, then n=η×ϕ and α=α, if ΔEp <0, then n=n×β and a=0. Provided that β<1, ϕ>1, nmin <n<nmax , ϕ and β are learning efficiency coefficients, η is a load correction coefficient, and α is an inertia coefficient. Consequently the batch correction of loads can be executed based upon these learning rules.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pattern learning method, and more particularly to a pattern learning method having a neural network recognition means.

[0002]

2. Description of the Related Art FIG. 7 is a block diagram showing the structure of a conventional pattern learning method. Referring to FIG. 7, in the conventional pattern learning method, the scanner 1 optically scans a recognition target and outputs a video signal a. The A / D converter 2 inputs the video signal a, digitizes an analog image, and outputs a digital image b. The image memory 3 stores the digital image b and outputs the image data c. The binarization processing unit 4 inputs the image data c, sets the pattern portion to “1”,
The background part is assigned to "0" and the binary image d is output. The pattern area detecting unit 5 inputs the binary image d, projects the binary image d, cuts out a recognition pattern, and creates pattern data e. The pattern data e is divided by the grid division unit 6 into division data f.
Is converted to. The neural network recognition unit 70 interprets the grid division data f as one input pattern data, performs learning, and outputs a calculation output signal g. The recognition determination unit 8 searches for the maximum value of the calculation output signal g and sets the category classification at the maximum value as the recognition result. The learning determination unit 9 confirms that the recognition result is equal to or less than the standard value, and finishes all learning.

FIG. E. FIG. RUMELHART etc. are PARAELL DISTRIBUTED PROC
ESSING, vol1, MIT Press, 198
6 is a flowchart showing the operation of the neural network recognition unit 70 shown in pages 6, 318 to 330. With reference to FIG. 5, the neural network recognition unit 70 of the conventional pattern learning method performs the weight and bias value correction calculation at the time of learning by using only the error back propagation method. Therefore, learning may stop at the convergence stage of learning. Further, the load value generated at that time is used as it is for the arbitrary pattern data for recognition.

[0004]

Since the conventional pattern learning method described above uses the simple error back-propagation method,
As shown in FIG. 6, learning is likely to stop, and learning efficiency decreases. In addition, since the generated weight value is not appropriate, the pattern recognition rate cannot be expected to be improved, so that the learning must be redone.

An object of the present invention is to provide a high-speed learning method by changing the weight correction coefficient according to a rule from the sign change of the sum of squared errors in the learning cycle when the weight is corrected, suppressing the stop of learning. is there.

[0006]

A pattern learning method of the present invention is a pattern learning method for reading and recognizing pattern information, wherein a scanner for optically operating a non-recognition target to pick up an image, and a non-scanner obtained from the scanner. Analog-to-digital conversion means for converting the image pickup signal of the recognition object into a digital signal, image data storage means for storing the digital signal and outputting image data, and binarization for binarizing the image data and outputting a binary image. Means, area detection means for detecting the area to be recognized from the binary image, and grid division means for creating grid division data to be a neural network recognition input pattern from the detected area data to be recognized. , A neural network recognition hand for recognizing the recognition target by calculating the grid division data using neural network learning And a recognition determination means that detects the maximum value of the neural network recognition means and uses the category at that time as a recognition result. The neural network recognition means includes the j-th neuron in the Mth layer in the learning process calculation. Δω ^(M) is the weight correction value of the k-th learning cycle of
_If the learning rule when _ij (k) is O ^(M-1) _j is the j-th neuron output value of the M-th layer, Δω ^(M) _ij (k + 1) = η · δ ^(M) _j · O ^(M-1) _j + α
Δω ^(M) _ij (k) The kth bias correction amount Δθ ^(M) _ij (k) of the jth neuron in the Mth layer is Δθ ^(M) _ij (k + 1) = η · δ ^(M) _j + α ・ Δθ ^(M) _ij
(K) where δ ^(M) _j = f ′ (j) · (1−f (j)) f (x) = 1 / (1 + exp (−X)) η is a load correction coefficient, and α is an elastic coefficient. Error learning function for one cycle of learning for all patterns of the neural network recognition input data when the learning data is y (j)

[0007]

In the process of setting and setting the learning value to ε ₁ or less, the neural network recognition input data p
Change of the sum of squared errors ΔE _p = E _p (k + 1) −E _p for each category pattern after inputting all patterns
Based on the sign change of (k), if ΔE _p > 0 THEN η = η × φ α = α if ΔE _p <0 THEN η = η × β α = 0 where β <1, φ> 1, η _min <Η <η _max φ, β is a learning efficiency improvement coefficient, η is a weight correction coefficient, and α is an inertia coefficient.

[0009]

Next, the present invention will be described with reference to the drawings. Referring to FIG. 1, which shows a block diagram of an embodiment of the present invention, in the pattern learning method of this embodiment, a scanner 1 optically scans an object to be recognized and outputs a video signal a. The A / D converter 2 inputs the video signal a, digitizes an analog image, and outputs a digital image b. The image memory 3 stores the digital image b and outputs the image data c. The binarization processing unit 4 inputs the image data c, assigns the pattern portion to “1” and the background portion to “0”, and outputs the binary image d. The pattern area detection unit 5 inputs the binary image d, projects it to cut out a recognition pattern, and creates pattern data e. The pattern data e is the grid division unit 6
Is converted into the divided data f. The neural network recognition unit 7 interprets the grid division data f as one input pattern data, performs learning, and outputs a calculation output signal g. The recognition determination unit 8 searches for the maximum value of the calculation output signal g and sets the category classification at the maximum value as the recognition result. The learning determination unit 9 confirms that the recognition result is below the standard value,
Finish all learning.

The difference from the conventional pattern learning method lies in the neural network recognition section 7. 2 is a flow chart showing the learning process of the neural network recognition unit 7 of this embodiment, FIG. 3 is a schematic diagram of the processing of the neural network recognition unit 7 shown in FIG. 2, and FIG. 4 is the learning correction method of this embodiment. FIG. 10 is a relationship diagram between the sum of squared errors and the progress of the weighted space learning in the case of occurrence. Referring to FIGS. 2, 3 and 4 together with FIG. 1, first,
The grid division data f is input to the neural network recognition unit 7 (procedure 10) and the teacher data y _j is set (procedure 1).
1). Next, the neuron output value O _j on the neural network output layer unit is calculated (procedure 12), and the sum of squared errors of all pattern inputs is calculated by the following equation (procedure 13).

[0011]

As shown in FIG. 3, since the neural network of the neural network recognition unit 7 is of a hierarchical type, the standard value ε _l
Until each condition is satisfied Δω
^(M) _ij (k) and bias value correction amount Δθ ^(M) _ij (k)
Is given by the following load correction equation (procedure 19).

Δω ^(M) _ij (k + 1) = η · δ ^(M) _j · O
^(M-1) _j + α ・ Δω ^(M) _ij (k) Δθ ^(M) _ij (k + 1) = η ・ δ ^(M) _j + α ・ Δθ ^(M) _ij
(K) where η is a weight correction coefficient, δ ^(M) _j is a correction value of the j-th neuron in the M-th layer, α is an inertia coefficient, and Δ
θ ^(M) _j (k) represents the bias value of the j-th neuron in the k-th learning cycle of the M-th layer, and Δω _ij (k) represents the weight correction amount. Furthermore, the inter-tier load Δω obtained here
^(M) _ij (k) and bias value correction amount Δθ ^(M) _ij (k)
Is stored in the memory (procedure 20). As is clear from the load correction formula and FIG. 4, the sign change of ΔEp corresponds to the learning progress and determines the convergence direction in the load space, and for the same neural network input pattern, ‖Δω ₁ ‖> ‖Δω ₂ ‖
Therefore, the minimum point (learning convergence position) is reached, that is, the learning speed is increased and the search efficiency is improved at the same time. The end of the load correction calculation value described above is confirmed (procedure 21), and if YES, it is considered that the load correction of one learning cycle is completed, and the batch correction of the load is executed (procedure 22). When this falls below the standard value ε ₁ (procedure 15), the neural network output calculation result is recognized by the recognition determination unit 8
To the recognition result category.
Further, the learning end judgment unit 9 confirms the end or continuation of the learning and finishes the learning.

[0014]

As described above, according to the present invention,
After inputting all input patterns of the hierarchical neural network, calculate the sum of squared errors, which is the difference between the output value of each category and the teacher signal, and learn from the sign change of the difference with the sum of squared errors in the next learning cycle. Since the load correction efficiency can be changed while estimating the proficiency level,
It is possible to speed up learning by suppressing the stop of learning that tends to occur when learning is performed only by the error back propagation method.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention.

FIG. 2 is a flow chart showing an operation of a neural network recognition unit of this embodiment.

FIG. 3 is a schematic diagram of a processing process of a neural network recognition unit.

FIG. 4 is a relationship diagram between the sum of squared errors and the number of times of learning when the learning correction method of this embodiment is used.

FIG. 5 is a flow chart showing an operation of a neural network recognition unit of a conventional pattern learning method.

FIG. 6 is a relationship diagram between the sum of squared errors and the number of times of learning when the learning that is likely to occur when the conventional load correction formula is used is stopped.

FIG. 7 is a block diagram showing a configuration of a conventional example.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Scanner 2 A / D conversion unit 3 Image memory 4 Binarization processing unit 5 Pattern area detection unit 6 Lattice division unit 7 Neural network recognition unit 8 Recognition determination unit 9 Learning end determination unit 30 Image processing unit

Claims (1)

[Claims] 1. A pattern learning method for reading and recognizing pattern information, wherein a scanner for optically operating a non-recognition target to capture an image, and an analog for converting an imaging signal of the non-recognition target obtained from the scanner into a digital signal. Digital conversion means; image data storage means for storing the digital signal and outputting image data; binarization means for binarizing the image data and outputting a binary image;
Area detection means for detecting the area to be recognized from the value image, grid division means for creating grid division data to be a neural net recognition input pattern from the detected area data to be recognized, and the grid division data A neural network recognition means for recognizing the object to be recognized by calculating using a neural network learning, and a recognition determination means for detecting the maximum value of the neural network recognition means and using the category at that time as a recognition result. In the neural network recognition means, the learning rule when the weight correction value of the j-th neuron in the M-th layer in the k-th learning cycle in the learning process calculation is Δω ^(M) _ij (k) is O ^{( If M-1)} _j is the j-th neuron output value of the M-th layer, Δω ^(M) _ij (k + 1) = η · δ ^(M) _j · O ^(M-1) _j + α
Δω ^(M) _ij (k) The kth bias correction amount Δθ ^(M) _ij (k) of the jth neuron in the Mth layer is Δθ ^(M) _ij (k + 1) = η · δ ^(M) _j + α ・ Δθ ^(M) _ij
(K) where δ ^(M) _j = f ′ (j) · (1−f (j)) f (x) = 1 / (1 + exp (−X)) η is a load correction coefficient, and α is an elastic coefficient. Error learning function for one cycle of learning for all patterns of the neural network recognition input data when the learning data is y (j) In the process of performing learning until the standard value ε ₁ or less is reached, the change amount ΔE _p = E _{p of the} sum of squared errors for each category pattern after all patterns of the neural network recognition input data p are input. Based on the sign change of (k + 1) −E _p (k), if ΔE _p > 0 THEN η = η × φ α = α if ΔE _p <0 THEN η = η × β α = 0 where β <1, φ> 1, η _min <η <η _max φ, β is a learning efficiency improvement coefficient, η is a weight correction coefficient, and α is an inertia coefficient. Pattern learning characterized by executing a batch correction of loads using a learning rule. Method.