JPH0363775A - Neural network system - Google Patents

Abstract

PURPOSE:To improve the recognition rate by recognizing only a pattern which is difficult for a first neural network, which outputs recognition results, to recognize out of all inputted patterns by another neural network. CONSTITUTION:If recognition results of a neural network NNA which outputs recognition results of all input patterns are not accurate for all input patterns, only input patterns whose recognition results are inaccurate are recognized by the other neural networks NNB and NNC. Since these neural networks NNB and NNC learn only input patterns, whose recognition is generally difficult, and recognize them, they are so organized that they remarkably react on the subtle difference hardly recognized by the general neural network, and they surely recognize input patterns. Thus, a neural network system of very high recognition precision is easily realized.

Description

DETAILED DESCRIPTION OF THE INVENTION Field of the Invention This invention relates to neural networks that recognize input patterns.

(b) Conventional Technology A neural network has been proposed as a method for efficiently recognizing characters, voices, etc. using a simple algorithm. The neural network associates the input cover turn with the expected output, which is the correct recognition result of this person's cover turn, and feeds it to the neural network, and adjusts each neuron so that the difference between the actual output obtained from the input cover turn and the expected output is small. A high recognition rate can be obtained with a simple configuration using learning algorithms such as backpropagation that modify connections.

fc) Problems to be Solved by the Invention However, even if such a learning algorithm is used, it is difficult to identify patterns with poor separation, such as multiple pattern groups that cannot be linearly separable in boat fishing. However, there are also patterns that are impossible to recognize. If such patterns were to be recognized, the number of learning cycles would have to be increased, which would require a long time before the neural network could operate. In addition, there are some patterns for which a high recognition rate cannot be obtained no matter how much learning is repeated, and if you try to recognize these patterns with a single neural network, you will have to complicate the structure of the network, which will take a long time to calculate. Moreover, since the recognition involves too much detail, a large amount of noise is mixed in, which has the disadvantage of actually lowering the overall recognition rate.

In view of the current situation, the present invention aims to provide a neural network system that increases the recognition rate by recognizing only difficult-to-recognize patterns using a separate neural network. (d) To solve the problem Means (1) The first invention of this application includes a first neural network organized to output recognition results for all input patterns, and a pattern that the first neural network recognizes as an easy to recognize pattern. The 20th neural network and the 10th neural network, which are organized to identify and output difficult patterns, have learned only the patterns that are difficult to recognize, and the first neural network has learned the recognition results for the difficult patterns. and a third neural network organized to output.

(2) The second invention of this application is a neural network system that classifies input patterns into a plurality of categories, the neural network system being organized to recognize and output the category to which all input patterns belong. and a second neural network organized to recognize and output patterns that are difficult for the 10th neural network to recognize, and a plurality of said second neural networks are provided for each category. Features.

(e1 Function of the invention The neural network system according to the first invention of this application has first to third neural networks.The first neural network outputs recognition results for all input patterns. .For this reason, all patterns that can be input are trained, and there is a possibility of confusion regarding minute differences between similar patterns.For this reason, recognition is performed by the first neural network based on the judgment of recognition ability through learning. The second neural network determines whether or not the pattern is difficult, and only this pattern is recognized by the No. 30 neural network.The first and second neural networks both recognize all input patterns. Although the recognition target is
Since the expected outputs are different from each other, the organization of the networks is also different, and the second neural network, which makes simpler judgments, has a higher judgment rate. In addition, the third neural network learns only patterns that were difficult for the first neural network to recognize and recognizes only these patterns, so it only needs to recognize so-called minute differences, achieving a high recognition rate. I can do it.

By combining a plurality of simple neural networks in this way, it is possible to improve the recognition rate, and it is also possible to prevent the complexity of each neural network and the increase in the number of times of learning.

Further, the second neural network system according to this application has first and second neural networks, and the first neural network outputs recognition results for all input patterns, so it Performing learning about. As a result, the second neural network recognizes to which category the patterns that are difficult to recognize belong. For this reason, a 20th neural network was provided for each category, and the 10th neural network recognized and output only patterns belonging to its own category that were difficult to recognize. As a result, reliable recognition is possible even when there are a wide variety of categories to be classified. FIG. 2 is a diagram showing the configuration of a network. Also, Figure 3 (
A) to (D) are diagrams showing learning contents of each neural network.

This system is a system for classifying input patterns into either category A or B, and outputs the result y by integrating the outputs of three neural networks NNA, NNB, and NNC and these neural networks. It has a control device 4 to Of these, Neural Network NNA No. 10 has been trained on all the patterns that can be input, and for all the patterns that can be input, the category to which it belongs (A,
The 0 judgment result for judging B) is output as "1', 0",
“1” means that the input cover turn belongs to category A, and “0°” means that the input cover turn belongs to category B. Also, a second neural network NN
B has learned whether or not the input patterns are accurately classified by the first neural network for all patterns that can be input. This learning is performed based on the learning results of the No. 10 neural network NNA. As a result, if the input pattern is difficult to classify correctly in the Neural Network NNA No. 10, 1'' is output.

The third neural network is trained only on patterns in which the second neural network outputs "1", and such input patterns are A and B.
Determine which category it belongs to.

Each neural network NNA, NNB, NNC is
As shown in Figure 2, it has a hierarchical structure consisting of three layers: input layer, middle layer, and output layer.The input layer consists of n neurons, and the middle layer consists of m (man) neurons. The output layer consists of one neuron. A pattern expressed by n row vectors XI to X7 is input to the n neurons of the input layer. Each neuron in the input layer is connected to all neurons in the intermediate layer using different coupling coefficients. Each neuron in the intermediate layer determines the value to output based on the sum of inputs from the input layer. This output value is determined by calculation using a sigmoid function. Furthermore, all neurons in the hidden layer are connected to neurons in the output layer.

The output layer neuron determines the value to be output based on the total value of input values from each neuron in the intermediate layer. This determination is also made by a sigmoid function.

Here, the input value from the neuron in the previous layer is the product of the output value O of the neuron in the previous layer and the coupling coefficient (strength of connection) between that neuron and W. Therefore, the output function of neurons in the middle layer and output layer is the sum of inputs from the previous layer net = ΣW
ij・0. is defined by a threshold function F (net) based on a sigmoid function f (net) for . This darkness value function F (net) is such that the value of f (net) is the threshold h
If it is 1 or more, output “1” and set the threshold h2 (hl>h
2) Output “O” in the following cases. That is, if
f (net)≧hl then F (
net) = 1if hl≦f (net)≦h2
then F (net) = f (net)
if f (net)≦h2 then
F (r+et) = 0.

◆ Neural networks NNA and NN with this configuration
B. Let NNC perform learning using the following procedure.

The first neural network NNA is shown in Figure 3 (A).
The combinations of input pattern groups A and B and their expected output data "1" and "O" as shown in the figure are learned n times. Learning is performed based on a known backpropagation algorithm. That is, the input cover pattern is input and the coupling coefficient is changed so that the difference between the output and expected output data becomes minimum. As a result, the 10th neural network NNA is organized so that it outputs 1" when the corresponding input pattern belongs to A, and outputs "0" when the input pattern belongs to B. .

After training the NNA in this way, we inspected the output data for all training data and learned correctly (
The input cover turns (which were correctly classified) and the cover turns which could not be learned correctly (could not be classified correctly) are distinguished (Figure 3 (B)). Here, failure to learn correctly means that the expected output is “1” even though the expected output is “1”.
” (“O”° or an intermediate value) or “0” even though the expected output is “°0”
This means that a value other than "1P" or an intermediate value is output. At this time, among the input pattern group A, the pattern group that the first neural network NNA was able to correctly identify is the one that Al and NNA were unable to correctly identify. Let A2 be the input pattern group.Also, among the input pattern group B, the pattern group that the first neural network NNA has correctly identified is Bl.

B2 is the human cover turn group that NNA could not correctly identify.
shall be. The second neural network is AI, Bl
When a pattern belonging to is input, outputs “0゛°,
A2. It is learned to output "1" when a pattern belonging to B2 is input (FIG. 3(C)). Through this learning, the second neural network NNB is organized so as to output "1" when a pattern that cannot be accurately classified by the first neural network NNA is input.

Furthermore, the third neural network NNC includes a pattern group A2 that the first neural network NNA could not correctly classify. Only B2 (Fig. 3 (D)) is learned. In this case, the expected output data is A2: “1”
, B2: "'O".

Through this learning, the third neural network NSC is organized so that it can correctly classify patterns belonging to A2 (A) and B2 (B). That is, a group of patterns that are difficult to recognize (a group of similar patterns) A2. By learning only B2, synaptic connections that respond greatly to subtle differences in patterns are formed, making it possible to classify them.

Further, the control circuit 4 has a second neural network NN.
This circuit determines which output data of the first neural network NNA or the third neural network NNC should be adopted as a conclusion based on the output data of the neural network B. In other words, when the output of NNB is "ON", it means that a pattern that can be accurately classified by NNA has been input, so the output data of NNA is adopted as the result output, and the output of NNB is "1". In this case, it means that a pattern that cannot be accurately classified by the NNA has been input, so the output data of the NNC is adopted as the result output.

This ensures that the system's output is always correct.

Here, in this embodiment, one set of NNB and NNC is provided, but when there are multiple combinations of patterns that are difficult to recognize or when there are three or more categories to be classified, NNB,
A plurality of NNCs may be provided.

FIG. 4 is a diagram showing the configuration of a neural network system that is an embodiment of the second invention of this application, and FIGS. It is a diagram showing the contents. Similar to the embodiment of the first invention described above, this system is configured to classify input patterns into either category A or B, and is composed of three neural networks NND, NNE, NNF and these neural networks. It has a control device 5 that integrates the outputs and outputs the result y. Each neural network is similar to that of the first embodiment described above.

The first neural network NND has been trained on all patterns that can be input, and determines the category (A, B) to which each input pattern belongs.

The judgment result is output as "1" and "O"°, where "1" means that the human cover turn belongs to category A, and "O' means that the input cover turn belongs to category B. The second neural networks NNE and NNF respectively receive patterns that belong to category A among patterns that can be input and are not accurately classified by the first neural network, and patterns that can be input that belong to category B and are classified by the first neural network. It is organized to only identify patterns that cannot be accurately classified.

The following learning is performed to organize the neural network in this way.

The first neural network NND is made to learn the combination of input pattern groups A and B and their expected output data "1"°O" as shown in FIG. 5(A), like the NNA of the above embodiment. Accordingly, the 10th neural network NND is organized so that it outputs “1” when the corresponding input pattern belongs to A, and outputs “0°” when the input pattern belongs to B. Ru.

After training the NNA in this way, we inspected the output data for all training data and learned correctly (
The input cover turns that were correctly classified and the input cover turns that could not be learned correctly (could not be classified correctly) are distinguished (FIG. 5(B)). At this time, the pattern group that the first neural network NND correctly identified among the input pattern group A is selected as A1. The incoming cover and turn group that NND could not correctly identify is designated as A2. Further, among the input pattern group B, the pattern group that the No. 10 neural network NND was able to correctly identify is Bl, and the input pattern group that the NND could not correctly identify is B2.

Of these, the second neural network NNE is
It is learned to output "1" only when a pattern belonging to is input (FIG. 5(C)). Through this learning, the second neural network NNE is organized so that it outputs "1" only for the patterns of category A that the tenth neural network NND cannot accurately classify. Furthermore, the second neural network NNF is trained to output "1" only when a pattern belonging to B2 is input (FIG. 5(D)). Through this learning, the second neural network NNF is organized so that it outputs °'1'' only for the patterns of category B that the 10th neural network NND cannot accurately classify.
2. The reason why it becomes possible to classify only B2 is that by learning only A2 or B2, synaptic connections that respond significantly to features that are significant only in these patterns are formed, making it possible to classify them.

Further, the control circuit 5 has a second neural network NN.
When E or NNF outputs "1", this circuit adopts this output value and outputs the result. That is, NN5
．． If the output of NNF is “0”, it means that a pattern that can be accurately classified by NND has been input, so NNF
The output data of ND is adopted as the result output, and if the output of NNE is ° "1", it means that the pattern cannot be accurately classified by NN'D but belongs to category A, so the result is 1". Also, if the output of NNF is "1°", it means that the pattern cannot be accurately classified by NND but belongs to category B, so "O" is output as a result. This ensures that the resulting output of this system is always correct.

(0 Effects of the Invention As described above, according to the neural network system of the present invention, if the neural network that outputs recognition results for all input pattern patterns is not accurate for all input pattern patterns, only inaccurate ones will be recognized by others. By having the neural network recognize it, this "other neural network" learns only about things that are generally difficult to recognize, and only needs to recognize these things, which is difficult to capture with a general neural network. It is organized to respond markedly to subtle differences in patterns, and can perform reliable recognition.In this way, a neural network that performs general operations and a neural network that performs specialized operations are combined. As a result, there is no need to complicate the configuration of each neural network, and there is no need to increase the number of times of learning, so it is possible to simply realize a neural network system with extremely high recognition accuracy.

[Brief explanation of drawings]

Figure 1 is a diagram showing the configuration of a neural network system that is an embodiment of the first invention of this application, Figure 2 is a diagram showing the configuration of a neural network used in the neural network system, and Figure 3 (A ) to (D) are diagrams showing the learning contents of the same neural network. Two major taste proverbs = 4d4 Figure 4 is a diagram showing the configuration of a neural network system that is an embodiment of the second invention of this application, Figure 5 (A)
-(D) are diagrams showing learning contents of each neural network used in the same neural network system. NNA to NNF - neural networks, 4.5 - control circuits.

Claims (2)

[Claims] (1) A first neural network organized to output recognition results for all input patterns, and organized so that the first neural network outputs patterns that are easy to recognize and patterns that are difficult to recognize. The second neural network and the first neural network are trained only on patterns that are difficult to recognize, and the third neural network is organized so that the first neural network outputs recognition results for patterns that are difficult to recognize. A neural network system comprising: a network; (2) a first neural network that is a neural network system that classifies input patterns into a plurality of categories and is organized to recognize and output categories to which all input patterns belong; a second neural network organized to recognize and output patterns that are difficult for the network to recognize; and a plurality of the second neural networks for each category.