JPH0424864A - Data processor - Google Patents

Abstract

PURPOSE:To obtain a data processor with the least number of neurons necessary for the processing purpose by providing plural neural layers and deciding the number of neurons of each layer based on the specific conditions. CONSTITUTION:A data processor 30 is provided with plural neural layers such as an input layer 31, an intermediate layer 32, and an output layer 33. Each of these layers 31-33 contains many neurons (n). Then the neurons (n) are divided into the groups including a fixed number of neurons (n) respectively. Some neurons (n) of each group belong to its adjacent group, and each neuron group is set opposite to the neurons of the neural layer of the next stage. At the same time, the neuron number Nn of the n-th stage is decided by an equation I, where Nn-1 means the neuron number of the (n-1)-th stage together with (a) meaning the neuron number of a neuron group, and (v) meaning the number of rows of neurons (n) belonging to another adjacent neuron group. As a result, all neurons can be effective used.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a data processing device based on the concept of neural networks.

[Conventional technology]

A neural network in this type of data processing device is constructed in a layered manner in which neuron models (hereinafter referred to as neurons) 1 shown in FIG. 4 are arranged in parallel. Neurons in each layer are synaptically connected to all neurons in adjacent layers to input and output data. That is, in neuron 1, externally input data It, I2
, I3...In respectively have weights W, , W, , W
.

, . . . Wn are multiplied, and data 0 is output according to the comparison result between the sum total and the threshold value θ.

Various methods are possible for this comparison, but for example, if normalization function 1 [f] is adopted, the output data O will be as follows: o=1 [ΣWn-In-θ] (1) expressed. That is, when ΣWn-In exceeds the dark value θ, the neuron fires and the output data becomes oG:rl], and ΣWn-In is greater than the threshold θ! At J, the output data 0 becomes "o".

A conventional neural network is constructed by arranging four neurons in parallel to form a neural layer, and by connecting these neural layers in series.

For example, the neural layer is based on Rosenblatt (Rose).
It consists of three layers, ie, an input layer, an intermediate layer, and an output layer, like the bersebutron proposed by Robert Nblatt, and neurons in each layer are synaptically connected to all neurons in adjacent layers.

[Problem to be solved by the invention]

In such a data processing device, neurons in the input layer must be staged in large numbers to process input data, but neurons in the output layer are relatively small in number because they essentially only output processing results. Enough. However, until now, there has been no established theory on how to determine the number of neurons in the hidden layer, and the same number of neurons has usually been provided for all neural layers. For this reason, there was a problem that the number of neurons was limited, and not all neurons were used efficiently.

The present invention was devised to solve these conventional problems, and an object of the present invention is to provide a data processing device configured with the minimum number of neurons necessary for processing purposes.

[Means to solve the problem]

In the data processing device according to the present invention, neurons in a neural layer are divided into neuron groups each consisting of a certain number of neurons, some neurons in each neuron group also belong to adjacent neuron groups, and each neuron group is Corresponds to the neurons of the next stage neural layer, and the number of neurons N11 of the nth stage neural layer is N
It is characterized in that it is determined according to the formula n1(ff-v)/(ET-V))2.

Here, N7-8 is the number of neurons in the (n-1)th stage, a is the number of neurons in one neuron group, and ■ is the number of columns of neurons that also belong to another adjacent neuron group.

〔Example〕

The present invention will be explained below with reference to illustrated embodiments.

FIG. 1 shows a character recognition system having a data processing device according to an embodiment of the present invention. This character recognition system includes a video camera 1o, a preprocessing device 20, a data processing device 3o, a postprocessing device 240, and a display 50. Video camera 1° is provided for inputting characters,
It is connected to the preprocessing device 20. The preprocessing device 2o is, for example, a conventionally known image processing device, and extracts feature data (for example, number of end points, number of branch points, etc.) of the input character, and outputs this feature data to the data processing device 3o.

The data processing device 30 configures a neural network as described later, recognizes characters based on character characteristic data input from the preprocessing device 20, and sends data corresponding to the recognition results to the postprocessing device W40. Output. The recognition signal is, for example, a character code, and the post-processing device 4o stores this output data as, for example, word processing data.
At the same time, it is output to the display 50. display 50
is composed of, for example, a CRT, and displays characters recognized by the data processing device 3 on the screen.

The neural network that constitutes the data processing device 30 is configured as part of computer hardware. This data processing device 3o is schematically represented in FIG. is input layer 31 and output layer 3
3. In this embodiment, each layer 31, 32, 33 has a large number of neurons n, and each neuron n of the input layer 31 has all the neurons n of the intermediate layer 32.
and the neurons n of the intermediate layer 32 are connected to the output layer 3
tlHJE to all neurons n of 3.

Each neuron n is defined as (
1) Output data of "1" or "0" based on the normalization function of the formula. The neuron n is composed of, for example, an operational amplifier, and the weight Wn by which data input to each neuron n is multiplied is obtained, for example, by a variable resistor connected to the input terminal of the operational amplifier. Further, the threshold function is realized by a switching element or the like. That is, by changing the magnitude of the variable resistance according to the output data of each neuron, the weight Wn is changed, the output data is corrected, and learning is performed.

Figure 2 (a), (b), and (C) show the input layer 31 and the intermediate layer 3.
2 and an output layer 33 schematically. The number of neurons decreases in the order of input layer 31, intermediate layer 32, and output layer 33. Here, for ease of explanation, the number of neurons in the input layer 31 is 64, the number of neurons in the intermediate layer 32 is 49, and the number of neurons in the output layer 33 is 36.

Furthermore, in the input layer 31, neurons in each layer are arranged in eight directions in the horizontal and vertical directions, in the intermediate layer 32, seven neurons are arranged in the horizontal and vertical directions, and in the output layer 33, neurons are arranged in the horizontal and vertical directions. There are 6 arrays. Here, the position of the neuron in the lower left corner of the figure is taken as the origin, and the neuron located i-th from the left and j-th from the bottom is n8.

Each neuron in the input layer 31 fires in response to character feature data obtained via the video camera 10. For example, n
The number of endpoints is expressed by the combination of firings of ll, n+zs n2is nzz, and n13, n14, n23, nz4
Suppose that the number of branching points is expressed by the combination of firings. That is, the firing pattern of the neurons in the input layer 31 is artificially determined according to the characters input manually.

On the other hand, in the output layer 33, a character is represented by, for example, a diagonal neuron n1i (neuron surrounded by a broken line) connecting neurons in the lower left corner and the upper right corner. That is, the neurons on this diagonal line represent the character code of the character, and the firing pattern of the neurons in the output layer 33 is artificially determined. In this embodiment, there are 64 types of firing patterns of neurons nB on the diagonal line, and therefore, in this embodiment, 64 types of characters can be recognized, for example, alphabets can be recognized.

Now, before character recognition learning is performed, even if character data is input to the data processing device 30, neurons in the output layer 33 do not fire. This neuron becomes capable of firing through learning, and the learning ends when it exhibits a predetermined firing pattern according to the input character data. Note that the input/output cover patterns for learning are representative data of the data to be processed by the neural network, and the data to be actually processed covers a wide range. Learning is then executed until association with this representative input/output data is properly performed. When learning is completed, the input layer 31 and the output layer 33 exhibit firing patterns that are artificially determined as described above in accordance with the input character data.

The correspondence between neurons in each layer, that is, in the neural layer will be explained.

In each layer, neurons are divided into neuron groups each consisting of a fixed number of neurons, and in this embodiment, one neuron group consists of four neurons. To explain the input layer 3I, as shown in FIG. 2(a), for example, neuron group a is neuron n1bNrl.
zb,, nets nz7, neuron group a2 has neurons nZ&, n3bsnz'y, n3. Each consists of In the same way, by shifting the neuron group a3\a4Sa to the right in the diagram by one neuron.
s\a6\a is defined. Each of these neuron groups a, ”-a.

are the neurons n16, nZ&, of the intermediate layer 32, respectively.
fl 3b~n 46\n sh-, n bb\n t
The neuron group of the input layer 31 corresponding to b is
In the intermediate layer 32, the lower left neuron of this neuron group corresponds to the neuron (n, , ) having the same coordinates as the neuron n16) in the knee group a1.

Similarly to the input layer 31, the neurons in the intermediate layer 32 are divided into neuron groups, and each neuron group is composed of four neurons. Each neuron group corresponds to the neuron of the output Jli 33 in exactly the same manner as described above.

As understood from FIG. 2(a), some of the neurons in each neuron group also belong to adjacent neuron groups. Specifically, for example, neurons nib and fLz1 of neuron group aI belong to neuron group a2 adjacent to the right side, and neuron group a
Neurons n16 and n21+ of No. 1 also belong to the lower adjacent neuron group a8. Thus, each neuron group overlaps other neighboring neuron groups.

Here, the number of neurons in the human layer 3I is Ni, and the input layer 31
Let ta be the number of neurons in a neuron group in , and let V be the number of columns of neurons that also belong to another adjacent neuron group. Assuming that the number of neuron groups in the input layer 31 is X, the following equation holds true as understood from FIG.

(Fi-v) Each numerical value Ni, a, and v should be selected such that may be different.

Since each neuron group in the input layer 31 corresponds one-to-one to one neuron in the intermediate layer 320, the number X of neuron groups in the input layer 31 is equal to the number of neurons in the intermediate layer 32. Therefore, if the number of neurons in the intermediate layer 32 is set as Nm, then Nm= ((FNT-v)/(Li-v))"<2
).

Generalizing such a relationship, the number of neurons N7 in the n-th neural layer is N −= ((, r
N;Lv)/(Fdv))'' (3) where Nn-1 is the number of neurons in the (n-1)th stage neural layer.

The number of neurons in the input layer 31 is determined by the processing content for input data. That is, in the case of character recognition, for example, the number of neurons in the input layer 31 is determined depending on the type of character feature data. - On the other hand, the number of neurons in the output layer 33 is determined by the output content. In the case of character recognition, it is determined by the type of character code of the character, and for example, in a system that recognizes alphabets, 64 codes may be provided. Therefore, the number of neurons in the input layer 31 and the output layer 33 is determined depending on the processing content of the data processing device. On the other hand, the number of neurons in the intermediate layer 32 is determined by the number of neurons in the input layer 31, the number of neurons in the neuron group, etc., as shown in equation (2). Of course, the relationship between the numbers of neurons in the intermediate layer 32 and the output layer 33 is similar to the relationship between the input layer 31 and the intermediate layer 32, and must satisfy equation (3).

Therefore, the three neural layers of input layer 31, intermediate layer 32, and output layer 33 are not necessarily provided as shown in FIGS. 1 and 2 (a) to (C), but two or more intermediate layers are provided. may be provided. Therefore, next, a method for determining the number of neural layers will be explained.

As described above, the number Ni of neurons in the input layer 31 and the number NO of neurons in the output layer 33 are determined depending on the purpose of the data processing device. On the other hand, the number a of neurons included in a neuron group and the number V (overlap rate) of columns of neurons that also belong to another adjacent neuron group are determined experimentally. For example, a is "4" and V is "1" (in this case, the overlap rate is 50%).

Now, when the number Ni of neurons in the input layer 31, the number a of neurons in the neuron group, and the overlap rate V are determined, the number N2 of neurons in the second layer (first intermediate layer) is determined.
is determined by equation (3).

Then, once the number N2 of neurons in the second layer is determined, the number N3 of neurons in the third layer (second intermediate layer) is similarly determined from equation (3). However, this calculation is the (n+
1) The number of neurons in the layer is the number of neurons in the output layer 33
This continues until the nth layer becomes smaller than the output layer 33, and the nth layer becomes the intermediate layer immediately before the output layer 33.

In the above embodiment, the number of neurons included in one neuron group was four, but the number is not limited to this, and any number can be selected as necessary.

Further, the number a of neurons in the neuron group and the overlap rate (2) do not need to be the same values from the input layer 31 to the output layer 33. That is, when a large number of intermediate layers 32 are provided, in order to make each neuron group in the final intermediate layer correspond one-to-one to each neuron in the output layer 33, this final intermediate layer is different from other intermediate layers. Different number of neurons a
Alternatively, it may be necessary to have an overlap ratio V.

Note that the output layer does not need to be configured to represent a character code by neurons on the diagonal, and may be configured to define a recognized character by all neurons.

Furthermore, in the above embodiment, each layer has a square shape, and the neurons are arranged like the eyes of the base, but this arrangement is conceptual, and in reality, it is not physically arranged like this. Not necessarily. For example, neurons may be constructed from computer memory, in which case the physical placement of neurons in memory does not matter at all.

Furthermore, the present invention can be applied not only to character recognition but also to graphic recognition or speech recognition.

[Effects of the Invention] As described above, according to the present invention, it is possible to obtain the effect that a data processing device can be configured with the minimum number of neurons necessary for a processing purpose.

[Brief explanation of drawings]

Fig. 1 is a schematic configuration diagram showing a character recognition system to which an embodiment of the present invention is applied. Fig. 2 is a conceptual diagram showing neurons in the input layer, intermediate layer, and output layer. Fig. 3 is an overlapping diagram of each neuron group. FIG. 4 is a conceptual diagram showing an example of a neuron. 1,,N... Neuron 31... Input layer 32... Intermediate layer 33... Output layer (a) (C) Fig. n Fig.

Claims (1)

[Claims] (1) A data processing device equipped with a multi-stage neural layer consisting of a plurality of neurons that fire and output predetermined data based on the result of performing predetermined processing on input data, the neural layer comprising: Neurons are divided into neuron groups consisting of a certain number of neurons, some neurons in each neuron group also belong to adjacent neuron groups, each neuron group corresponds to a neuron in the next neural layer, and The number of neurons N_n in the nth stage neural layer is N_n = {(√N_n_-_1-v)/(√a-v)}
A data processing device characterized in that it is determined according to the formula ^2. However, N_n_-_1 is the number of neurons in the (n-1)th stage, a is the number of neurons in one neuron group, and v is the number of columns of neurons that also belong to another adjacent neuron group.