JPH0464183A - Weighted majority decision device by neural network and its manufacture - Google Patents

Abstract

PURPOSE:To perform weighted majority decision by performing relation between input variables i.e. weighing by performing learning by using a neural network. CONSTITUTION:Signals on which weight W1-Wn are multiplied, respectively are outputted from neuro units 7-1 to 7-n in an intermediate layer. Input signals X1-Xn to the neuro units 6-1 to 6-n of an input part are inputted to the intermediate layer and division parts 1-n, 5 as it is, and X1W1-XnWn are outputted from the neuro units 7-1 to 7-n in the intermediate layer, and are inputted to the division parts 1-n and 5, and weighted mean i.e. a weighted majority decision arithmetic operation is performed. The weighing can be performed by performing the learning by plural input X1-Xn and a teacher signal D for them. In such a way, it is possible to realize the weighted majority decision by the neural network.

Description

[Detailed description of the invention] [Table of contents] Overview Industrial field of application Conventional technology Problems to be solved by the invention Means for solving the problems (Fig. 1) Working examples (Figs. 2 to 7) Effects of the invention [Summary] Regarding a weighted majority voting device using a neural network and a method for manufacturing the same, the purpose of the present invention is to provide a device that can perform weighted majority voting calculations even when the weight values of multiple inputs are not known. A network-configured data processing device that sends input from the previous layer to the middle layer and performs calculations has a plurality of middle layer units composed of neuro units, the output of the first layer unit, and the middle layer. There is a front-stage division section that performs division processing based on the output of the layer unit node, and a rear-stage division section that performs division processing based on the output of the front-stage division section and the output of the middle layer unit. The output of the intermediate layer unit is transmitted to the intermediate layer unit and the preceding division section, and the output of the intermediate layer unit is weighted and transmitted to the preceding division section and the subsequent division section, and a weighted majority decision is performed on a plurality of input signals. Configure to perform calculations.

Further, the device is manufactured by separately constructing the front-stage division section and the rear-stage division section, and connecting these to the front-stage unit and the intermediate-layer unit.

[Industrial application field]

The present invention relates to a weighted majority voting circuit using a neural network.

In recent years, IF-T has been used to control multi-input, multi-output systems such as plants.
There are examples of applying expert systems and fuzzy systems based on HEN-type rules.

Particularly in plant control, the weighting (level of importance) between input variables is important, and control is performed based on control rules created based on this. The present invention uses a neural network to implement a weighted majority circuit necessary for constructing such a control system.

[Conventional technology]

When building a plant control system, IF-T is usually
Constructed using HEN rules. For example, it is necessary to determine what kind of operation should be performed under what pressure and temperature conditions. Depending on the type of plant, for example, there are multiple checkpoints for pressure and temperature, and there are also multiple checkpoints for temperature, so there are dozens of these variables.
There are times when it exceeds the individual.

Therefore, it is important to weight each variable, but when constructing a plant control system like this, it is necessary to specifically analyze what processing is being performed depending on the state of each variable in time with the on-site operator. It is necessary to.

However, since the degree of importance differs depending on the variable, determining which variable to focus on in operations depends on the state of each variable. It is impossible to get an answer from the operator. Moreover, it was difficult for even the listeners to accurately and consistently understand the importance of these multiple variables.

[Problem to be solved by the invention]

For example, when building a plant control system, if you have no knowledge of the target process, or if you have very little knowledge, you will have to resort to a trial-and-error approach to determining control rules, making it difficult to efficiently design the system. It was impossible to do.

Moreover, the control rules determined in this way require a great deal of effort in tuning work after the system is in operation, that is, adjusting the weights between input variables.

Therefore, the present invention solves these problems and makes it possible to use a neural network to learn the relationship between input variables, such as in plant control, that is, weighting, and this weighting realizes majority voting. The purpose is to

[Means to solve the problem]

In order to achieve the above object, in the present invention, as shown in FIG. A neural network is constructed, and weighting is used to obtain a majority voting circuit.

Neuro unit 6-1, which is an input section, receives input signal X1
is directly output to the neuro unit 7-1 of the intermediate layer and the neuro unit l-1-2 which is the input section of the division section 1. In the neuro unit 71, this input signal
XIWI multiplied by the weight Wl is output to the neuro unit 1-1, which is the input part of the divider l, and the neuro unit 5-1, which is the input part of the divider 5, respectively.

Similarly, the input unit neuro unit 62.6-3
-6-n are input signals X2, X3-Xn, respectively
Output each. And the middle layer neuro unit 7-2.7-3-4-n receives input signals X2, Xs-
X2W, which is Xn multiplied by weight Wn and W3-Wn, respectively
2, output X 3 W 3-X n W n.

The divider l divides the input signal of the neuro unit it of the input part by the input signal of the neuro unit 1-2 and outputs the value from the neuro unit 1-3 of the output part, In the above case, XIWI:X1=W1 is output.

Similarly, the dividing sections 2.3-...-n divide the input signal of the neuro unit 2-1.3-1-n-1 of the input section into the neuro-unit 2-2.3-2-n-2. The value divided by the input signal of is output to the neuro unit 2-3.3 of the output section.
-3...-n-3, and in the above case, WnI and Ws---Wn are output, respectively.

The division section 5 is constructed in the same manner as the division section 2.3-n, but the neuro unit 5-1 of the input section is provided with the neuro unit 7-1.7-2.7 of the intermediate layer. -3-4-n output signal X I W 1, XtWa, X3W3-・
X n W n are transmitted and their sum becomes the input signal. The neuro unit 5-2 as an input section has the neuro unit 1-3 as an output section of each of the division sections 1 to n.
2-3. Output signals W1, Wn, W from 3-3n-3
3-W n are transmitted and their sum becomes the input signal.

[Effect]

Divider 1.2.3-n and divider 5 are input x
, y is configured to perform a division operation such that the output Z becomes 2=□, and the middle layer neuro unit 7-1.
7-2.7-: From 3-7-n, each weight W 1
, Wn, and a signal multiplied by W3-Wn is output.

Therefore, the neuro unit 6-1.62.6- of the input section
3...6-n are input signals XI, X!, respectively. ,X5-
When Xn is input, these input signals are directly input to the middle layer and the division section as shown in FIG. 1, and the middle layer neuro units 7-1, 7-2, ? -3-...1-
From n, respectively, XIWl, XaW2, X '3 W
3-X nWn is output and input to the divider 1.2.3-n and the divider 5 as shown in FIG.

Therefore, in the divider 1, X IWI÷X 1 = W1
is calculated and output, and similarly from the dividing unit 2.3n to Wn
, Ws-Wn are output.

With these, the neuro unit 5 of the input section of the division section 5
-1 has (XtW++XxWs+X5W3-+ X n
W n ) is input and calculated, and the neuro unit 5-2 also receives (W s + W 2 + W 3-+W
n ) is input and calculated. Therefore, in the division section 5, these divisions are performed, and the neuro unit 5-
3 outputs the following output signal Z.

X s +X 2 +X 3-4Xn In this way, a weighted average, that is, a weighted majority calculation can be performed.

Note that the division section 1.2.3-n and the division section 5 may be constructed of neuron or other calculation means, and are capable of dividing x/y in advance. Configure it.

Then, as described later, multiple inputs X1, X2, X3
− and its teacher signal, it becomes possible to weight it by learning it, so weighted majority voting can be done by neural
It can be realized by a network, and by learning the neural network, the input variables X1, X2, X3
.about.Xn, or weighting becomes possible. , [Example] An example of the present invention will be described based on FIGS. 2 to 7.

Fig. 2 is a block diagram of an embodiment of the present invention, Fig. 3 is a block diagram of its division section, Fig. 4 is a diagram explaining the learning state, Fig. 5 is a basic block diagram of the neuro unit, and Fig. 6 is a block diagram of an embodiment of the present invention. 7 is a basic configuration diagram of a hierarchical network in a neuro system, and FIG. 7 shows a general neuro system, with (A) showing an example of a neuro unit and (B) showing an example of a hierarchical network.

First, before describing the present invention in detail, the neuro unit constituting the present invention will be explained with reference to FIG.

In FIG. 5, 21 is a neuron unit, and 22 is a weight value of each internal connection (Wl, W2 to
23 is an accumulation processing section that adds all the multiplication results X1.W1 to X5.Ws output from the multiplication processing section 22; Reference numeral 24 denotes a dark value processing section that performs nonlinear dark value processing on the accumulated value of the accumulation processing section 23 using, for example, an S-shaped function (sigmoid function). Of course, other functions, such as the identity function of f (X)=X, can be used instead of the S-shaped function, depending on the application.

The calculations performed by this neuro unit 21 are expressed as follows.

If the input signals are Xl and X2X5, then the output Y of the adder 23 is as follows: Y=(X1-Wl +X2 ・W2+-+Xs-
Ws)=ΣXl-Wi (In the example of FIG. 5, n=5.) Threshold processing unit 2
When the darkness value is θ, Wi and θ are variable in each of the above equations.

Note that the neuro unit 21 is specifically configured as shown in FIG. 7(A). In FIG. 7(A), 22a is a multiplication type D/A converter, 23 is an accumulation processing section that includes an analog adder 23a and a sample hold circuit 23b, 24 is a threshold processing section, and 25 is an output holding section. 26 is an output inch part, 27 is an input inch part, 28 is a weight holding part, and 29 is a control circuit. In FIG. 7(A), the same part as in FIG. 5 indicates the same part.

The input switch section 27 receives input signals Xs, X2, X3--
are inputted sequentially, and turned on is controlled in synchronization with this input timing. Then, weight signals w1, w2, w3- are transmitted according to this input timing, but at this time, the weight input control signals are sequentially output from the control circuit 29 to the receiver, and this weight signal W s
, W 2 and W 3- are sequentially sent to the weight holding unit 28.

In this way, in the multiplication type D/A converter 22a, XIWl, X2W2, X3W3.- are sequentially calculated.

The accumulation processing section 23 includes an initial sample hold circuit 23b.
has been cleared to zero, so XIW1 and this zero are added in the analog adder 23a, and the obtained X s
W 1 is retained. Next time X2W2 is input,
The analog adder 23a adds this X 2 W 2 to X I W 1 held in the sample and hold circuit 23b, and holds the obtained (XIWI+X 2 W 2 ) in the sample and hold circuit 23b. In this way, the accumulated value Y=(XsWx+X 2 W 2 + X s
W3...) is calculated.

In this manner, when the accumulation processing in the accumulation processing section 23 is completed, the control circuit 29 outputs a conversion control signal, and then outputs an output control signal. In response to this, the threshold processing unit 24 performs threshold processing on the accumulated value Y, and the obtained output Z
is temporarily held by the output holding unit 25, and the output switch 26
This output Z is output by being controlled to be turned on.

The neurocomputer uses the above neuron unit 21 to perform parallel and distributed processing using a hierarchical network as shown in FIG.

In FIG. 6, a plurality of neuro units 21h constituting an input layer directly distribute and output the input signals to a plurality of neuro units 211 constituting an intermediate layer without performing weighting or threshold processing on the input signals. be.

NeuroUnino 21i, which constitutes the intermediate layer, weights and accumulates these input signals, subjects them to dark value processing, and outputs them.

Similarly to the intermediate layer, the neuro unit 21j constituting the output layer weights and accumulates each input signal, and
This is then subjected to threshold processing.

Then, assuming that the intermediate layer i is the first stage and the output layer j is the second stage, the accumulation processing section of the iNi-th neuro-uni-node executes the calculation of equation 11 below, and the dark value processing section executes the following calculation. (2) Process so that the second dark value calculation process is executed.

Ypi=ΣXph Wih -1f)Zpi=1/(
1 + exp (ypi + θi) - (21, however, h Unit, To number of Sh layer (input layer) p: Input signal pattern number 018 Darkness value of the 1st unit of the 1st layer (intermediate layer) Wih: h
Weight value Xph of internal connection between
It is specifically constructed as shown in the figure.

In FIG. 7(B), 70 is an analog bus,
As shown in FIG. 7(B), a device 71 connects the neuro knee y) 21h that makes up the input layer, the neuro unit 21i that makes up the intermediate layer, the neuro unit 21j that makes up the output layer, etc., as shown in FIG. 7(B). 72 is a signal holding circuit that holds an input signal to the neuro unit 21h constituting the input layer; 73 is a synchronization control signal that is a control signal for data transfer; A synchronous control signal line 74 for transmitting the data is a main control unit that comprehensively controls the hierarchical network.

Note that the weight value of the neuro unit 21h in the input layer is 1 as described above, so these neuro unit 2
A numerical value 1 is written in each of the weight output circuits 71 of 1h.

Incidentally, in a data processing apparatus having a hierarchical network configuration as shown in FIG. 6, it is necessary to obtain weight values and the like of the hierarchical network structure that define the data conversion function through a learning process. As an algorithm for this learning process, the bank propagation method is attracting attention due to its high practicality. This bank propagation method learns by adjusting the weight values and shadow values of a hierarchical network using error feedback.

For example, the first numerical input signal (Xi, X 2 ...)
The neuro unit 21 of the output layer at this time
Each output (Zt, Z2...) of j is compared with the numerical teacher signal (Dt, D21.) by the subtraction unit 30, and its error Δdl-(Dl-Zl), Δd2=(D2-2
2) Find −・−.

First, the dark value and weight value Wji of each neuro unit 21j in the output layer are adjusted so that Δdl and Δd2- are small, and then the dark value and weight value Wjh of the neuro unit 21i in the middle layer are similarly adjusted. adjust.

Then, the second numerical input signal (X,', Compared with the numerical teacher signal (D, D2'-...), its error Δd,'=(D-z+'...), Δdz'
Find = (Dz'Zz') -. Then, the threshold value and weight value of the neuro unit 21j in the output layer are adjusted in the same manner, and then the darkness value and weight value of the neuro unit 21i in the intermediate layer are adjusted. This is done based on a plurality of input signals and a teacher signal, and learning is performed.

Naturally, such learning is necessary for the embodiment of the present invention shown in FIG. By the way, as shown in FIG. 2, the present invention includes dividing sections 1, 2, 3, 4 and 5. Therefore, if these dividers 1 to 5 are configured as a unit, and a circuit as shown in FIG. Part is neuro unit 7-1.7-2.7-3.7-4
This makes learning very easy.

The configuration of this dividing section will be explained based on FIG. 3, taking the dividing section 1 shown in FIG. 2 as an example.

As shown in FIG. 3(A), the division section includes neuro units 1-1, 1-2 in the input layer, neuro units I-4 and 1-51 in the intermediate layer, and neuro units 1 in the force layer. −
It consists of 3. Each neuro unit is configured as illustrated in FIGS. 5 and 7(A), but the input layer neuro unit outputs the input value as it is, and the weight value is 1.

By the way, the learning of the neuro unit of the division section is performed in the following manner using the subtraction section 10 as shown in FIG. 3(A). As data used for learning, for example, the third
The material shown in Figure (B) was used. x and y are input values, and D is a teacher signal.

First, as pattern 1, x = 0.500y = 1. Use ooo D =0.500. X is input to the neuro unit 1-1 of the input section, and y is input to the neuro unit 1-2 of the input section. The teacher signal is input to the subtractor 10, and the output signal Z of the neuro unit 1-3 of the output section is input to the subtracter 10, and the subtracter 10 outputs an error signal corresponding to the difference between Z and D. For example, (Z-D)" is output.

In this case, learning is performed by the back propagation method.

■ By inputting the pattern 1, the error signal is output from the subtraction unit 10, but the neuro unit 1
-3, adjust the weight values WI6, W, 6 and the threshold QIff. Naturally, these are adjusted so that the error signal becomes small.

■ Following the above adjustment for neuro units 1-3, now neuro unit 1 for the same pattern 1.
Adjustment of the weight value W1 and W1□ and the dark value Q at -4,
Weight values W11, W in neuro unit 1-5,
4. Adjustment of threshold value Q, 2 is performed.

After learning is performed for pattern 1 in this manner, adjustment for pattern 2 is performed in the same manner.

Adjustments are made for pattern 3.4-.- below. After adjustment is made to each pattern, adjustment is made again to patterns 1.2-.-. This adjustment is repeated several times until the error of Z = x / y falls within an acceptable range. When the allowable range is reached, the division part l is obtained.

In this way, the dividing sections 2.3.4.5 are configured independently in advance. And, as shown in Figure 2,
The input layer neuro units 61 to 6-4 and the intermediate layer neuro units 7-1 to 7-4 are combined to form a weighted majority voting circuit. In the example shown in FIG. 2, there are four input signals x1 to x4.

Here, the neuro units 6-1 to 6-4 use the identity function whose weight value is the numerical value l and f (X)=X as the dark value processing function, so the input values are output as they are.

Therefore, when the input signal X1 is applied to the neuro unit 6-1, the neuro unit 6-1 directly transmits the input signal Xr to the neuro unit 7-1 and the divider 1.
input to the neuro unit 1-2 of the input section. When the input signal X2 is applied to the neuro unit 6-2, the neuro unit 6-2 transmits the input signal X2 to the neuro unit 7-2 and the neuro unit 2-2 at the input section of the dividing section 2. input.

The same applies to the neuro unit 6-3 and 6-4.

Further, the weight value of the neuro unit 7-1 is Wl,
Since the identity function f (X) = X is used as the dark value processing function, when the signal X1 is input, X I W
1 to the neuro unit 1-1 at the input section of the division section 1 and the neuro unit 5- at the input section of the division section 5, respectively.
1.

Since the neuro unit 7-2 has a weight value of Wz and uses an identity function as a dark value processing function, when the signal 2-1 and the neuro unit 5-1 of the division section 5.

The weight values of NAO and Neuro Unisol l-7-3 and 7-4 are W 3 and W 4, respectively, and both use an identity function as the dark value processing function, so the signal X3 or X4 is input. , X3W3 is divided by neuro・
Unit 3-1 and neuro unit 5- of division section 5
1 and inputs X4W4 to the neuro unit 4-1 of the division section 4 and the neuro unit 7-51 of the division section 5.

Therefore, in the division section 1, X I W ! ÷X1=W1 is calculated, the divider 2 calculates X 2 W 2÷X2=W2, the divider 3 calculates X 3W 3÷X3°W3, and the divider 4 calculates X a W 4 ÷XA=W4 is calculated. And these outputs W1 to W4 are each divided by 3E
Parts 1-4 Neuro-uni-no) i3.23.3-3.
4-3 and input to the neuro unit 5-2 of the input section of the dividing section 5.

At this time, the other neuro unit 5-1 of the input section of the division section 5 has X t W l, X2W!, as described above.
, X 3 W 9, X4W4 are input. Therefore, the division unit 5 can obtain the weighted majority calculation (weighted average calculation) as follows, and can obtain the output Z of the calculation result.

By the way, an example of learning data for the weighted majority circuit configured as described above is shown in FIG. 4(A).

Patterns 1 to 30 shown in FIG. 4(A) were used as X1 to X4 and teacher signals input to the neuro units 6-1 to 6-4 of the input layer.

In the present invention, as described above, the division units 1 to 5 have been trained separately prior to this learning, so there is no need to adjust these neuro units. Further, as described above, the input layer neuro units 6-1 to 6-4 have weight values of 1 and the input values are output as they are, so there is no need for adjustment. Middle layer neuro unit 7
-1 to 7-4, as mentioned above, the dark value processing function uses the identity function, so there is no need to adjust the threshold, and in the end, learning is based on the difference between the output value Z and the teacher signal (Z -D), for example, (Z-D)'', the update amount of the weight values W! to W4 may be determined.

By performing learning in this way, as shown in FIG. 4(B), before learning, W s = 0.012 W 2 = 0.038 W 3 -0.025 W 4 = 0.089. However, after learning, W s = 0.981 W 2 = 1.547 W 3 = 2.048 W 4- 2.485 By the way, D in Figure 4 (A) is Wl : Wz
:Ws :Wa=1 :1.5 :
2.0: Calculated value when 2.5. As a result, after learning, the weights become approximately close to the design values, and thus the weighted majority calculation can be performed as described above.

In the above embodiment, an example was explained in which the number of input signals was four, X1 to X4, but the present invention is of course not limited to this, and the number can be increased or decreased.

Furthermore, although an example has been described in which the division section is constructed of a neural network, the invention is of course not limited to this, and a division section that is not based on a neural network can also be used.

〔Effect of the invention〕

According to the present invention, as long as it is known what kind of output is produced in what state of each of a plurality of input signals, by learning based on that, weighted majority voting among input variables can be performed as described above. Realization becomes possible.

In this way, for example, it has become possible to learn functions between input variables in plant control using a neural network, and it is no longer necessary to spend a great deal of time and effort on tuning work after operation as in the past.

Pattern diagram, Fig. 4 is an explanatory diagram of the learning state, Fig. 5 is an explanatory diagram of the neuro unit, Fig. 6 is a basic configuration diagram of a hierarchical network using neuro units, and Fig. 7 (A) is the neuro unit. An example configuration diagram of
(B) shows an example of a hierarchical network.

1.2.3.4.5 - Division unit 6-1 to 6-4 - Human power layer neuro unit 7-1 to
'1-4-・~Middle layer nonneuro uni・7to22~
Multiplication processing section 23-accumulation processing section 24-dark value processing section

[Brief explanation of the drawing]

FIG. 1 is a diagram explaining the principle of the present invention, FIG. 2 is a configuration diagram of an embodiment of the present invention.

Claims (3)

[Claims] (1) In a network configuration data processing device that sends input from the previous layer to the middle layer and performs calculations,
A plurality of middle layer units (7-1,
7-2...), a pre-stage unit that performs division processing based on the output of the previous layer unit (6-1, 6-2...), and the output of the middle layer unit (7-1, 7-2...) The output of the division section (1, 2...), the output of the previous stage division section (1, 2...), and the intermediate layer unit (7-1, 7
A post-stage division section (5) is provided which performs division processing based on the output of -2...), the output of the previous stage unit is transmitted to the intermediate layer unit and the previous stage division section, and the output of the intermediate layer unit is A weighted majority decision device using a neural network, characterized in that the weighted signals are transmitted to the first-stage division section and the second-stage division section, and perform a weighted majority decision operation on a plurality of input signals. (2) The weighted majority voting device using a neural network according to claim 1, wherein a part or all of the first-stage division section and the second-stage division section are constructed by arithmetic means using a neuro unit. (3) The front-stage division section and the rear-stage division section are each
A weighted majority voting device using a neural network characterized in that it is composed of units, each of which is adjusted individually, and then connected to a front layer unit consisting of neuro units and a middle layer unit to perform adjustment. manufacturing method.