JPH0464132A - Learning method for neural unit with fully coupled rule part - Google Patents

Abstract

PURPOSE:To improve the efficiency of learning by initializing a weight value and a threshold value at each of an antecedent part membership function realizing part, rule part, and consequent part membership function realizing part, and then learning only the weight of the rule part using a system learning data prepared preliminarily. CONSTITUTION:The weight value and the threshold value based on knowledge stored preliminarily or random numbers are initialized at a rule part entire connection neuro 100 being a hierarchical network construction, constituted of an antecedent part membership function realizing part 10, rule part 12, and consequent part membership function realizing part 14, which realizes the fuzzy control rule of an object system. Then, only the rule part 12 is learned by using the learning data prepared preliminarily. Thus, the learning to operate a control suited to an object system can be realized efficiently.

Description

[Detailed Description of the Invention] [Summary] A control output corresponding to an input according to a hierarchical name network structure that realizes a fuzzy control rule consisting of an antecedent membership function realization part, a rule part, and a consequent part membership function realization part. Regarding the learning method of the rule part fully connected neuron that generates, the purpose is to efficiently perform learning for controlling suitable for the target system. After setting the initial values of weights and thresholds for each section by introducing pre-existing knowledge, using random numbers, or a combination thereof, only the weights of the rule section are learned using system training data prepared in advance. Configure it as follows.

[Industrial application field]

The present invention relates to a learning method for a fully connected rule part neuron that combines fuzzy control and a neural network.

In information processing using neurons, learning data consisting of a set of input patterns of the target system and desired output patterns for that input pattern is presented to a hierarchical neural network to learn and perform adaptive processing. Let them do it. In particular, a processing method called pack propagation method is attracting attention due to its high practicality in this learning process.

On the other hand, fuzzy theory was proposed by Zadeh in 1960, and is based on the ambiguity that humans perform, such as whether the temperature is "low" or "high" (it is a theory that models the process of thinking and judgment. The theory is based mathematically on fuzzy set theory, which introduces membership functions to represent ambiguity.

As an application of fuzzy theory, in 1974 Maqmd
Ani first used it to control a steam engine.
Various types of fuzzy control are performed.

Since the system that performs this fuzzy control handles uncertain things, it is necessary to have a configuration that can adaptively respond to adjustments and changes in the control algorithm.

[Conventional technology]

Generally, the following procedures are taken to construct a fuzzy control system.

■Obtain ambiguous control rules such as ``If the temperature is high, reduce the fire'' held by experienced operators.

■ Quantify the meaning of the words (life B) in the control rules in the form of membership functions, and convert the control rules into rlF-THEN~
” Described using a type rule expression.

■Conduct inspections through simulations and on-site tests.

■Evaluate inspection results and improve membership functions and rules.

To explain in more detail, the fuzzy control rules are, for example, rif x, is big and Xg is s
It is written as +wall then.

Here, the IF part is called the antecedent part, and is the control state quantity such as temperature (
This is the part that describes the conditions for (control input). Further, the THEN section is called a consequent section, and is a section that describes conditions regarding the control operation amount (control output) of the operating end, etc.

Fuzzy control manages multiple fuzzy control rules prepared as the control logic of the target system, and also manages the meaning of ambiguous linguistic expressions such as "large" or "small" written in each fuzzy control rule. Quantify and manage it as a ship function.

The control procedure of the completed fuzzy control system is as follows.

■When a controlled state quantity such as temperature is input from the controlled object, the truth value is calculated according to the membership function of the antecedent part that has been set.

(2) Next, in accordance with the antecedent computation such as selecting the minimum value, a process is executed to determine the value to be applied to the consequent in each fuzzy control rule.

For example, the truth value of "r XI is big" = 0.8'
If the truth value of ``2 is small'' = 0.5, then according to the antecedent operation that selects the minimum value, the truth value = 0.
．． 5 is output as the applied value to the consequent of the fuzzy control rule.

(2) Next, a process is executed to determine the output application value from the input value (common value) from the antecedent part to the membership function of the consequent part, which has the same amount of control operation.

For example, with two fuzzy control rules, ' Vr
Application value of is bigj -0,5ry, is bi
If the common value -0.6 of "g" is obtained, the common value = 0.6 is selected and determined according to the consequent operation that selects the maximum value.

■Subsequently, the membership function of the control operation amount is reduced (or expanded) according to the determined applied value, and the center of gravity of the figure of the function part of the membership function for the same control operation amount is calculated. The control operation amount, which is a fuzzy inference value, is calculated and output.

By the way, fuzzy control rules are planned to be generated according to the knowledge possessed by a skilled operator, and it is difficult to establish fuzzy control rules that can realize optimal control from the beginning. Therefore, it is necessary to evaluate the generated fuzzy control rules through simulations and on-site tests, and to improve them through trial and error.

Therefore, the inventors of the present application developed a hierarchical network structure that realizes fuzzy control rules by applying neurology.
In other words, we are proposing an adaptive fuzzy control system using the Briwire neuro in the rule section (Patent Application No. 2002-6).
No. 0256 "Hierarchical Network Configuration Data Processing Device and Data Processing System"). That is, the rule section Briwire Neuro has a hierarchical network configuration that realizes a fuzzy control rule consisting of an antecedent membership function realization section, a rule section, and a consequent membership function.

Considering the automatic extraction of fuzzy control rules by the pre-wired neuron of the rule section, the following procedure may be taken.

■Set the membership function of the rule section prewire neuro.

■The rule section pre-wire neuro is presented with an input pattern of the target system and a set of desirable patterns for that input pattern, and is made to learn.

■Extract the optimal fuzzy control rule for the target system by analyzing the weights of connections in the rule section.

[Problem to be solved by the invention]

However, the learning that is performed to extract fuzzy control rules for the rule part fully connected neuron involves the learning process, including the weights of the antecedent membership function part, the rule part, and the consequent part membership function realization part. There is no fixed learning method for how to perform this, and there is a need to develop an efficient learning method that can maximize the performance of fully connected rule-based neurons.

The present invention was made in view of the above situation, and an object of the present invention is to provide a learning method for a fully connected rule section neuron that can efficiently perform learning for controlling a target system. .

[Means to solve the problem]

FIG. 1 is an illustration explaining the principle of the present invention.

First, the present invention provides antecedent member simplification function realization unit 10.1
Or, it has a hierarchical network structure consisting of the rule section 12 and the consequent membership function realization section 14 in multiple layers, and between the internal layers of the rule section 12 and the preceding antecedent membership function realization section 10 and All units are internally coupled with the consequent membership function realization unit 14 in the subsequent stage, and the input control state quantities (X, , X, ,
...X, ,) corresponding to one or more control operation amounts (
The target is a fully connected rule part neuron that outputs Y).

In the present invention, as a learning method for such a rule part fully connected neuron, <First process> Each of the antecedent part membership function realization part 10, the rule part 12, and the consequent part membership function realization part 14 is Set initial values of weights and thresholds.

<Second process> Only the rule section 12 is trained using learning data prepared in advance.

A learning method with a processing process is constructed.

Here, as a method for setting initial values of weights and thresholds in the first process, initial values created based on knowledge held in advance (indicated as "priori knowledge J" in the figure) are set.

Another method for setting initial values of weights and thresholds in the I-th process is to set initial weight values created using random numbers.

Further, in setting the initial values of the weights and thresholds in the first process, those that can be set using knowledge held in advance are initialized using this knowledge, and other initial values may be set using random numbers.

[Effect]

According to the learning method according to the present invention for a fully connected rule part neuron having such a configuration and defined in FIG. 3, the following effects can be obtained.

First, we need to know in advance about the rule part fully connected neuron, which has a hierarchical network configuration consisting of an antecedent membership function realization part, a rule part, and a consequent membership function realization part, which realize the fuzzy control rules of the target system. Initial values of weights and thresholds are set based on random numbers, initial values of weights and thresholds are set based on random numbers, or both are used in combination.

Next, the learning process is started, the input pattern for learning is presented to the rule part fully connected neuron, and the output pattern from the rule part fully connected neuron is a learned output pattern (teacher signal ), learning is performed using, for example, a backpropagation method.

Furthermore, learning efficiency can be improved by initializing the weights and thresholds of the antecedent membership function realization section and the consequent membership function realization section into which knowledge held in advance is introduced.

Furthermore, the performance of the network can be maximized by initializing the weights and thresholds of the antecedent membership function implementation section, the rule section, and the consequent membership function implementation section using random numbers.

〔Example〕

FIG. 2 is a learning operation flowchart showing a processing procedure as a first embodiment of the rule part fully connected neuro learning method according to the present invention. The rule part fully connected neuron that is the object of the learning method according to the procedure shown in FIG.
2, and a consequent membership function implementation unit 14 (including a centroid calculation implementation unit 18), the system has a hierarchical network structure for realizing fuzzy control rules.

In the learning process shown in FIG. 2, the following Ste knobs 31 to S
The process consisting of 6 is performed.

<Step 31> Weight Cab of the antecedent membership function realization unit 10.

Initial settings are made by introducing knowledge that is previously held so as to realize a membership function given Cbc.

<Step S2> The weights Ccd and Cde of the rule section 12 are initialized using random numbers.

<Step S3> The consequent part membership function realization unit 140 introduces and initializes the knowledge held in advance so as to realize the membership function given the weight Cef.

<Step S4> The weight Cfg of the center of gravity calculation realization unit 18 is initially set to realize center of gravity calculation. Incidentally, in FIG. 5, it is included in the consequent membership function realization unit 14.

<Step S5> A group correspondence table for each phase in which the learning process is performed (a table that specifies the group of neurons to be learned, that is, a hierarchy) and a connection correspondence table for each group (a table that indicates the group of connections to be input to the group of units) )
Configure settings.

<Step S6> The learning scheduler is activated and learning processing is performed using the backpropagation method.

Hereinafter, steps 31 to S6 shown in the operation flow of FIG.
will be explained in detail separately.

(Configuration of rule section fully connected neuron) The rule section fully connected neuron that is the object of the present invention has a hierarchical network configuration for realizing the fuzzy control rule shown in FIG. 5, for example.

In Fig. 5, the nodes shown in each part constitute one basic unit, and the basic unit consists of two units: a sigmoid function unit shown by a black circle, and a linear function unit shown by a white circle.
seeds are used.

FIG. 4 shows the configuration of the basic unit in the rule part fully connected neuron of FIG. 5.

In FIG. 4, the basic unit 20 constitutes a multi-input one-output system, and includes a multiplication section 22 that multiplies a plurality of inputs by the weight value of each internal connection, and an addition section 22 that adds the multiplication results of the multiplication section 22. and a threshold processing section 26 that performs threshold processing on the output of the addition section 24.

Here, if the first layer is the h layer and the second layer is the i layer, then the multiplier 2
2 and the addition unit 24 executes the calculation of equation (1) below, and the threshold processing unit 26 executes the calculation of equation (2) below.

x9. =ΣyW i h (
1) However, h: i: p: θ,: W, h: x, i: h layer unit number i layer unit number Input signal pattern number i layer 1st unit threshold internal between h-i layers Connection weight value Sum of products of inputs from each unit of the layer to the 1st unit of the i layer Vphj Output from the hth unit of the h layer for the input signal of the Pth pattern)'piThe i layer for the input signal of the CPth pattern i of
Output from the first unit Here, the above equation (2) is known as a sigmoid function, and the threshold processing section 26 of the basic unit indicated by a black circle in the rule part fully connected neuron in Fig. 5 calculates the above equation (2). Performs threshold processing according to the sigmoid function.

On the other hand, the linear function unit with white circles in FIG. Excluding the input section which becomes the following. As the hardware constituting the basic unit shown in FIG. 4, "Network configuration data processing device" disclosed in Japanese Patent Application No. 1983-216865 can be used.

Referring again to FIG. 5, the configuration of each part of the rule part fully connected neuron will be explained.

First, the antecedent membership function realization unit 10 receives the input variables XI and Xz, and outputs the grade value of the membership function of the input variables X, to x7.

In the figure, each membership function is a one-variable function for simplicity, but it may be a multi-variable function. In other words, variable X1
It may be a membership function of X, &X2 (SA), where and X2 are inputs.

The rule part is a hierarchical neural network. Input variable X1. The grade value of the membership function of Xz is input, and the grade value of the membership function of output variable Y is output.

Further, the consequent membership function realization unit 14 (including the centroid determination unit) generates a membership function y (SA) of the output variable.
Input the grade value of ~y (LA) and output the value of the output variable Y.

In Figure 5, each unit is explained as one neuron, but if the function of the unit is specified for parts other than the rule section, it can be used as a gate circuit, arithmetic unit, etc. that realizes the unit function without being considered a neuron. good.

In FIG. 5, input X1. Taking as an example a case where two inputs X2 are given to obtain a control output Y, the configuration of each part will be explained in detail with reference to the same figure as follows.

The input section 16 is composed of separation units 2B-1 and 28-2, and divides the inputs X+ and Xz into two and supplies them to the antecedent membership function realization section 10. The antecedent membership function realization unit 10 includes four sigmoid function units 30-1 to 30.
-4 and the same four linear function units) 32-1 which input the outputs of each sigmoid function unit 30-1 to 30-4.
~32-3.

The rule part 12 is a sigmoid function Uni-node 34-1 to 3
4-5, each sigmoid function unit 34-1~
34-5 Antecedent Membership Function Realization Unit 10
From there, connections are made to all Uni nodes. The outputs of the sigmoid function units 34-1 to 34-5 are linear function units 36-1 to 36 that perform consequent part operations.
-2, all the units are connected.

The consequent membership function realization unit 14 is composed of nine linear function units 38-1 to 38-9, and generates membership functions based on two types of output values (grade values) output from the rule unit 12. is reduced (or enlarged) and the function sum of the two reduced membership functions is calculated.

Furthermore, the center of gravity calculation realization unit 18 is configured to output device 40 for determining the center of gravity.
The weighting of the output from the consequent membership function realization unit 14 is performed by two linear function units (40-) provided in the center-of-gravity determining element output device 40.
1.40-2 to calculate the two centroid calculation elements Y,, Y! Finally, the center of gravity calculation device 42 calculates the center of gravity from the two center of gravity calculation elements Y+ and Yz to generate the control output Y.

FIG. 6 shows neuron groups and connection groups for realizing the learning method according to the present invention for the rule section fully connected neuron of FIG. 5.

In Figure 6, the neuron group is Ga-Gg 7
However, among these, the input neuron group Ga
and the neuron group Gg of the center of gravity calculation implementation unit 18 are excluded from the learning target because they merely perform separation and addition/synthesis, and the present invention learns only the rule part. There are two neuron groups, Gd and Ge, which are the learning targets of the present invention.

On the other hand, the connection group connecting each neuron group is Ca
b=Cfg, and learning is performed to correct the weight value of the input connection located before the neuron group to be learned by the back propagation method.
Connection groups Ccd and Cde are the targets of the learning process according to the present invention.

This fifth. Regarding the learning process according to the operation flow shown in FIG. 2 for the fully connected rule section neuron having the configuration shown in FIG. 6, the initial setting of weight values for each section will be explained first as follows.

(Initial setting of the antecedent membership function realization unit) The initial setting of the weight value for the antecedent membership function realization unit 10 is for each of the connection groups Cab and Cbc of the antecedent membership function realization unit shown in FIG. The weight of the connection,
Initialize to implement the given membership function.

FIG. 7 shows the control input Xl side of the antecedent membership function realizing section 10 of FIG. 5.

In FIG. 7, the sigmoid function unit 30-1.3
The output value y output from 0-2 is y=1/(1+e-'
""-") (3)

In the initial setting of the weight value for the antecedent membership function realization unit 10 of the present invention, based on knowledge that is available in advance, for example, the weight value W is set to the input connection of the sigmoid function unit 30-1, so that W=− 20 is set, and θ--IO is set as the threshold θ in the sigmoid function unit 30-1.

Further, the input connection of the other sigmoid function unit 30-2 is set to W=20 as the weight value W, and at the same time, the threshold value θ is set to θ=10 in the sigmoid function unit 30-2. In addition, linear function unit) 32-1゜32-2
The weight value of the input connection is set to an initial weight value of 1.

In this way, in the sigmoid function unit 30-1, W<Q
When the weight value and threshold value are initially set to θ〈0, the eighth
Calculation of the truth value of a membership function with a function shape such that the unit output value X, (SS) for the control human power X shown in the characteristic curve 44 in the figure, that is, the unit output value X, (SS) decreases as the control human power X increases. Functions can be set.

Also, in the sigmoid function unit 30-2, W>0
By initially setting the weight value and threshold value such that θ〉0, the unit output value X, (LA) for the control human power X shown in the characteristic curve 46 in FIG. It is possible to realize a function that calculates the truth value of a membership function with a function shape such that LA) is thick.

Sigmoid function unit 30-1 3 shown in FIG.
The initial setting of the weight value and threshold for 0-2 is such that the desired antecedent membership function is also realized for the remaining sigmoid function units 30-2 and 30-4 in the antecedent membership function realizing unit 10. Initial settings are made based on prior knowledge.

(Initial setting of rule section) Initial setting of the weight value for the rule section 12 shown in FIG. 3 is to set the weight value of each connection belonging to the connection groups Ccd and Cde included in the rule section shown in FIG. Initial settings are made using randomly generated numbers (for example, in the range of +1 to -1).

Sigmoid function unit 34-1 in the rule section 12
The function of the fuzzy logic operation based on the weight value initially set to the input connection of ~34-5 and the threshold value set to the unit itself will be explained as follows.

FIG. 9 shows one sigmoid function unit 34 provided in the rule section 12, in which the grade
, (SS) ~
~y (LA).

Above X+ (SS)=X+, X+ (LA)=X
2.

X2 (SS) -X3. Xz (LA) =X<
Then, the output value y output from the sigmoid function unit 34 in FIG. 9 is as follows: 5=WX1・X1+WX□・X2+Wxff・X3+W
X4・X3−θ (4)y=1/ (
1+e-”) (5) The unit output value y given by this equation (5) is
By arbitrarily setting the weight values WXI to Wx4 and the threshold value θ of the input connections, an appropriate function shape can be realized.

Note that the linear function unit 3 provided at the final stage of the rule section
The weight value of the connection for 6-1.36-2 is initially set to 1, and the threshold processing is a linear function,
Threshold value θ=0 is set.

(Initial setting of consequent part membership function realization part) Initial setting of the weight value for the consequent part membership function realization part 14 in FIG. Initial settings are made to realize the given membership function based on the knowledge of the given membership function.

FIG. 10 is an explanatory diagram specifically showing the initial setting of weight values for the consequent membership function realization unit 14.

In FIG. 10, the consequent membership function realization unit 14
Nine linear function units 38-1 to 38- provided in
9, a connection input is performed from the linear function units 36-1 to 36-2 of the rule section 12 at the previous stage as shown in the figure. These nine units 3B-1 to 38-9 are
They correspond to points in the range of 0 to 1 on the straight line of the control output Y at equal intervals.

The initial setting of the weight values for the input connections to the linear function units 38-1 to 38-9 is as follows: unit 3 according to the characteristic curve 46 of the graph where is the control output Y and the vertical axis is the unit output value.
Weight values from the B-1 side to the unit 38-9 side: 1, 1. 1. 0.75. 0.5. 0.25.
Initialize O, 0, 0.

Conversely, the rule section 12 that produces the unit output y (LA)
For the connection from the linear function unit 36-2, the weight values from the unit 381 to the unit 38-9 according to the characteristic curve 48 are 0, O, 0, 0, 25, 0, 5, 0, 75, 1, 1
, 1 are initialized.

Furthermore, the linear function units 38-1 to 38-9 do not perform nonlinear threshold processing, and the threshold value θ is set to θ-〇.

Due to the initial setting of the weight values in the consequent part membership function realization part 14, the linear function unit of the rule part
Unit 8 force value y output from 36-1.36-2
(SS) and y (LA) are reduced by multiplication by initial weight values, added, and output to the next centroid calculation implementation unit 18.

In FIG. 10, a weight value of 1 or less is initially set so as to reduce the unit output, but it is also possible to set a weight value of 1 or more to enlarge it.

(Initial Setting of Center of Gravity Calculation Implementation Unit) The weight values for the center of gravity calculation implementation unit 18 shown in FIG. 5 are initialized so as to realize center of gravity calculation for the connection group Cfg shown in FIG. 6.

FIG. 11 shows a specific example of initial setting of weight values for realizing the center of gravity calculation performed on the center of gravity determining element output device 40 of the center of gravity calculation implementation unit 18.

For the two linear function units 40-1 and 40-2 provided in the centroid determining element output device 40, the nine linear function units 38- provided in the consequent membership interval realization unit 14 of the preceding stage are Each of 1 to 38-9 is connected. Then, as shown in the figure, weight values for calculating two gravity center derived values Y, , Y, used in the gravity center calculation are set for this input connection.

In fuzzy control, a method is often adopted in which the control output Y, which is a fuzzy inference value, is calculated by finding the center of gravity of the figure of the sum of functions of reduced membership functions for the same control output according to the following formula: .

Y = S grade(y)ydy / S grade
de(y)dy (6) In order to realize this formula (6), the linear function unit 3B provided in the consequent membership function realization unit 14 for the linear function unit 7 40-1
As weight values for the connections from -1 to 389, weight values that increase from 0 at equal intervals starting from the minimum value Y=O of the control output Y and reaching the maximum value Y=1 are assigned.

On the other hand, for the linear function unit 40-2, the weight decreases from 0 to -1, for example, at the same equal intervals as above, from the maximum value Y = 1 of the control output Y to the minimum value (arY = 0). Assign a value.

By weighting the input connections to the linear function unit 40-1, 40-2, if the unit output values of the preceding stage linear function units 3B-1 to 38-9 are 01 to C3, the linear function unit 40-1 becomes , Yl = I C, +0.875 Cm +0.75C
300.625 C, +0.5 Cs +0.375
C6 0.25C, +0.1.25C1l 0C9 (7
), and the linear function unit 40-2 outputs YZ
=OC, -0,125Cz -0,25C.

0.375 C= 0.5 Cs O,625C6
Output -0,75C, -0,875cs +IC, (8).

The gravity center derived values Yl and Y2 output from the i-type quantity unit) 40-1.40-2 are given to the gravity center calculation device 42, and Y=Yz / ('yz - Yl)
Using (9), the center of gravity, that is, the control output Y, which is a fuzzy inference value, is calculated.

By substituting the above equations (7) and (8) into the above equation (9), it can be seen that Y in the equation (9) matches Y in the equation (6).

With the above processing, the initial setting of the weight values for the rule part fully connected neurons shown in 31 to S4 in FIG. 2 is completed, and the process proceeds to step S5.

(Setting of Learning Schedule) In step S5 of FIG. 2, following the completion of the initial settings, a learning plan, ie, a learning schedule, for causing the rule section fully connected neuron to perform a learning process is set. This learning schedule is set by two steps: setting the phase/group correspondence table shown in FIG. 12(a) and setting the group/connection correspondence table shown in FIG. 12(a).

First, the phase-group correspondence table specifies the neuron groups to be learned in each phase as the learning phase progresses. In the present invention, since only the rule section 12 is trained after setting the initial weight values, as shown in FIG.
As shown in a), two neuron groups Gd and Ge shown in FIG. 6 are set as learning target groups in phase 1.

On the other hand, the group/connection correspondence table sets the correspondence between neuron groups Ga to cg and input connection groups in the rule part fully connected neuron, and as shown in FIG. 12(b), for neuron groups cb to Gg. The input connection groups Cat+-Cgf are associated with each other (see FIG. 6).

This kind of phase group correspondence table and group
When the setting of the connection correspondence table is completed, proceed to S6 in Fig. 2, start the learning scheduler, actually give the learning data to the rule part fully connected neuron, and change the weight value of the input connection using the back propagation method. Have students learn how to do things.

(Processing and configuration of learning scheduler) FIG. 13 shows a learning processing flow diagram of the rule part fully connected neuron of the present invention. This operation flow diagram can be realized, for example, by the device configuration shown in FIG.

First, to explain the device configuration for learning processing, the 14th
In the figure, a learning instruction unit 50 and a learning unit 70 are provided for the rule part fully connected neuron 100.

The learning instruction unit 50 has a learning scheduler 52,
For the learning scheduler 52, there is a learning signal storage section 54 that stores learning signals consisting of pairs of control inputs and desired control outputs for the control inputs, and a phase group storage section 54 that stores the phase group correspondence table shown in FIG. A correspondence table 56 and a group/connection correspondence table 58 storing a group/connection correspondence table are connected. Further, a learning convergence determination section 60 is provided for the learning scheduler 52.

On the other hand, the learning unit 70 is provided with a learning instruction reading section 72, a learning flag setting section 74, and a weight changing section 76 that changes weight values using the pack propagation method.

Furthermore, a feature of the device configuration in FIG. 14 is that a learning regulator 80 is provided in the connection connecting each unit in the rule section fully connected neuron 100.

The learning adjuster 80 has the configuration shown in FIG.

Note that the weight change unit 76 is shown together with the learning adjuster 80.

The learning adjuster 80 is provided with a flag storage section 82, a weight change information reading section 84, and a weight change amount adjustment section 86.

On the other hand, the weight change unit 76 includes a weight calculation unit 88 that performs weight calculation using a backpropagation method, and a weight storage unit 9.
0 and a weight change amount storage section 92 are provided.

In this way, by providing the learning regulator 80 for each connection of the rule section fully connected neuron, the learning scheduler 52 of the learning instruction unit 50 can control the rule section fully connected neuron 100.
A learning flag can be set to determine whether or not to change the weight value of a connection, and the weight can be changed, that is, learning can only be performed for connections for which the learning flag is valid, that is, the flag is on. becomes possible.

Furthermore, the flag set for the learning coordinator 80 sets the antecedent membership that can execute fuzzy rules for a general hierarchical network that does not have a network structure that can implement fuzzy rules as shown in FIG. It is also used when specifying a hierarchical network consisting of the function realization section 10, the rule section 12, and the consequent membership function realization section 14.

Adjustment of weight values when actually learning by the hack propagation method with a hierarchical network conforming to fuzzy rules configured is performed as follows.

In FIG. 15, the weight change information reading unit 84 of the learning adjuster 800 monitors whether the weight change amount is written in the weight change amount storage unit 92 of the weight change unit 76 by the weight calculation unit 88. When the weight change information reading unit 84 detects that the weight has changed, it notifies the weight change amount adjustment unit 86 of the fact. The weight change amount adjustment unit 86 checks the learning flag in the flag storage unit 82, and does nothing if the learning flag is on. On the other hand, if the learning flag is off, change the weights! The adjustment unit 86 is a weight change amount storage unit 92 of the weight change unit 76.
Set to zero to disable weight changes. The hardware constituting this learning scheduler is patent application No. 63-227.
No. 825, [Learning method for network configuration data processing device].

Next, with reference to the learning processing flow shown in FIG. 13, the learning processing of the rule section Briwire Neuro according to the device configuration shown in FIG. 14 will be described.

First, when the learning scheduler 52 of the learning instruction unit 1-50 is activated, it first sets the phase counter i, which indicates the progress of the learning phase, to i=1 in sl, proceeds to S2, and stores the data in the phase/group correspondence table 56. The neuron group number corresponding to the learning phase i=l is read out with reference to the correspondence table. In the present invention, the neuron groups Gd and Ge of the rule section 12 of the rule section fully connected neuron 100 are set in phase 1.

Next, the process advances to S3, and the connection number corresponding to the neuron group number is read out with reference to the group/connection correspondence table 58.

Next, the process proceeds to S4, where the connection group number is output and the learning flag of the learning adjuster 80 provided to the connection belonging to the connection group number is turned on.

Specifically, the learning scheduler 52 of the learning instruction unit 50 outputs a connection group number to the learning instruction reading section 72 of the learning unit 1-70, and the learning instruction reading section 72 reads the connection group number and sets a learning flag. Section 74. The learning flag setting unit 74 sets the learning adjuster 8 provided in the connection belonging to the connection group number that received the learning instruction.
The learning flag is set by instructing the flag storage unit 82 of o to turn on the flag.

Next, the process proceeds to S5, where the learning scheduler 52 issues a learning execution command to start the learning process. This learning process reads the control human power X prepared in the learning signal storage section 54 of the learning instruction unit 5o and the teacher signal d that is a desirable control output for the control human power 100, and the teacher signal d is given to the weight change section 76 and the learning convergence determination section 60. Then, by learning human power X, the rule part fully connected neuron 100
The control output Y outputted from the control output Y is taken into the weight change unit 76 and the learning convergence determination unit 60, the weight change unit 76 changes the weight value according to the back propagation method, and the learning convergence determination unit 60 converts the teacher signal d. When the error in the control output Y becomes equal to or less than a specified value, it is determined that learning has ended, and the learning process is ended.

When the learning process ends based on the learning execution command in S5,
Increment the phase counter I by one in S6, and
Check the end of the learning phase at 7. Since the learning process of the present invention ends at phase 1, the series of processes ends as is.

(Principle of learning by backpropagation method) Next, the principle of learning by the backpropagation method of the present invention will be explained together with the hierarchical network.

FIG. 16 shows a general hierarchical neural network, which is composed of a type of node called a unit and internal connections with weights.

In Figure 16, the h layer is the input device, and the i layer is the intermediate layer.
layer, as the output layer. Here, one layer as an intermediate layer is one layer, but it is the same even if there are a plurality of intermediate layers. As shown in FIG. 4, each unit includes a multiplier 22 that multiplies a plurality of inputs by the weight of each internal connection, and an adder 2 that adds all the multiplication results of the multiplier 22.
4, and a threshold processing section 26 that performs threshold processing on the addition result and outputs one output. The calculations performed in this unit are shown in (1) above regarding the basic unit 20 (
2) Equation becomes. In the backpropagation method, the weight value W i h and the threshold value θi of the hierarchical network are adaptively and automatically adjusted and learned using error feedback. As is clear from equations (1) and (2) above, it is necessary to adjust the weight value Wih and the threshold value θi at the same time, but this work is a difficult work that interferes with each other. Therefore, in addition to the normal unit, the applicant of this application has
By providing a unit whose output value is always 1 in the layer and assigning a threshold value θi as a weight value to the output of this unit with an output value of 1, the threshold value θi is incorporated into the weight value Wih and the threshold value θi is used as the weight value. We have proposed a backpropagation method that can handle
No. 84).

By doing this, the above equations (1) and (2) are converted to )[p,
=Σ)'phW=h (11)y
pi=1/(1+exp(-xp=)) (12
), and the threshold value θi does not appear.

By expanding from equations (11) and (12) above, we get the following (
13) Equation (14) is obtained.

x, J=Σ3'piWJi (1
3))'p=-1/ (1+exp(XpJ))
(14) However, j: unit number of layer j, WJi: weight value X of internal connection between layers i-j, sum of products Ypj'p-th pattern of inputs from each unit of layer jei to the first unit of layer j 1 of layer j for the input signal of
The weight changing unit 76 in FIG. 14 outputs the input cover turn X for learning.
When receiving the output cover turn ypJ output from the output layer j when presented with the teacher pattern dpj, which is the signal that the output cover turn ypJ should take, calculate the difference value (dpJ)'pj) between the two, and then δpJ=VpJ(1-ypJ) (dpJ-)'p=
) Calculate (15). Subsequently, ε: Learning constant ζ: Momentum t: Update amount of weight values between i layer and j layer according to the number of learning times △W j 1(
1) Calculate. Here, the reason why the update amount of the weight value determined in the previous update cycle is added in the second term on the right side of the equation (16) is to speed up the learning.

Subsequently, the weight value changing unit 76 changes δ9. calculated using the above equation (15). Using δpi=3'p, (1yet) ΣδpJWJt(t
-1) Calculate (17), and then calculate the update amount ΔW, h(t) of the weight values of the h layer and the i layer.

Next, the weight change unit 76 uses the above equation (16) and (18).
The weight value for the next update cycle is determined according to the update amount calculated using the formula W==(t) =Wji(t-1)+△Wji(t)
(19) Wzh(t) = Wzh(t l
) to 10 as W=h(t).

The above learning process is repeated so that a weight value is obtained in which the output cover turn y, J for the input cover turn for learning matches the teacher pattern d□.

On the other hand, if the units constituting the hierarchical network in FIG. 16 are linear units, the above formula (14) is y, J= x p;
(20). The above equations (15) and (17) in this linear function unit are δpj = d9j-y pj
(21) δ2. -Σδp, w, (t-1)
(22).

(Weight Learning Process of the Present Invention) FIG. 17 is a process flow diagram of weight learning performed by the hack propagation method on the rule section fully connected neuron of the present invention shown in FIG.

The parameters of each part in this learning process flow are the 18th
It is defined as shown in the figure.

Figure 18 schematically shows the hierarchical structure of the rule section fully connected neuron in Figure 5, with layers 1 to 6 as shown.
It has a six-layer structure, with the first and third layers made up of sigmoid function units, and the rest made up of linear function units. Furthermore, since the present invention learns only the rule part, the learning targets are the third and fourth layers. Furthermore, the sixth layer in the final stage centroid calculation implementation unit 18 is excluded from weight learning.

Here, the fuzzy control value output by realizing the fuzzy rule by the fully connected neuron of the rule part, that is, the control output value, is y, and the learning process is performed from the 6th layer to the 5th layer, the 4th layer, the 3rd layer, The process is performed sequentially such as the second layer and the first layer. The progress of learning for each layer is indicated by an i counter. That is, the i counter is set in the i layer 6 in the initial state.

To further simplify the explanation, W i + i −1 and ΔW□1. -I indicates a matrix, and the size is (number of neurons in the i-th layer) x (number of neurons in the i-1th layer). Furthermore, δi and yi represent vectors whose size is the same as the number of neurons in the i-th layer.

The weight learning process flow shown in FIG. 17 will be explained based on the parameters of each part shown in FIG. 18 as follows.

First, step 31 (hereinafter "step" is omitted)
(described as ), initialize the counter i that sets the learning target layer. Specifically, since there is a six-layer configuration as shown in FIG. 18, the counter i is initially set in the i-layer 6. With this initial setting, the sixth layer is designated as the learning target.

Next, the process proceeds to step 32, where it is determined whether the unit is a sigmoid function unit. In this case, since it is a linear function unit, 314
The process proceeds to step 315, where it is determined whether the layer is the final layer, and since the sixth layer is the final layer, the process proceeds to step 315, where the difference value δ6 is obtained using the teacher signal d and control output y6 obtained at that time.

Next, the process proceeds to step 36, where it is determined whether the unit is to learn weights. At this time, the 6th layer is excluded from the learning target (
Specifically, since the learning flag of the learning adjuster 80 provided in the connection is off), the process advances to S8 and the weight value update amount ΔW6. is set as ΔW65=O, and the process proceeds to S9. In this case, since ΔWbs=O, the weight value W6. will not be updated.

Next, the process proceeds to S12, where it is determined whether the counter i = 1, that is, whether learning of all layers has been completed, and at this time, i
-6, the process proceeds to step 513, where the counter i is decremented by one to set i=5, and the process returns to S2 to proceed to learning the next fifth layer.

Since the fifth layer is also a linear function unit, 82
Proceeds to 314, but since it is not the final stage layer, it goes to 516
Proceed to step 6 and calculate the difference value δ6 obtained in the learning process of the previous 6 layers.
and weight value W6. The difference value δ5 is calculated from the equation (22) and the same equation as above.

Next, proceed to 86 to determine whether it is a learning target,
Since the fifth layer is not the learning target, the weights are not updated (
38→S9→512→513).

Similarly, the process is repeated for all units in the 4th layer, and when the process for the 4th layer is completed, the process proceeds to the 3rd layer.

Since the 4th layer is also a linear function unit, S16→S6→
The weight is updated by S7→Sσ→S12→513.

Further, when processing the third layer, since a sigmoid function unit is used in the third layer, the process proceeds from 32 to S3, and since it is not the final layer unit, the difference value δ3 unique to the sigmoid function unit is calculated in S5. It is determined using the same arithmetic expression as the above-mentioned equation (17). Since this third layer is also a learning target, the process proceeds from 36 to 37 and the difference value δ calculated in S5.

The weight update amount ΔW3□ is calculated using , and updated in S9.

Repeat the same process up to the final unit of these three layers, and
It proceeds to the first layer and performs learning processing on linear function units, and then proceeds to the first layer to perform learning processing on sigmoid function units (in the present invention, learning is performed on units in the third and fourth layers). (Since the learning flag is off for the other layers, no learning is performed.) Then, when the processing of the final unit of the first layer is completed and the process proceeds from SIO to 312, since the counter i has become i-1 at this time, it is determined that learning of all layers has been completed, and the process continues. Terminates the process.

(Initial Setting of Weights Using Random Numbers) Next, another embodiment of the learning method of the present invention will be described with reference to the operation flow diagram of FIG. 19.

In the learning process shown in the operation flow diagram of FIG.
L, S2. Random numbers were used as the initial setting of weight values (including thresholds) for the antecedent membership function realization unit 10, rule unit 12, and consequent membership function realization unit 14 of the rule unit fully connected neuro in S3. It is characterized by Other 34. S5. The processing at S6 is the same as the first embodiment shown in FIG.

Furthermore, if there is some knowledge that is previously held, settings may be made based on this knowledge.

be.

In addition, in the above embodiment, y=1/ (1+e-"') (
23), the unit outputs of the two sigmoid function units 94-1 and 94-2 are subtracted by the unit 9
6 to realize the sigmoid function shown in FIG. 6(b). In this case, the unit output y is y=1/(1-e-", +1) l/(1,-W
ZX-#2).

(Others) The above-mentioned specific example is as shown in FIG.
In this example, a fully connected neuron in the rule section that realizes a fuzzy rule targeting two control human forces XI and X2 is taken as an example, but it goes without saying that the number of control inputs may be 1 or 3 or more as appropriate. [Effects of the Invention] As explained above, according to the present invention, initial settings or random numbers based on knowledge held in advance in each of the antecedent membership function realization unit, the rule unit, and the consequent membership function realization unit are By learning only the rule part of the fully connected neuron after initial setting using or a combination thereof, efficient learning is possible, and it is possible to realize fuzzy rules that can be used in any fuzzy control adaptive system. A more efficient connection weight analysis can be realized.

[Brief explanation of drawings]

Fig. 1 is an explanatory diagram of the principle of the present invention, Fig. 2 is a flowchart of the learning operation of the present invention, Fig. 3 is an explanatory diagram of the definition of the rule section fully connected neuron, Fig. 4 is an explanatory diagram of the basic unit, and Fig. 5 is an explanatory diagram of the definition of the rule section fully connected neuron. A diagram showing a specific configuration of an embodiment of the rule section fully connected neuron of the present invention, FIG. 6 is an explanatory diagram of the neuron group and connection group of FIG. 5, and FIG. 7 is an implementation of the antecedent section membership function of the present invention. FIG. 8 is an explanatory diagram of the membership function of the antecedent part, FIG. 9 is an explanatory diagram of the rule part unit of the present invention, and FIG. 10 is an explanatory diagram of the membership function realization part of the consequent part of the present invention. FIG. 11 is an explanatory diagram of the initial settings of the center of gravity calculation implementation unit of the present invention. FIG. 12 is an explanatory diagram of the settings of the phase/group correspondence table and group/connection correspondence table of the present invention. FIG. 14 is a diagram of a learning process flowchart according to the present invention. FIG. 15 is a diagram of a learning adjuster provided for each connection in FIG. 14. FIG. 16 is a diagram of a general hierarchical neural system. 17 is a flow diagram of the weight learning process of the present invention, FIG. 18 is an explanatory diagram of parameters of each part of FIG. 17, and FIG. 19 is an operation flow diagram showing another embodiment of the present invention. FIG. 20 is an explanatory diagram of another membership function used in the present invention. 10...Antecedent part membership function realization part, 12
...Rule part, 14...Consequent part Membership function realization part, 16...Input part, 18...
... Center of gravity calculation realization section, 20 ... Basic unit, 22 ... Multiplication section, 24 ... Addition section, 26 ... Threshold processing section, 28-1°28-2.
31-1 to 32-4.38-1 to 38-9.40-1.
40-2...Linear function unit, 30-1 to 3
0-2.34-1 to 34-5.941 94-2...
... Sigmoid function unit, 40 ... Center of gravity determining element output device, 42 ... Center of gravity calculation device, 5
0...Learning instruction unit, 52...Learning scheduler, 54...Learning signal storage unit, 5
6... Phase group correspondence table, 58
...Group phase correspondence table, 60.
...Learning convergence determination unit, 70...Learning unit, 72...Learning instruction reading unit, 74...
. . . Learning flag setting section, 76 . . . Weight changing section,
80... Learning adjuster, 82... Flag storage unit, 84... Weight change information reading unit, 86...
... Weight change amount adjustment section, 88 ... Weight calculation section, 90 ... Weight storage section, 92 ...
Weight change amount storage unit, 96...Subtraction unit, 1
00...Rule part fully connected neuro. Agent Patent Attorney Sakimi Honma's learning masterpiece flow episode 2 The antecedent part of the month from Figure E, Merepashi, Tsubu G number, back bank part, the sword period setting R'' Just before the exit of the month figure unit Part membership function explanatory diagram (Figure 1) Rule rhyme unit explanation (Figure 1) Calculate the t of the unit (Figure 1) ) Phase/group interpersonal table and group/connection correspondence table of the present invention. Flow chart/q Figure 94-/ 〈 θ2 (a) (b) Quick explanation of other membership functions to use

Claims (5)

[Claims] 1. An antecedent membership function realization unit (10) configured by providing a plurality of basic units that calculate and output membership function values of input signals, and calculation result values output from the preceding processing layer and the calculations. Equipped with one or more processing layers consisting of at least one basic unit that calculates and outputs a calculation result value according to the assigned operation, using as input the multiplication value of the weight value of the internal connection associated with the result value. , and between the basic units of the adjacent processing layers, the basic unit of the first processing layer, and the antecedent membership function realization unit (10).
A rule section (12) configured by adopting a configuration in which the basic units are mutually connected internally, and a calculation result value output by the corresponding basic unit of the last processing layer of the rule section (12). a hierarchy consisting of a consequent membership function realization unit (14) constituted by at least one basic unit that calculates and outputs an output signal value derived according to the membership function value of the output signal with input as input; It has a network and input control state quantity (X_1~
In the rule part fully connected neuron that outputs the control operation amount (Y_1 to Y_m) corresponding to Part (14
), and a second step of learning the weights of the rule section using training data prepared in advance. How to learn connective neurons. 2. The initial values of the weights and thresholds of the antecedent membership function realization unit and the consequent membership function realization unit of the first process are set using initial values of weights and thresholds created based on knowledge held in advance. 2. The learning method of a fully connected rule part neuron according to claim 1, wherein the learning method is performed by setting the rule part, and the weight and threshold value of the rule part are set using random numbers. 3. 2. The method for learning a fully connected rule part neuron according to claim 1, wherein initial values of the weights and thresholds in the first step are set using random numbers. 4. In setting the initial values of the weights and thresholds in the first process, the weights and thresholds that can be set using prior knowledge are initialized using this knowledge, and the initial values of the other weights and thresholds are set using random numbers. 2. The method of learning rule part fully connected neurons according to claim 1. 5. At the start of learning in the second step, a learning flag is set for each connection of the hierarchical network in the rule section (12) to determine whether or not learning is to be performed, and a weight value is determined by a learning process based on the learning flag set for the connection. 2. The method of learning a fully connected rule part neuron according to claim 1, wherein the learning method comprises optimizing the rule part fully connected neuron.