JPH0464133A - Learning method for neural unit with fully coupled rule part - Google Patents

Abstract

PURPOSE:To improve the efficiency of learning by initializing a weight value and a threshold value at each of an antecedent part membership function realizing part, rule part, and consequent part membership function realizing part, and then learning the weight of the antecedent part membership function realizing part after learning the weight of the rule part. CONSTITUTION:The weight value and the threshold value based on knowledge stored preliminarily or random numbers are initialized at each of an antecedent part membership function realizing part 10, rule part 12, and consequent part membership function realizing par 14. Then, the weight of the antecedent membership function realizing part 10 is learned after learning the weight of the rule part 12 using a learning data prepared preliminarily. Thus, the learning to operate a control suited to an object system can be realized efficiently.

Description

[Detailed Description of the Invention] [Summary] A control output corresponding to an input is generated according to a hierarchical network structure that realizes a fuzzy control rule consisting of an antecedent membership function realization part, a rule part, and a consequent part membership function realization part. Regarding the learning method of the generated rule part fully connected neuron, the aim is to efficiently perform learning for controlling suitable for the target system. After setting the initial values of weights and thresholds for each section by introducing prior knowledge, using random numbers, or a combination of both, the weights of the rule section were learned using system training data prepared in advance. After that, the weight of the antecedent membership function realization unit is configured to be learned.

[Industrial Field of Application] The present invention relates to a learning method for a fully connected rule part neuron that combines fuzzy control and a neural network.

In information processing using neurons, learning data consisting of a set of input patterns of the target system and desired output patterns for that input pattern is presented to a hierarchical neural network to learn and perform adaptive processing. Let them do it. In this learning process, a processing method called the backpropagation method is particularly attracting attention because of its high practicality.

On the other hand, fuzzy theory was proposed by Zadeh in 1960, and is a theory that models the process of thinking and judgment based on ambiguity that humans make, such as whether the temperature is "low" or "high." This fuzzy theory is mathematically based on fuzzy set theory, which introduces membership functions to represent ambiguity.

As an application of fuzzy theory, in 1974 Maqmd
Ani first used it to control a steam engine.
Various types of fuzzy control are performed.

Since the system that performs this fuzzy control handles uncertain things, it is necessary to have a configuration that can adaptively respond to adjustments and changes in the control algorithm.

[Prior Art] Generally, the following procedure is taken to construct a fuzzy control system.

■Obtain ambiguous control rules such as ``If the temperature is high, reduce the fire'' held by experienced operators.

■ Quantify the meaning of the words (life B) in the control rules in the form of membership functions, and convert the control rules into rIF-THEN~
” Described using a type rule expression.

■Conduct inspections through simulations and on-site tests.

■Evaluate inspection results and improve membership functions and rules.

To explain in more detail, the fuzzy control rules are, for example, rif x, is big and X2 is s
It is described as mall theny+ is big J.

Here, the IF part is called the antecedent part, and is the control state quantity such as temperature (
This is the part that describes the conditions for (control input). Further, the THEN section is called a consequent section, and is a section that describes conditions regarding the control operation amount (control output) of the operating end, etc.

Fuzzy control manages multiple fuzzy control rules prepared as a control theory for the target system, and also manages the meaning of ambiguous linguistic expressions such as "large" or "small" written in each fuzzy control rule. Quantify and manage it as a ship function.

'Ml' of the completed fuzzy control system? Step 11 is as follows.

■When a controlled state quantity such as temperature is input from the controlled object, the truth value is calculated according to the membership function of the antecedent part that has been set.

(2) Next, in accordance with the antecedent computation such as selecting the minimum value, a process is executed to determine the value to be applied to the consequent in each fuzzy control rule.

For example, the truth value of rx+is big J=0,8rxz
If the truth value of is small j = 0.5, then according to the antecedent operation that selects the minimum value, the truth value = 0
．． 5 is output as the applied value to the consequent of the fuzzy control rule.

(2) Subsequently, a process is executed to determine an output application value from the human power value (application value) from the antecedent part for the membership function of the consequent part, which has the same amount of control operation.

For example, with two fuzzy control rules, ryl i
Common value of s big -0,5ry+ is big
When the applied value of J=0.6 is obtained, the applied value=0.6 is selected and determined according to the consequent operation that selects the maximum value.

■Subsequently, by reducing (or expanding) the membership function of the control manipulated variable according to the determined applied value, the same I
Processing such as finding the center of gravity of the figure of the function sum of the membership functions for the IJ m ti production amount is performed, and the control operation amount, which is a fuzzy inference value, is calculated and output.

By the way, fuzzy control rules are planned to be generated according to the knowledge possessed by a skilled operator, and it is difficult to establish fuzzy control rules that can realize optimal control from the beginning. Therefore, it is necessary to evaluate the generated fuzzy control rules through simulations and on-site tests, and to improve them through trial and error.

Therefore, the inventors of the present application developed a hierarchical network structure that realizes fuzzy control rules by applying neurology.
In other words, we have proposed a general-purpose fuzzy control system using pre-wired neuro in the rule section (Patent Application No. 2002-6).
No. 0256 "Hierarchical Network Configuration Data Processing Device and Data Processing System j).

That is, the rule section Briwire Neuro has a hierarchical network configuration that realizes a fuzzy control rule consisting of an antecedent membership function realization section, a rule section, and a consequent membership function.

Considering the automatic extraction of fuzzy control rules by the pre-wired neuron of the rule section, the following procedure may be taken.

■Set the member shift function of the rule section prewire neuro.

■The rule section Briwire Neuro is presented with a human cover turn of the target system and a set of desirable output patterns for that person cover turn, and is made to learn.

■Extract the optimal fuzzy control rule for the target system by analyzing the weights of connections in the rule section.

[Problem B to be solved by the invention] However, the learning performed to extract the fuzzy control rule for the rule part fully connected neuron is based on the antecedent membership function part, the rule part, and the consequent part membership function realization part. The learning method for learning, including the weights of the rules, has not been decided yet, and there is a need to develop an efficient learning method that can maximize the performance of the fully connected rule section neuron. There is.

The present invention was made in view of the above situation, and an object of the present invention is to provide a learning method for a fully connected rule section neuron that can efficiently perform learning for controlling a target system. .

[Means to solve the problem] FIG. 1 is a diagram explaining the principle of the present invention.

The present invention has a hierarchical network structure consisting of an antecedent membership function realization unit 10.1 or a rule unit 12 having multiple layers and a consequent membership function realization unit 14, and the layers inside the rule unit 12 All units are internally coupled between the antecedent membership function realization unit 10 in the previous stage and the consequent membership function realization unit 14 in the subsequent stage, and control state quantities (X+, Xz ,...
The target is a fully connected rule part neuron that outputs l or a plurality of control operation variables (Y) corresponding to X, ).

In the present invention, as a learning method of such a rule part fully connected neuron, the following steps are performed: First step> Each of the antecedent membership function realization part 10, the rule part 12, and the consequent part membership function realization part 14 Set initial values of weights and thresholds.

Second process> After learning the weights of the rule unit 12 using the training data prepared in advance, the antecedent membership function realization unit lO
learn the weights of.

A learning method with a processing process is constructed.

Here, the method for setting the initial values of the weights and thresholds in the first process is based on the knowledge you have in advance. (Indicated as "a priori knowledge" in the figure) The initial value created based on this is set.

Another method for setting the initial values of the weights and thresholds in the first step is to set initial values created using random numbers.

Further, in setting the initial values of the weights and thresholds in the first process, those that can be set using knowledge held in advance are initialized using this knowledge, and other initial values may be set using random numbers.

Furthermore, at the start of learning in the second process, a learning flag is set for each connection in the hierarchical network structure consisting of the antecedent membership function realization unit IO and the rule unit 12, and the learning flag set for the connection is set. The weight values are optimized through learning processing based on the flags.

[Operations] According to the learning method according to the present invention for the rule part fully connected neuron having such a configuration and defined in FIG. 3, the following effects can be obtained.

First, we need to know in advance about the rule part fully connected neuron, which has a hierarchical network configuration consisting of an antecedent membership function realization part, a rule part, and a consequent membership function realization part, which realize the fuzzy control rules of the target system. Initial values of weights and thresholds are set based on random numbers, initial values of weights and thresholds are set based on random numbers, or both are used in combination.

Next, the learning process is started, the input pattern for learning is presented to the rule part fully connected neuron, and the output pattern from the rule part fully connected neuron is a learning ratio pattern (teacher signal ), learning is performed using, for example, a backpropagation method.

Furthermore, learning efficiency can be improved by initializing the weights and thresholds of the antecedent membership function realization section and the consequent membership function realization section into which knowledge held in advance is introduced.

Furthermore, the performance of the network can be maximized by initializing the weights and thresholds of the antecedent membership function implementation section, the rule section, and the consequent membership function actualization section using random numbers.

[Embodiment] FIG. 2 is a learning operation flowchart showing a processing procedure as a first embodiment of a learning method for rule part fully connected neurons according to the present invention. The rule part fully connected neuron that is the object of the learning method according to the procedure shown in FIG.
2, and a consequent membership function implementation unit 14 (including a center of gravity calculation implementation unit 18)).

In the learning process of FIG. 2, the following steps 81 to S6
Performs processing consisting of.

Step Sl> Weight of the antecedent membership function realization unit 10 Ca b +
Initial settings are made by introducing knowledge that is previously held so as to realize a membership function given Cbc.

Step S2> The weights Ccd and Cde of the rule section 12 are initialized using random numbers.

(Step S3) Preliminary knowledge is introduced and initialized so as to realize the membership function given the weight Cef of the consequent membership function realization unit 14.

(Step S4) The weight Cfg of the center of gravity calculation implementation unit 18 is initialized so as to realize the center of gravity calculation. Incidentally, in FIG. 5, it is included in the consequent membership function realization unit 14.

Step S5> A group correspondence table for each phase in which the learning process is performed (a table that specifies the group of neurons to be learned, that is, a hierarchy) and a connection correspondence table for each group (a table that indicates the group of connections to be input to the group of units) )
Configure settings.

<Step S6> The learning scheduler is activated and learning processing is performed using the backpropagation method.

Hereinafter, steps S1 to S6 shown in the operation flow of FIG.
will be explained in detail separately.

(Configuration of rule section fully connected neuron) The rule section fully connected neuron that is the object of the present invention has, for example, a hierarchical network configuration that can realize the fuzzy control rule shown in FIG.

In Fig. 5, the nodes shown in each part constitute one basic unit, and the basic unit consists of two units: a sigmoid function unit shown by a black circle, and a linear function unit shown by a white circle.
seeds are used.

FIG. 4 shows the configuration of the basic unit in the rule part fully connected neuron of FIG. 5.

In FIG. 4, the basic unit 20 constitutes a multi-power single output system, and includes a multiplication section 22 that multiplies a plurality of inputs by the weight value of each internal connection, and an addition section that adds the multiplication results of the multiplication section 22. and a threshold processing section 26 that performs threshold processing on the output of the addition section 24.

Here, if the first layer is the h layer and the second layer is the i layer, then the multiplier 2
2 and the addition unit 24 executes the calculation of equation (1) below, and the threshold processing unit 26 executes the calculation of (2) 2 below.

x, ・−Σ YphVhh
(1) yp*=1/((1+e-(X9'-")
(2) However, h: unit number of the h layer i: unit number of the i layer p: input signal pattern number θ,: threshold value W of the first unit of the i layer, h: weight of internal coupling between h and i layers value xp;'hll
Sum of products yph of the inputs from each unit to the first unit of the i layer: Output ypr from the h-th unit of the h-layer for the input signal of the p-th pattern: it of the i-layer for the input signal of the p-th pattern
Output from the unit Here, the above equation (2) is known as a sigmoid function, and the threshold processing section 26 of the basic unit shown in the black circle in the rule section fully connected neuron in FIG. Perform threshold processing according to.

On the other hand, a linear function unit in which ypi=Xpi is set in the threshold value processing section 26 becomes a linear function unit indicated by a white circle in FIG. However, the input section, which is the first unit of the antecedent membership function implementation section 10, is excluded. As the hardware constituting the basic unit shown in FIG. 4, "Network configuration data processing device" disclosed in Japanese Patent Application No. 63-216865 can be used.

Referring again to FIG. 5, the configuration of each part of the rule part fully connected neuron will be explained.

First, the antecedent membership function realization unit 10 receives each input variable X, , X2 as input, and outputs the grade value of the membership function of the input variables X+ to Xn. In the figure, each membership function is a one-variable function for simplicity, but it may be a multi-variable function.

In other words, with variables X1 and X2 as input, x &X2 (SA) It may also be a membership function.

The rules section is a hierarchical neural software work. Input variable X1. The grade value of the membership function of Xz is input, and the grade value of the membership function of output variable Y is output.

The consequent membership function realization unit 14 (including the centroid determination unit) calculates the membership functions y(S A) to Y of the output variables.
Input the grade value of (LA) and output the value of output variable Y.

In addition, although each unit is explained as one neuron in Figure 5, units whose functions can be specified for parts other than the rule part are not treated as neurons but as uni-neurons.
It may also be a gate circuit, an arithmetic unit, etc. that realizes the 7-bit function.

In FIG. 5, input X1. Taking as an example a case where two inputs Xz are given to obtain a control output Y, the configuration of each part will be explained in detail with reference to the same figure as follows.

The input section 16 is composed of separation units 28-1, 28-2, and human power X1. X2 is divided into two parts and given to the antecedent membership function realization unit 10. The antecedent membership function realization unit 10 includes four sigmoid function units 30-1 to 30.
−4 and the same four linear function units 32 1 to which the outputs of each sigmoid function unit 30 1 to 30-4 were input.
~32-3.

The rule section 12 includes sigmoid function units 341 to 34-
5, and each sigmoid function unit 34-1 to 34-
The antecedent membership function realization unit 10 for 5 is connected to all units. The outputs of the sigmoid function units 34-1 to 34-5 are all connected to linear function units 36-1 to 36-2 which perform consequent calculations.

The consequent membership function realization section 14 is composed of nine linear function units 381 to 38-9, and the rule section 1
The membership function is reduced (or enlarged) based on the two types of output values (grade values) output from 2, and the function sum of the two types of reduced membership functions is calculated.

Furthermore, the center of gravity calculation realization unit 18 is configured to output device 40 for determining the center of gravity.
and a centroid calculation device 42, and the weighting of the output from the consequent membership function realization unit 14 is performed by calculating the sum of the two linear functions provided in the centroid determining element output device 40) 40-
1.40-2 respectively to obtain the two center of gravity calculation elements Y+ and Yz, and finally the center of gravity calculation device 42 calculates the center of gravity from the two center of gravity calculation elements Y+ and Yz to produce the disaster control output Y.
will occur.

Figure 6 shows the rule part fully connected neuron in Figure 5.
4 shows neuron groups and connection groups for implementing the learning method according to the present invention.

In Figure 6, the neuron group is Ga-Gg 7
However, among these, the input neuron group Ga
and the neuron group Gg of the center of gravity calculation implementation unit 18 are excluded from the learning target because they are simply separated and added and combined. Since the realization unit is to be trained, there are four neuron groups to be studied in the present invention: Gb, Gc, Gd, and Ge.

On the other hand, the connection group connecting each neuron group is Ca
b = Cfg, and since learning is performed to modify the weight value of the input connection located in the previous stage of the neuron group to be learned by the puncture propagation method,
The connection group Cab-Cde is the object of the learning process according to the present invention.

Regarding the learning process according to the operation flow of FIG. 2 for the rule section fully connected neuron having the configuration shown in FIG. 5.6, the initial setting of weight values for each section will be explained first as follows.

(Initial setting of the antecedent membership function realization unit) The initial setting of the weight value for the antecedent membership function realization unit 10 is performed in the connection groups Cab and Cbc of the antecedent membership function realization unit shown in FIG. Initialize the weight of each connection so as to realize the given membership function.

FIG. 7 shows the control input X1 side of the antecedent membership function realizing unit IO in FIG. 5.

In FIG. 7, the sigmoid function unit 301.3 (
)-2, the output value y is y=1/(1+e
−””−”' ) (3).

In the initial setting of the weight value for the antecedent membership function realizing unit 10 of the present invention, for example, the weight value W is set to the input connection of the sigmoid function unit 3 (1-1) based on the knowledge that is held in advance. In addition, θ−10 is set as the threshold value θ in the sigmoid function unit 30-1.

Further, the input connection of the other sigmoid function unit 30-2 is set to W=20 as the weight value W, and at the same time, the threshold value θ is set to θ-10 in the sigmoid function unit 302. Note that the weight value of the input connection of the linear function unit 32 1.322 is set to an initial weight value of 1.

In this way, in the sigmoid function unit 30-1, W〈0
．． When the weight value and threshold value are initially set such that θ<0, the unit output value XI (SS) for the control input X1 shown in the characteristic curve 44 in FIG. 8, that is, as the control human power X increases, the unit output value It is possible to set a truth value calculation function for a membership function having a function shape such that (SS) becomes small.

Further, in the sigmoid function unit 30-2, W>0. θ〉
By initially setting the weight value and threshold value to be 0, the unit output value XI (LA) for the control human power X shown in the characteristic curve 46 in FIG. A function is realized to calculate the truth value of a membership function whose function shape is increasing.

The sigmoid function unit 301.3 (
The initial setting of the weight value and threshold value for 12 is based on the knowledge that is held in advance so that the remaining sigmoid function units 302 and 304 in the antecedent membership function realization unit 10 also realize the desired antecedent membership function. Initialize based on.

(Initial setting of the rule section) The initial setting of the weight value for the rule section 12 shown in FIG. 3 has in advance the weight value of each connection belonging to the connection groups Ccd and Cde included in the rule section shown in FIG. Random numbers generated under certain conditions by introducing knowledge (for example, +1
to -1).

Sigmoid function unit 34-1 in the rule section 12
The function of the fuzzy logic operation based on the weight value initially set to the input connection of ~34-8 and the threshold value set to the unit itself will be explained as follows.

FIG. 9 shows one sigmoid function unit 34 provided in the rule section 12, in which the grade value X, (SS) to Xz (LA) is extracted from the antecedent membership function realization section 10 located in the previous stage. is input, and the grade value y (SS) of the output value y is calculated by product-sum calculation and threshold processing.
yields y(LA).

Above X+ (SS) -X+ , X+ (LA) -X2
When Xz(SS)=X:l and Xz(LA)=X4, the output value y output from the sigmoid function unit 34 in FIG. 9 is as follows: S = WXl ・X + +Wxz Hx2+ WX
3 ・X3 +WX4・x3−θ
(4) y=1/ (1+e-3)
It is calculated as (5). The unit output value y given by (5) 2 can realize an appropriate function shape by arbitrarily setting the weight values WXI to WX4 of the input connections and the threshold value θ.

Note that the linear function unit 3 provided at the final stage of the rule section
The weight value of the connection for 6-1.36-2 is initially set to 1, and the threshold processing is a linear function,
A threshold value θ-0 is set.

(Initial setting of consequent part membership function realization part) Initial setting of the weight value for the consequent part membership function realization part 14 in FIG. Initial settings are made to realize the given membership function based on the knowledge of the given membership function.

FIG. 10 is an explanatory diagram specifically showing the initial setting of weight values for the consequent membership function realization unit 14.

In FIG. 10, the consequent membership function realization unit 14
Nine linear function units 381 to 38-9 provided in
For, the rule part 120 linear function unit 3 of the previous stage
Connection input is performed from 6-1 to 3B-2 as shown in the figure. These nine units 38-1 to 38-9 are
They correspond to points in the range of 0 to 1 on the straight line of the control output Y at equal intervals.

The initial setting of the weight value for the input connection for this linear function Unisol l-38-1 to 38-9 is shown on the right for the connection weight value of the output y (SS) from the unit) 36-1 of the rule section 12. According to the characteristic curve 46 of the graph in which the horizontal axis is the control output Y and the vertical axis is the unit output value, weight values are set from the unit 38-1 side to the unit 38-9 side, 1, 1. 1 0.75. 0.5. 0.25.
0. 0. Initialize to 0.

Conversely, for the connection from the linear function uni-node 36-2 of the rule section 12 that produces the unit output y (LA), the weight values 0 0, 0. 0.25. 0.5. 0.75.
1. 1. Initialize 1.

Furthermore, the linear function units 38-1 to 38-9 do not perform nonlinear threshold processing, and the threshold value θ is set to θ-0.

Due to the initial setting of the weight values in the consequent part membership function realization part 14, the linear function unit of the rule part
Unit output value V output from 36-1.36-2
(SS) and y (LA) are reduced by multiplication by initial weight values, then added, and output to the next centroid calculation implementation unit 18.

In FIG. 10, a weight value of 1 or less is initially set so as to reduce the unit output, but it is also possible to set a weight value of 1 or more to enlarge it.

(Initial Setting of Center of Gravity Calculation Implementation Unit) The weight values for the center of gravity calculation implementation unit 18 shown in FIG. 5 are initialized so as to realize center of gravity calculation for the connection group Cfg shown in FIG. 6.

FIG. 11 shows a specific example of initial setting of weight values for realizing the center of gravity calculation performed on the center of gravity determining element output device 40 of the center of gravity calculation implementation unit 18.

For the two linear function units 40-1 and 40-2 provided in the center-of-gravity determining element output device 40, nine linear function units l-38 provided in the consequent membership function realization unit 14 of the preceding stage are provided. -1 to 38-9 are each connected. Then, for this input connection, as shown in the figure, two centroid derived values Y1. A weight value for calculating Yz is set.

In fuzzy control, a method is often adopted in which the control output Y, which is a fuzzy inference value, is calculated by finding the center of gravity of the figure of the sum of functions of reduced membership functions for the same control output according to the following formula: .

In order to realize this equation (6), the linear function unit 40-
As a weight value for the connections from the linear function units 38-1 to 38-9 provided in the consequent membership function realization unit 14 for 1, starting from the minimum value y=o of the control output Y up to the maximum value y=i Assign weight values that increase from 0 at equal intervals.

On the other hand, the linear function Uniso! -40-2, 0 from the maximum value Y=1 of the control output Y to the minimum value Y=0.
For example, weight values are assigned that decrease from -1 to -1 at the same equal intervals as above.

By weighting the input connections to the linear function units 401 and 40-2 in this way, if the unit output values of the linear function units 38-1 to 38-9 in the previous stage are C9 to C1, the linear function units 40- 1 is Y, = I C, +0.875 cz +0.75C
3+ 0.625C4+ 0.5 cs + 0.37
5C6+ 0.25 C7+ 0.125CB+ OC
q (7) and also outputs linear function unit 40-2
is yz =OCI -0,125CZ -0,25
C30,375C, -0,5Cs -0,625C60
, 75C, -0,875C, -I C9(8) is output.

The centroid derived values Y+ and Yz output from the linear function Unisol l-40-1 and 40-2 are given to the centroid calculation device 42, and Y=Yz/(Y2-Y, > (9)
is used to calculate the center of gravity, that is, the control output Y, which is a fuzzy inference value.

By substituting the above equations (7) and (8) into the above equation (9), it can be seen that Y in the equation (9) matches Y in the equation (6).

With the above processing, the initial setting of the weight values for the rule part fully connected neurons shown in 31 to S4 in FIG. 2 is completed, and the process proceeds to step S5.

(Setting of Learning Schedule) In step S5 of FIG. 2, following the completion of the initial settings, a learning plan, ie, a learning schedule, for causing the rule section fully connected neuron to perform a learning process is set. This learning schedule is set by setting the phase/group correspondence table shown in FIG. 12(a) and setting the group/connection correspondence table shown in FIG. 12(b).

First, the phase-group correspondence table specifies the neuron groups to be learned in each phase as the learning phase progresses. In the present invention, after setting the initial weight values, the weights of the rule unit 12 are first learned, and then the weights of the antecedent membership function realizing unit 10 are learned.
As shown in Figure (a), in Phase 1, neuron groups Gd and Ge shown in Figure 6 are used as learning target groups.
Set Gb and Gc in phase 2.

On the other hand, the group/connection correspondence table sets the correspondence between neuron groups Ga to Gg and input connection groups in the rule part fully connected neuron, and as shown in FIG. 12(b), The input connection groups Cab to Cgf are associated with each other (see FIG. 6).

When the setting of the phase/group correspondence table and the group/connection correspondence table is completed, proceed to step 86 in Figure 2 to start the learning scheduler and actually give the learning data to the fully connected neuron in the rule section and perform the pack. Learning is performed to change the weight values of input connections using the propagation method.

(Processing and configuration of learning scheduler) FIG. 13 shows a learning processing flow diagram of the rule part fully connected neuron of the present invention. This operation flow diagram can be realized, for example, by the device configuration shown in FIG.

First, to explain the device configuration for learning processing, the 14th
In the figure, a learning instruction unit 50 and a learning unit 70 are provided for the rule part fully connected neuron 100.

7 Jin instruction unit 50 has a learning scheduler 52,
For the learning scheduler 52, there is a learning signal storage section 54 that stores learning signals consisting of pairs of control inputs and desired control outputs for the control inputs, and a phase group storage section 54 that stores the phase group correspondence table shown in FIG. A correspondence table 56 and a group/connection correspondence table 58 storing a group/connection correspondence table are connected. Further, a learning convergence determination section 60 is provided for the learning scheduler 52.

On the other hand, the learning unit 70 is provided with a learning instruction reading section 72, a learning flag setting section 74, and a weight changing section 76 that changes weight values using a backpropagation method.

Furthermore, a feature of the device configuration in FIG. 14 is that a learning regulator 80 is provided in the connection connecting each unit in the rule section fully connected neuron 100.

The learning adjuster 80 has the configuration shown in FIG.

The weight change unit 76 is shown together with the learning adjuster 80.

The learning adjuster 80 is provided with a flag storage section 82 , a weight change information reading section 84 , and a weight change amount adjustment section 86 .

On the other hand, the weight change unit 76 includes a weight calculation unit 88 that performs weight calculation using a backpropagation method, and a weight storage unit 9.
0 and a weight change amount storage section 92 are provided.

In this way, by providing the learning regulator 80 for each connection of the rule section fully connected neuron, the learning scheduler 52 of the learning instruction unit 50 can control the rule section fully connected neuron 100.
A learning flag can be set to determine whether or not to change the weight value of a connection, and the weight can be changed, that is, learning can only be performed for connections for which the learning flag is enabled, that is, the flag is on. It becomes possible.

Furthermore, the flag sent to the learning coordinator 80 is an antecedent membership that can execute fuzzy rules for a general hierarchical network that does not have a network structure that can implement fuzzy rules as shown in FIG. It is also used when specifying a hierarchical network consisting of the function realization section 10, the rule section 12, and the consequent membership function realization section 14.

Adjustment of weight values when actually learning by the back propagation method with a hierarchical network conforming to fuzzy rules configured is performed as follows.

In FIG. 15, the weight change information reading unit 84 of the learning adjuster 80 monitors whether the weight change amount is written in the weight change amount storage unit 92 of the weight change unit 76 by the weight calculation unit 88. . When the weight change information reading unit 84 detects that the weight has changed, it notifies the weight change amount adjustment unit 86 of the fact. The weight change amount adjustment unit 86 checks the learning flag in the flag storage unit 82, and does nothing if the learning flag is on.

On the other hand, if the learning flag is off, the weight change amount adjustment unit 8
6 sets zero in the weight change amount storage section 92 of the weight change section 76 to invalidate the weight change. The hardware constituting this learning scheduler is shown in Japanese Patent Application No. 63-227825 entitled "Learning Method for Network Configuration Data Processing Apparatus".

Next, with reference to the learning processing flow shown in FIG. 13, the learning processing of the rule section fully connected neuron according to the device configuration shown in FIG. 14 will be described.

First, when the learning scheduler 52 of the learning instruction unit 50 is activated, first, in Sl, the phase counter i indicating the progress of the learning phase is set to i=1, and the process proceeds to S2, where the process is stored in the phase/group correspondence table 56. The neuron group number corresponding to learning phase i=l is read out with reference to the correspondence table. In the present invention, the antecedent membership function realizing section 10 of the rule section fully connected neuron 100 and the neuron group Gb-Ge of the rule section 12 are set in phase 1.

Next, the process advances to S3, and the connection number corresponding to the neuron group number is read out with reference to the group/connection correspondence table 58.

The process then proceeds to S4, where the connection group number is output and the learning flag of the learning adjuster 80 provided to the connection belonging to the connection group number is turned on.

Specifically, the learning scheduler 52 of the learning instruction unit 50 outputs a connection group number to the learning instruction reading section 72 of the learning unit 70, and the learning instruction reading section 72 reads the connection group number and sends it to the learning flag setting section 74. give. The learning flag setting unit 74 selects a learning adjuster 80 provided in the connection belonging to the connection group number that received the learning instruction.
The learning flag is sent by instructing the flag storage unit 82 to turn on the flag.

Next, the process proceeds to S5, where the learning scheduler 52 issues a learning execution command to start the learning process. This learning process reads the control human power X prepared in the learning signal storage section 54 of the learning instruction unit 50 and the teacher signal d that is a desirable control output for the control human power X, and inputs the control human power X into the rule section fully connected neuro 100. The teacher signal d is given to the weight change section 76 and the learning convergence determination section 60. Then, the control output Y output from the rule unit fully connected neuron 100 by the learning human power
The weight change unit 76 changes the weight values according to the hack propagation method, and the learning convergence determination unit 60 determines the end of learning when the error of the control output Y with respect to the teacher signal d becomes equal to or less than a specified value. to end the learning process.

When the learning process ends based on the learning execution command in S5,
Increment the phase counter i by one in S6, and
Check the end of the learning phase at 7. Since the learning process of the present invention ends at phase 1, the series of processes ends as is.

(Principle of learning by pack propagation method) Next, the learning principle by the pack propagation method of the present invention will be explained together with the hierarchical network.

FIG. 16 shows a general hierarchical neural network, which is composed of a type of node called a unit and internal connections with weights.

In Figure 16, the h layer is the input device, and the i layer is the intermediate layer.
layer, and the j layer as the output layer. Here, there is only one i-layer as the intermediate layer, but the same holds true even if there are a plurality of intermediate layers. As shown in FIG. 4, each unit includes a multiplier 22 that multiplies a plurality of inputs by the weight of each internal connection, and an adder 24 that adds all the multiplication results of the multiplier 22.
and a threshold processing unit 26 that performs threshold processing on the addition result and outputs one output. The calculations performed in this unit are as shown in (1) and (2) above for the basic unit 20.
). In the pack propagation method, the weight value Wih and threshold value θi of the hierarchical network are adaptively and automatically adjusted and learned using error feedback. Above (1)
(2) As is clear from Part 2, the weight value Wih and the threshold value θi
It is necessary to perform the adjustment at the same time, but this is a difficult task that interferes with each other. Therefore, in addition to the normal unit, the applicant of the present application provides a unit whose output value is always 1 in the h layer on the input side, and assigns a threshold value θi as a weight value to the output of this unit with an output value of 1. Then, by incorporating the threshold value θi into the weight value Wih, the threshold value θi is
We have proposed a hack propagation method that can treat i as a weight value (Japanese Patent Application No. 333484/1984).
.

By doing this, the above equations (1) and (2) can be changed to xp・−
ΣyphW・h (11)"jp, -
1/ (1+exp(-x pt)) (12)
Therefore, the threshold value θi does not appear.

The development from (11) and (12) above leads to the following (
13) Equation (14) is obtained.

xpJ−Σy,,WJ, (13)
VpJ=1/(1+exp(XpJ))
(14) where, j: unit number W of the j layer, B: weight value xpj' of the internal connection between the i-j layers, sum of products Vpj'P of inputs from each unit of the first layer to the third unit of the j layer 3 of the j layer for the input signal of the pattern
The weight changing unit 76 in FIG. 14 outputs the input cover turn X for learning.
When receiving the output cover turn ypJ output from the output layer j when presented with the output cover turn y, and the teacher pattern dpj which is the signal that J should take, calculate the difference value (dp; ``1p=) between the two, Next, δ9, -ypi(1-ypJ) (dpJ-YPj)
(15) is calculated. Subsequently, the weight value changing unit 76 calculates δ9. calculated in (15) 2 above. Using , the update amount ΔWih(t) of the weight values of the h layer and the i layer is calculated according to the following.

Next, the weight change unit 76 changes the weight value for the next update cycle according to the update amount calculated by the above equations (16) 2 and (18), where ε: learning constant ζ: momentum t: the i layer according to the number of learning times. Update amount of weight values between j layers ΔWJi(t)
Calculate. Here, the reason why the update amount of the weight value determined in the previous update cycle is added in the second term on the right side of the equation (16) is to speed up the learning.

W1□(t) =wi1. (t-1) +Δwith
(t).

The above learning process is repeated to update the weight values so that a weight value is obtained in which the output pattern y□ corresponding to the learning input pattern matches the teacher pattern dpj.

On the other hand, if the units constituting the hierarchical network in FIG. 16 are linear units, the above equation (14) becomes: y pJ= Xpj
(20). The above equations (15) and (17) in this linear function unit are δ, J= dp=-ypi (2
i) δ2. -Σδ,, WJi (t-1)
(22).

(Weight Learning Process of the Present Invention) FIG. 17 is a processing flow diagram of weight learning performed by the backpropagation method for the rule section fully connected neuron of the present invention shown in FIG.

The parameters of each part in this learning process flow are the 18th
It is defined as shown in the figure.

Figure 18 schematically shows the hierarchical structure of the rule section fully connected neuron in Figure 5, with layers 1 to 6 as shown.
It has a six-layer structure, with the first and third layers consisting of sigmoid function units, and the rest consisting of linear function units. Also, the learning target is the first one.
layer to the fourth layer, and the center of gravity calculation realization unit 18 at the final stage.
The sixth layer in is excluded from weight learning.

Here, the fuzzy control value outputted by realizing the fuzzy rule by the fully connected neuron of the rule part, that is, the control output value, is yb, and 1. The learning process is performed from the 6th layer to the 5th layer and the 4th layer.
layer, third layer, second layer, first layer, and so on.

The progress of learning for each layer is indicated by an i counter. That is, in the initial state, the i counter is sent to the i layer 5.

To further simplify the explanation, Wi+i−+ and ΔWi
+i-1 indicates a matrix, the size of which is the number of neurons in the i-th layer)X (the number of neurons in the i-1th layer).

Furthermore, δi and yi are shown as vectors, and the size is the same as the number of neurons in the i-th layer.

The weight learning processing flow in FIG. 17, for example, learning in phase 1, will be explained as follows based on the parameters of each part shown in FIG. 18.

First, in step SL (hereinafter "step" will be omitted and simply described as 81), a counter i for setting a learning target layer is initialized. Specifically, since the device has a six-layer configuration as shown in FIG. 18, the counter i is initially set to i=6. With this initial setting, the sixth layer is designated as the learning target.

Next, the process proceeds to 82, where it is determined whether the unit is a sigmoid function unit. In this case, since it is a linear function unit, 314
The process proceeds to step 315, where it is determined whether the layer is the final layer, and since the sixth layer is the final layer, the process proceeds to step 315, where the difference value δ6 is obtained using the teacher signal d and control output y6 obtained at that time.

Next, the process proceeds to 86, where it is determined whether the unit is to learn weights. At this time, the 6th layer is excluded from the learning target (
Specifically, since the learning flag of the learning adjuster 80 provided in the connection is off), the process proceeds to S8, sets the weight value update amount ΔW6S to ΔWhs=0, and proceeds to S9. In this case,
Since ΔWbs=O, the weight value Was is not updated. Next, proceed to S12 and check whether the counter i=1 or not.
That is, it is determined whether learning of all layers has been completed, and since i=6 at this time, the process proceeds to 313 and the counter i is incremented to 1.
Decrement by one to set i=5, and return to S2 to start the next fifth
Proceed to learn layers.

Since the fifth layer is also a linear function unit, 82
Proceeds to 314, but since it is not the final stage layer, it goes to 316
Proceed to step 6 and calculate the difference value δ6 obtained in the learning process of the previous 6 layers.
and the weight value W65, the difference value δ is determined according to a formula similar to the above formula (22).

Next, proceed to 86 to determine whether it is a learning target,
Since the fifth layer is not a learning target in the present invention, the weight value W6. Do not update (S8-39-312-3
13).

Next, all units of the fourth layer are processed Sl 6-
36-37-39-312-513) When the fourth layer processing is completed, the process proceeds to the third layer.

Proceeding to the third layer processing, since the third layer uses a sigmoid function unit, the process proceeds from 82 to 83, and since it is not the final layer unit, the difference value δ3 unique to the sigmoid function unit is calculated in step S5. (17) Calculate using the same formula as in 2.

Since this third layer is also the learning target, 36 to 8
7, the weight update amount ΔW3□ is calculated using the difference value δ3 calculated in S5, and updated in S9.

Repeat the same process up to the final unit of these three layers, and
Proceeding to the first layer, learning processing regarding linear function units is performed, and further advancing to the first layer, learning processing regarding sigmoid function units is repeated. Then, when the processing of the final unit of the first layer is completed and the process proceeds from S10 to 312, since the counter i has become i-1 at this time, it is determined that learning of all layers has been completed, and a series of Finish the process.

(Initial Setting of Weights Using Random Numbers) Next, another embodiment of the learning method of the present invention will be described with reference to the operation flow diagram of FIG. 19.

In the learning process shown in the operation flow diagram of FIG.
l, S2. Random numbers were used as the initial setting of weight values (including thresholds) for the antecedent membership function realization unit 10, the rule unit 12, and the consequent membership function realization unit I4 of the rule unit fully connected neuron in S3. It is characterized by Other 34. S5. The processing at S6 is the same as the first embodiment shown in FIG. Furthermore, if there is some knowledge that is previously held, settings may be made based on this knowledge.

(Others) The above-mentioned specific example was an example of a rule section fully connected neuron that realizes a fuzzy rule targeting two control inputs X+ and Xz, as shown in FIG. However, it goes without saying that the number of control inputs may be one or three or more.

In addition, in the above embodiment, y=1/ (1+e-"") (23
) was taken as an example of the membership function, but
Two sigmoid function units 9 as shown in FIG.
4-1.94-2 uni-nod output subtraction unit 96
may be given to realize the sigmoid function shown in FIG. In this case, the unit output y is '! =1/(1-e-IIX+θli -1/(1-e
−””θ2).

[Effects of the Invention] As explained above, according to the present invention, the antecedent membership function implementation unit, the rule unit, and the consequent membership function implementation unit each have knowledge (initial Efficient learning is possible by training the antecedent membership function realization part after learning the rule part of the fully connected neuron after setting or initial setting using random numbers or a combination thereof. Therefore, it is possible to realize a more efficient connection weight analysis than the fuzzy rule implementation used in an arbitrary fuzzy system <B Jl response system.

[Brief explanation of drawings]

Fig. 1 is an explanatory diagram of the principle of the present invention, Fig. 2 is a learning operation flow diagram of the present invention, Fig. 3 is an explanatory diagram of the definition of the rule section fully connected neuron, Fig. 4 is an explanatory diagram of the basic unit, and Fig. 5 is an explanatory diagram of the basic unit. 6 is an explanatory diagram of the neuron group and connection group of FIG. 5, and FIG. 7 is an antecedent membership function of the present invention. An explanatory diagram of the initial settings of the realization section, FIG. 8 is an explanatory diagram of the membership function of the antecedent section, FIG. 9 is an explanatory diagram of the rule section unit of the present invention, and FIG. 10 is an explanatory diagram of the membership function realization section of the consequent section of the present invention. FIG. 11 is an explanatory diagram of the initial settings of the center of gravity calculation implementation unit of the present invention, FIG. 12 is an explanatory diagram of the settings of the phase/group correspondence table and group/connection correspondence table of the present invention,
Fig. 13 is a learning process flow diagram of the present invention, Fig. 14 is a configuration diagram of a device for learning processing according to the present invention, Fig. 15 is a configuration diagram of a learning regulator provided for each connection in Fig. 14, and Fig. 16 is a general diagram. Fig. 17 is a flowchart of the weight learning process of the present invention, Fig. 18 is an explanatory diagram of parameters of each part of Fig. 17, and Fig. 19 shows another embodiment of the present invention. The operation flow diagram, FIG. 20, is an explanatory diagram of another membership function used in the present invention. 10...Antecedent part membership function realization part, 12
...Rule part, 14...Consequent part membership function realization part, 16
...Input section, 18.. Center of gravity calculation realization section, 20.. Basic unit, 22.. Multiplication section, 24.. Addition section. , 26...Threshold processing unit, 28-1.28-2.32-1~32-4°38-1~
38-9.40-1.40'-2...Linear function unit 30-1 to 30-2.34-1 to 34-5°94-1.
94-2...Sigmoid function unit 40...
... Center of gravity determining element output device, 42 ... Center of gravity calculation device, 50 ... Learning instruction unit, 52 ... Learning scheduler, 54 ... Learning Signal storage unit, 56... Phase group correspondence table, 5
8...Group phase correspondence table, 60
... Learning convergence determination section, 70 ... Learning unit, 72 ... Learning instruction reading section, 74 ... Learning flag setting section, 76 ... ... Weight change section, 80 ... Learning adjuster, 82 ... Flag storage section, 84 ... Weight change information reading section, 86 ...
-Weight change amount adjustment unit, 88...Weight calculation unit, 90...Weight storage unit, 92...Weight change amount storage unit, 96...Subtraction Unit, 100... Rules section fully connected neuro agent Patent attorney Mitsuo Honma's learning masterpiece flow diagram Buzu Ho Hatsuro's antecedent section Memberbashi Tsubu Kankyo Ikutsutsu section initial setting explanatory diagram Genealogy unit's Mushikai immediate precondition Meshibaship Seki@, explanatory diagram Figure Mizuhamoe's rules rhyme 2 knit theory diagram Figure t of the present invention, ((calculation actual trap refreshing initial t9jl theory Yamifu 11 phase ゛ Nigel loop Versus i: Front Group: Counterintelligence and Flying Friends This invention's phase, group vs. table and σ゛ group, Aya line feudal table setting V Creation flow episode showing an example of the place of origin of water in the same figure /q Figure 4-l θ2 (a) (b) Honsharo 1: Quick explanation of other membership functions used

Claims (5)

[Claims] 1. An antecedent membership function realization unit (10) configured by providing a plurality of basic units that calculate and output membership function values of input signals, and calculation result values output from the preceding processing layer and the calculations. Equipped with one or more processing layers consisting of at least one basic unit that calculates and outputs a calculation result value according to the assigned operation, using as input the multiplication value of the weight value of the internal connection associated with the result value. , and between the basic units of the adjacent processing layers, the basic unit of the first processing layer, and the antecedent membership function realization unit (10).
A rule section (12) configured by adopting a configuration in which the basic units are mutually connected internally, and a calculation result value output by the corresponding basic unit of the last processing layer of the rule section (12). a hierarchy consisting of a consequent membership function realization unit (14) constituted by at least one basic unit that calculates and outputs an output signal value derived according to the membership function value of the output signal with input as input; It has a network and input control state quantity (X_1~
In the rule part fully connected neuron that outputs the control operation amount (Y_1 to Y_m) corresponding to Part (14
), and a first process of setting weights and initial threshold values for each of the rule units (1) using pre-prepared learning data.
2) After learning the weights, a second step of learning the weights of the antecedent membership function realization unit (10). 2. The initial values of the weights and thresholds of the antecedent membership function realization unit and the consequent membership function realization unit of the first process are set to initial values created based on knowledge held in advance. 2. The method for learning a fully connected rule part neuron according to claim 1, wherein the weights and threshold values of the rule part are set using random numbers. 3. 2. The method for learning a fully connected rule part neuron according to claim 1, wherein the initial values of the weights and thresholds in the first step are set using random numbers. 4. In setting the initial values of the weights and thresholds in the first process, the weights and thresholds that can be set using prior knowledge are initialized using this knowledge, and the initial values of the other weights and thresholds are set using random numbers. 2. The method of learning rule part fully connected neurons according to claim 1. 5. At the time of starting learning in the second process, a learning flag is set for each connection of the hierarchical network of the antecedent membership function realization unit (10) and the rule unit (12), and is set on the connection. 2. The method for learning a fully connected rule part neuron according to claim 1, wherein the weight values are optimized through a learning process based on the learned flags.