JPH0831116B2 - Learning adjustment method of connection in neural network - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Outline] Regarding a learning adjustment method of connection in a neural network when a fuzzy control system is configured by using a neural network, a fully connected neural network is converted into a network having a certain special structure. And to adjust the degree of learning for each connection. For each of all the connections between the units that make up the neural network, the weight of the weight obtained by the backpropagation method during learning of the neural network A learning adjustment constant holding unit that holds a multiplication constant for multiplying the change amount to obtain an actual weight change amount as a learning adjustment constant, and a weight change amount determination unit that determines the actual weight change amount according to the learning adjustment constant. One of the learning adjustment constant holding means for each connection provided A learning adjustment constant means for receiving the learning adjustment constant setting instruction to the learning adjustment constant holding means and setting the instructed learning adjustment constant in the learning adjustment constant holding means corresponding to the instructed combination. To configure.

[Industrial applications]

The present invention relates to a learning method for a neural network,
More specifically, it relates to a learning adjustment method for coupling in a neural network when a fuzzy control system is constructed using the neural network.

In recent years, pattern recognition devices using neural networks and adaptive filters have been extensively studied. Furthermore, in fields where fuzzy models have been conventionally applied, such as plant control, fuzzy models have been implemented as neuro models.

When the fuzzy model is realized by a neuro model, in order to create a model that describes the relationship between the membership functions of the fuzzy control rules, for example, the output from each unit in the first layer is second It is necessary to use a neural network with some special structure, unlike a fully connected neural network which is input to all units of a layer.

In addition, in the fuzzy neuro fusion system, if it is not necessary to learn much for some connections and the learning speed needs to be increased for other parts, learning of each connection that constitutes the neural network is performed. It is also necessary to fine-tune the degree. For this reason, it is desired to convert a fully connected neural network into a network having a specific structure and finely adjust the degree of learning of each connection.

[Conventional technology]

Fuzzy control is becoming popular as a new control processing method. In this fuzzy control, a control algorithm including ambiguity such as human judgment is expressed by if-then type, and when the control algorithm is executed according to fuzzy inference, the control operation amount is detected from the detected control state amount. It calculates and controls the controlled object. In order to realize this fuzzy control, it is necessary to tune the membership function that numerically expresses the "ambiguous language expression" described in the fuzzy control rule to one that matches the control target. .

For example, the generation of fuzzy control rules described in the form of if x ₁ is big and x ₂ is small then y is big (if part is called antecedent part and then part is called consequent part) According to the knowledge of the target process of the operator who is familiar with the control of the controlled object, first, the linguistic meanings regarding the values of the control state quantity and the control operation quantity are quantified as a membership function, and then these A coarse model of fuzzy control rules is generated by describing the relationships between membership functions. Then, the generated fuzzy control rule model is tuned including the membership function while being evaluated by simulations and field tests to achieve a fuzzy control rule that matches the control target. It had been.

As described above, conventionally, the fuzzy control rule generation process and the membership function tuning process described in the rule are familiar to the designer who is going to construct the fuzzy controller and the control of the controlled object. It was done manually in collaboration with the operator. Since the work for generating the fuzzy control rule and tuning the membership function needs to be performed by trial and error while making full use of know-how, it usually takes several months at the shortest. When the control knowledge regarding the controlled object is small, this period is even longer.

From the above background, recently, a fuzzy system using a neural network having a learning function, that is, a fuzzy neuro fusion system has been realized. In a conventional computer with a built-in program, it was necessary to specify the operation in all cases.
In a neural network, learning is performed by presenting a typical learning pattern several times, and it becomes possible to output an appropriate solution even for a pattern that is not given. It is widely known that there are a hierarchical network and a backpropagation method as a typical structure and learning method of the neural network.

[Problems to be Solved by the Invention]

In the fuzzy neuro fusion system as described above, since the model describing the fuzzy control rule is created as a neural network, as in the past,
For example, the outputs of all the units in the first layer are input to all the units in the second layer, that is, unlike a fully connected neural network, there is not necessarily a coupling between the units even between two consecutive layers. Neural networks with a special structure are required.

In addition, when the learning of such a neural network is performed and the goodness of fit of a certain membership function to the controlled object is high, the learning of the connection corresponding to that part is slower than the learning of other connections. It is necessary to fine-tune the degree of learning for each combination, such as good. However, conventionally, there is a problem that a fully connected neural network cannot be converted into a network having a special structure, and the degree of learning cannot be finely adjusted for each connected neural network. It was

The present invention aims at converting a fully connected neural network into a network having a special structure and adjusting the degree of learning for each connection for the purpose of realizing a fuzzy model by a neuro model. .

[Means and Actions for Solving the Problems]

FIG. 1 (a) is a block diagram of the principle of the present invention. This figure is a principle block diagram of a learning adjustment method of coupling in a neural network as a model of a fuzzy control system. The learning adjustment constant holding means (1a, 1b, ...) Is provided for all the connections between the units forming the neural network and multiplies the weight change amount obtained by the backpropagation method during learning of the neural network. Then, the multiplication constant for obtaining the actual weight change amount is held as the learning adjustment constant.

Similarly, the weight change amount determining means 2a, 2b, ... Are respectively provided for all the connections, and when the weight change amounts obtained by the backpropagation method during learning of the neural network are input to the connections, for example. By multiplying the change amount by the learning adjustment constant held in the learning adjustment constant holding means 1a, 1b, ..., the actual weight change amount is determined, and the value is given to the direct combination, for example, The weight of the binding is changed.

The learning adjustment constant setting means 3 is for setting the learning adjustment constants for the learning adjustment constant holding means 1a, 1b, ... Corresponding to each connection, and of the learning adjustment constant holding means for each connection. In response to the learning adjustment constant setting instruction to one or more learning adjustment constant holding means, the designated learning adjustment constant is set in the learning adjustment constant holding means corresponding to the instructed combination.

FIG. 1 (b) shows a connection between some units from a fully connected neural network in which the output from each unit in the first layer of the neural network is input to all the units in the second layer. It is a functional block diagram of a conversion method of a network having a certain special structure. In the figure, the initial value of the weight is set to 0 at 4 for one or more connections that do not need to be connected among all the connections between the units forming the neural network. Then, in 5, the learning adjustment constant setting means 3 sets 0 or off as a learning adjustment constant in the learning adjustment constant holding means corresponding to the one or more connections, and 6 according to the setting corresponds to the one or more connections. The learning adjustment constant holding means holds 0 or off as the value of the learning adjustment constant.

After that, when learning the neural network,
Even if the weight change amount determining means for the one or more combinations described above is input to the connection, the weight change amount determining means multiplies the value by the learning adjustment constant 0 or sets the weight change amount. By directly setting 0, 7 continues to output 0 as the actual weight change amount.

As described above, the weight change amount continues to be 0 at the time of learning by the back-propagation method for one or more connections whose weight is set to 0 as an initial value. Therefore, the weight for the one or more connections is always 0, the connections are the same as they do not exist, and a network having a specific structure is realized.

As described above, according to the present invention, the learning adjustment constant can be set for all the connections between the units forming the neural network.

〔Example〕

FIG. 2 is a block diagram of the entire configuration of a fuzzy neuro fusion system using the learning adjustment method of connection in the neural network of the present invention. In the figure, 10-h is a plurality of input units forming an input layer of the hierarchical network unit 11, 10-i is a plurality of basic units forming an intermediate layer (a plurality of stages are provided) of the hierarchical network unit 11, 10-j.
Is one or more basic units forming an output layer of the hierarchical network unit 11, 12 is a hierarchical network configuration data processing device, 13 is a weight value storage unit, and a weight value assigned to the internal connection of the hierarchical network unit 11 Is to manage.

Reference numeral 21 denotes a learning signal storage unit that stores learning control data used for learning the weight value. Reference numeral 22 denotes a learning signal presenting unit that stores learning control data from the learning signal storage unit 21. And presents the control presentation data in it to the hierarchical network unit 11, and presents the other pair of control teacher data to the weight value changing unit 14 described later and the learning convergence determination unit 23 described next. Numeral 23 is a learning convergence determining unit, which receives the control amount data from the hierarchical network unit 11 and the control teacher data from the learning signal presenting unit 22, and determines the error in the data processing function of the hierarchical network unit 11. The learning signal presenting unit 22 is notified of the determination result of whether or not it is within the allowable range.

A weight value changing unit 14 receives the control teacher data from the learning signal presenting unit 22 and the network output data of the hierarchical network unit 11, calculates the weight value update amount according to the back propagation method, and weights it. By updating the value, learning is performed so that the weight value converges to the convergent value.

Before going into the detailed description of the fuzzy neuro fusion system in the present invention, in order to learn the weight value assigned to the internal connection of the hierarchical network unit 11, the back value which is the learning algorithm executed by the weight value changing unit 14 is learned.・ Propagation method (DERumelhart, G, E.Hinton, a
nd RJWilliams, “Learning Internal Representation
s by Error Propagation ", PARALLEL DISTRIBUTED PROCE
SSING, Vol.1, pp.318-364, The MIT Press, 1986) will be described in detail.

The hierarchical network unit 11 basically forms a hierarchical network from a kind of node called a basic unit and an internal connection having a weight value. The structure of the basic unit 10 is shown in FIG. The basic unit 10 is a multi-input / single-output system, and has a multiplication processing unit 15 that multiplies a plurality of inputs by weight values of respective internal couplings, and an accumulation processing unit 16 that adds all multiplication results. And a threshold processing unit that performs non-linear threshold processing on the accumulated value and outputs one final output.
17 is provided.

Assuming that the h layer is the front layer and the i layer is the rear layer, the accumulation processing unit 16 executes the calculation of the following equation (1) and the threshold processing unit
In step 17, the calculation of the following equation (2) is executed.

y _pi = 1 / (1 + exp (x _pi + θ _i )) (2) where, h: h layer unit number i: i layer unit number p: input signal pattern number θ _i : i layer i unit Threshold value w _ih : h− weight value of internal coupling between i layers x _pi : sum of inputs from units in layer i to unit i in layer y _ph : h in layer h for input signal of p-th pattern Output from the unit No. y _pi : Output from the unit No. i of Layer i for the input signal of the p-th pattern Then, in the hierarchical network unit 11, a large number of basic units 10 having such a configuration distribute the input signal value as it is. The input unit 10-h that outputs the
As shown in the figure, a hierarchical network is constructed by connecting them hierarchically, and the parallel data processing function of converting an input pattern (input signal) into a corresponding output pattern (output pattern) is exhibited. Become.

In the back propagation method, the weight value W _ih of the hierarchical network and the threshold value θ _i are adaptively automatically adjusted and learned by feedback of the error. (1),
As is clear from the equation (2), the adjustment of the weight value W _ih and the threshold value θ _i needs to be executed at the same time, but this work is a difficult work that interferes with each other. Therefore, the applicant
As disclosed in the previous application “Japanese Patent Application No. 62-333484 (filed on December 28, 1987,“ Network configuration data processing device ”)”, always output “1” to the h layer on the input side. the example threshold theta _i to its output by providing a unit assigned as a weight value, and suggested to treat the threshold theta _i as the weighting value incorporates threshold theta _i in the weight value W _ih together. By doing this, (1),
Equation (2) is y _pi = 1 / (1 + exp (−x _pi )) (4) Equation (4)

Next, a weight value learning processing method by the back propagation method will be described according to the description format of the expressions (3) and (4). Here, this description will be made assuming that the hierarchical network unit 11 has a three-layer hierarchical network of h layer-i layer-j layer shown in FIG.

By analogy with the equations (3) and (4), the following (5),
Expression (6) is obtained. That is, y _pj = 1 / (1 + exp (−x _pj )) (6) where, j: j layer unit number W _ji : i−j layer internal coupling weight value x _pj : i layer to j layer Sum of the inputs to the j-th unit of y _pj : Output from the j-th unit of the j-th layer for the input signal of the p-th pattern The weight value changing unit 14 outputs when the learning input pattern is presented. Output pattern y from output layer
and _pj, receives a teacher pattern d _pj is the signal to be taken by the output pattern y _pj (teacher signal to the j layer j-th unit with respect to the input signal of the p-th pattern), first of all, the output pattern y _pj The difference value [d _pj −y _pj ] from the teacher pattern d _pj is calculated, and then α _pj = y _pj (1-y _pj ) (d _pj −y _pj ) (7) is calculated, and then , However, ε: learning constant ζ: momentum t: update amount ΔW _ji (t) of the weight value between the i-th layer and the j-th layer according to the learning number
To calculate. Here, “ζΔW _ji (t−1)”, which is related to the update amount of the weight value determined in the previous update cycle, is added to speed up the learning.

Subsequently, the weight value changing unit 14 uses the calculated α _pj ,
First of all, And then According to the update amount ΔW _ih (t) of the weight value between the h layer and the i layer
To calculate.

Subsequently, the weight value changing unit 14 calculates the weight value for the next update cycle according to the calculated update amount, W _ji (t) = W _ji (t−1) + ΔW _ji (t) W _ih (t) = By repeating the method of determining W _jh (t−1) + ΔW _ih (t), the output pattern y _pj from the output layer when the learning input pattern is presented, and its output pattern y _pj The weighting values W _ji and W _ih are matched so that the output pattern y _pj and the teacher pattern d _pj, which is the desired signal, match.

Then, the hierarchical network unit 11 includes the g layer-h layer-i layer-
When having a four-layer hierarchical network of j layers, the weight value changing unit 14 first And then According to the calculation of the update amount ΔW _hg (t) of the weight value between the g-layer and the h-layer, the update amount ΔW of the weight value between the layers on the preceding stage side is calculated by using the value obtained from the output stage and the network output data. It is a process that makes decisions while using it.

In the present invention, the middle layer of the hierarchical network unit 11 has four layers.
It shall consist of layers.

When executing the tuning of the membership function, for example, when the antecedent part describes the condition regarding the control state quantity and the consequent part describes the fuzzy control rule describing the condition regarding the control operation amount, the hierarchical network is processed. Function for calculating and outputting the truth value of the membership function for the control state quantity to be tuned using the basic unit 10 in the middle layer of the front part of section 11 (the truth value of the antecedent section membership function Function to output the truth value of the membership function for the control operation amount to be tuned, using the basic unit 10 of the fourth intermediate layer (the consequent). The function of outputting the truth value of the department membership function) is realized.

FIG. 5 illustrates an example of the membership function. The truth value calculation function of the antecedent part membership function is, specifically, the output value y output from the basic unit 10. As shown in FIG. 6 (a), when ω <0, θ <0, the function shape is similar to the membership function “low temperature” in FIG. 5, and ω> 0, θ When> 0, attention is paid to the fact that the function shape is similar to the membership function “high temperature” in FIG. 5, and as shown in FIG. By appropriately setting the weight value ω and the threshold value θ, a membership function having a function shape of “low temperature” or “high temperature” in FIG. 5 is realized.

Further, the difference value y between the output values output from the two basic units 10 As shown in Fig. 7 (a), "temperature is normal" in Fig. 5
Paying attention to the fact that the function shape is similar to that of the membership function, as shown in FIG. 7 (b), two basic units 10 and output values of these two basic units 10 have weight values " A subtraction unit 10a which is an input that is obtained by multiplying 1 "and" -1 "and that is not provided with the threshold processing unit 17 (according to this configuration, these two basic units 10
And the threshold values θ ₁ and θ ₂ are appropriately set to adjust the weight values ω ₁ and ω ₂ for the inputs of the two basic units 10, The assignment process of the membership function having the function shape of “normal temperature” in the figure is realized.

By adopting such a configuration, it is possible to change the function shape of the antecedent part membership function by changing the weight value and the threshold value with respect to the input values of these basic units 10.

The output function of the processed value of the consequent part membership function is, as shown in FIG. 8A, specifically, the consequent part membership function is finely divided into the truth values y _{i of the} respective parts. , And then, as shown in FIG. 8 (b), the threshold processing unit
Prepare the truth value output devices 10b constituted by the basic unit 10 not including 17 for the number (n) of the truth values, and set the weight value “y _i ” for the input of the truth value output device 10b. It is realized by doing. Here, as shown in FIG. 8B, the truth value of the consequent part membership function relating to the control operation amount of the same type such as the opening degree of the valve A is the same truth value output device 10b prepared. And the truth value output device 10b is configured to output the functional sum of the truth values of the reduced membership function for the control operation amount to be assigned.

Since such a configuration is adopted, by changing the weight value with respect to the input value of the truth value output device 10b, the function shape of the case part membership function can be changed thereafter.

Next, using the basic unit 10 of the second middle layer of the hierarchical network unit 11 to realize the antecedent operation function of the fuzzy control rule to be processed, and the third middle layer of the middle layer. Realization of the consequent operation function of the fuzzy control rule to be processed using the basic unit 10 will be described.

This antecedent part arithmetic function is realized in conformity with the antecedent part arithmetic function of the fuzzy control rule to be processed. Therefore, when the antecedent part arithmetic function calculates and outputs the average value of the input values, for example, as shown in FIG. 9 (a), a product obtained by multiplying the weight value by "1 / the number of input points" is used. As input,
Further, it is realized by including the average value calculator 10c configured by the basic unit 10 that does not include the threshold value processing 17. Similarly, this consequent part calculation function is realized by matching with the consequent part calculation function of the fuzzy control rule to be processed. Therefore, when the consequent part calculation function is to calculate and output the added value of the input values, for example, as shown in FIG. 9 (b), the one multiplied by the weight value “1” is input, Further, it is realized by including the addition value calculator 10d configured by the basic unit 10 not including the threshold value processing unit 17.

Hereinafter, for convenience of explanation, an example will be described in which the average value calculator 10c is provided to realize the antecedent part arithmetic function, and the addition value calculator 10d is provided to realize the antecedent part arithmetic function.

Then, the basic unit 10 to which the antecedent membership function is assigned and the average value calculator 10c to which the antecedent arithmetic function is assigned are internally coupled in a format according to the description of the fuzzy control rule to be processed. At the same time, the average value calculator 10c and the additional value calculator 10d assigned with the consequent part arithmetic function are internally coupled. The setting process of the inner connection is specifically realized by assigning a weight value of zero value which is not changed by learning to the inner connection having no connection relationship as described later.

The operation value output from the consequent part arithmetic function basically gives the reduction rate of the truth value of the corresponding consequent part membership function. For example, as shown in FIG. If the membership function "Set small opening of" is expressed as and the membership function "Set large opening of valve A" is expressed as According to the output value of the consequent part calculation function that is set, the applied value of the membership function "Set the opening of valve A small" is determined, and according to the output value of the consequent part calculation function that is associated with the membership function of The applied value of the membership function "setting the opening of the valve A large" is determined. And most commonly, by calculating the center of gravity of the figure of the functional sum of this reduced membership function according to the following equation, A configuration is adopted in which the opening Y of the valve A, which is a fuzzy inference value, is calculated.

FIG. 11 is an overall block diagram of a fuzzy neuro fusion system in which fuzzy control rules are mapped. In the figure, the circle without a symbol indicates the basic unit 10,
With the basic unit 10, the input unit 10-h, and the units 10a to 10d in which the basic unit is partially modified, the antecedent part membership function value calculation function 24, the antecedent part calculation function 25, the antecedent part calculation function 26, Consequent part membership function value output function
27 and a fuzzy inference value calculation function 28 are configured as a neural network. In addition, X1 (SA) in the figure is X1
Represents the truth value of X1 found by the membership function SA (small) for the control state quantity. MM in parentheses means'normal 'and LA means'large'. further
LHS-1 represents the operation result value of the antecedent operation, and Y (SA) is the membership function SA for the control operation amount Y.
It corresponds to (small).

In FIG. 11, the output of the fuzzy inference value calculation function 28 is used to calculate the final value of the center of gravity as control operation amount data by a calculation unit (not shown), and the value is the learning convergence determination unit 23 of FIG. And is output to the weight value changing unit 14.
Then, the learning convergence determining unit 23 gives an instruction to update the weight to the learning signal presenting unit 22, and the weight changing unit 14 according to the back propagation method described above, from the antecedent part membership function value calculating function 24 of FIG. The weights are updated in the neural networks that constitute the fuzzy inference value calculation function 28.

The fuzzy neuro fusion system shown in FIG. 11 is not a fully connected system in which the output of each unit in one layer is input to all the units in the next layer like a general neural network, and the fuzzy control rules are By being mapped, it becomes a neural network with a special structure. Further, in the tuning of the neural network, that is, in the learning of the neural network performed for adapting the membership function to the controlled object, it is desirable that the degree of learning can be adjusted between the portion having a high degree of conformity to the controlled object and the portion having a low degree of conformity. For this purpose, the present invention provides a learning adjuster for each connection of the fully connected neural network.

FIG. 12 shows an embodiment of a learning adjustment constant setting device for setting a learning adjustment constant in the learning adjustment device provided in each connection.
In the figure, learning adjustment devices 31a, 31b, ... Are provided for each connection forming the neural network 30, and a learning adjustment constant setting device for setting learning adjustment constants in these learning adjustment devices. 32 are provided. The learning adjustment constant setting device 32 is a neural network.
The instruction reading unit 33a receives an instruction to set a learning adjustment constant for one or more couplings in 30 and a learning adjustment constant setting unit 33b that sets a learning adjustment constant in the learning adjustment device for the corresponding coupling according to the instruction. ing.

It should be noted that in FIG. 12, each connection in the neural network 30 seems to pass through the corresponding learning adjustment device, but in reality, the learning adjustment device outputs the actual weight change amount for the corresponding connection. The connection itself does not go through the learning adjustment device.

FIG. 13 is a configuration block diagram of an embodiment of a learning adjustment device corresponding to each combination. In the figure, the learning adjustment device 31 is
A constant storage unit 34 for storing the learning adjustment constant set for the connection by the learning adjustment constant setting device 32, weight change information for reading the weight change information given to the connection by the backpropagation method at the time of learning of the neural network. A weight change amount adjustment unit that determines the actual weight change amount by multiplying the learning adjustment constants in the reading unit 35 and the constant storage unit 34 by the weight change amount based on the back propagation method read by the weight change information reading unit 35. Composed of 36.

In FIG. 13, a combination 37 corresponding to each learning adjustment device 31 has a weight storage unit 38 for storing the weight for the combination, and a weight change amount storage for transmitting the weight change amount obtained by the backpropagation method at the time of learning. The actual weight change amount determined by the weight change amount adjustment unit 36 in the learning adjustment device 31 according to the weight change amount based on the backpropagation method transmitted to the unit 39 and the weight change amount storage unit 39 is the weight change amount. A weight changing device that changes the weight for the combination 37 using the actual weight change amount after being transmitted to the storage unit 39.
40 are provided.

FIG. 14 is an embodiment of the control flow of the learning adjustment constant setting device. In the figure, when the processing is disclosed, the learning adjustment constant setting device 32 reads the combination for setting the learning adjustment constant and the value of the constant set therein by the instruction reading unit 33a in step (S) 41, and instructing in S42. The constant that has been set is set to that constant, for example, an arbitrary value between 0 and 1. And
In S43, it is determined whether or not the instruction is over. If it is not over, the process from S41 is repeated. When the instruction is over, the process ends.

FIG. 15 is a flowchart of an embodiment of a process for converting a fully connected neural network into a network having a specific structure. In the figure, when the process is started, S4
The information of the bond to be cut is read by the instruction reading unit 33a of the learning adjustment constant setting device 32 in 4, and the combination 3 instructed in S45 is read.
0 is set as the initial weight in the weight storage unit 38 of 7 and 0 (off) is set as the value of the learning adjustment constant in the learning adjustment device 31 corresponding to the combination instructed in S46, and whether the instruction ends in S47. Is determined, and if it is not the end, the processing from S44 is repeated. Then, when it is determined that the instruction has been completed, all the processing ends.

FIG. 16 is a flowchart of a processing embodiment of the learning adjustment device 31 when designating the presence or absence of learning of each connection during learning of the neural network. In the figure, S4
At 8 when the weight change amount storage unit 39 of FIG. 13 transmits the weight change amount obtained by the back propagation method to the weight change amount storage unit 39, the weight change information reading unit 35 outputs the weight change information. Is read, and it is determined in S49 whether the value of the learning adjustment constant stored in the constant storage unit 34 is 0 (off).

When the value is 0 (off), the weight change amount adjustment unit 36 sets the weight change amount to 0 in S50, and the weight change amount is given to the weight change amount storage unit 39. Is not 0 (on), the new weight change amount by the weight change amount adjustment unit 36 is not transmitted to the weight change amount storage unit 39, and the weight change in the combination 37 is the weight change amount obtained by the backpropagation method. Is performed by the weight changing device 40 according to the instruction (S55) to use as is.

FIG. 17 is a flowchart of a processing embodiment of the learning adjustment device in the case of multiplying the learning adjustment constant by the amount of weight change by the back propagation method. In the figure, in S51, the weight change information reading unit 35 reads the information about the weight change from the weight change amount storage unit 39 and the change amount, and S5
In 2, the value of the learning adjustment constant is output from the constant storage unit 34 to the weight change amount adjustment unit 36, and in S53, the low weight value is multiplied by the change amount read by the weight change information reading unit 35 to change the actual weight. The amount is obtained, and the value is set as a new weight change amount in the weight change amount storage unit 39 of the combination 37 in S54.
Then, the weight is changed by the weight changing device 40, and the value of the weight is stored in the weight storage unit 38.

FIG. 18 shows an embodiment of realizing a neural network having a specific structure and adjusting the learning speed. In the figure, the neural network having the structure shown in (a) is realized by using the fully connected neural network shown in (b). In this case, first, the learning adjustment device is provided for all the connections of the fully connected neural network of (b).

Then, as shown in FIG. 7C, first, 0 is set as an initial weight for a combination that is not shown in FIG. After that, as shown in (d), 0 (off) is set as the value of the learning adjustment constant in the learning adjustment device for the combination that is not in (a), and 1 (on) is set in the other learning adjustment devices. It Note that the learning adjustment device in which 1 is set as the value of the learning adjustment constant is not shown in (d).

FIG. 18 (e) shows an example of learning speed adjustment. In the figure, the learning speed between the second layer and the third layer is halved with respect to the learning speed between the first layer and the second layer when viewed from the input side. A learning adjustment constant of 1 is set in the learning adjustment device in between, and 0.5 is set in the learning adjustment device for the coupling between the second and third layers.

〔The invention's effect〕

As described in detail above, according to the present invention, it is possible to easily realize a network having a specific structure from a fully connected neural network as in the related art.
In addition, the fine adjustment of the degree of learning for each connection becomes easy, and it has a great contribution to the practical application of the fuzzy neuro fusion system.

[Brief description of drawings]

FIG. 1 (a) is a block diagram of the principle of the present invention, FIG. 1 (b) is a functional block diagram of a method for realizing a neural network having a specific structure, and FIG. 2 is a learning of connection in the neural network of the present invention. Fig. 3 is a block diagram showing the configuration of the entire system using the adjustment method. Fig. 3 is a diagram showing the configuration of basic units in the hierarchical network. Fig. 4 is a diagram showing the basic configuration of the hierarchical network unit. Fig. 5 is an explanation of the membership function. 6 and 6 are explanatory diagrams of the truth value calculation function of the antecedent part membership function (No. 1), and FIG. 7 is an explanatory diagram of the truth value calculation function of the antecedent part membership function (No. 2). ), FIG. 8 is an explanatory diagram of the truth value output function of the consequent part membership function, FIG. 9 is an explanatory diagram of the antecedent part arithmetic function and the consequent part arithmetic function, and FIG. 10 is a fuzzy inference. Illustration of values, 11th Is a block diagram showing a configuration of an entire system in which fuzzy control rules are mapped, FIG. 12 is a block diagram showing a configuration of an embodiment of a learning adjustment constant setting device, and FIG. 13 is a learning adjustment corresponding to each combination. FIG. 14 is a block diagram showing a configuration of an embodiment of the device, FIG. 14 is a diagram showing an embodiment of a control flow of the learning adjustment constant setting device, and FIG. 15 is a process of converting a fully connected neural network into a network of a specific structure. FIG. 16 is a flowchart of a processing embodiment of the learning adjustment device when designating the presence or absence of learning of each combination, and FIG. 17 is a learning adjustment constant for the amount of weight change by the backpropagation method. FIG. 18 is a flowchart of a processing example of the learning adjustment device when multiplying by, and FIG. 18 is a diagram illustrating an example of implementation of a neural network having a specific structure and learning speed adjustment. 24 …… Antecedent part membership function value calculation function, 25 …… Antecedent part calculation function, 26 …… Consequent part calculation function, 27 …… Consequent part membership function value output function, 28 …… Fuzzy inference value Calculation function, 30 …… Neural network, 31a, 31b ……
Learning adjustment device, 32 …… Learning adjustment constant setting device, 33a ……
Instruction reading unit, 34 ... Constant storage unit, 35 ... Weight change information reading unit, 36 ... Weight change amount adjustment unit.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Asahi Kawamura 1015 Kamiodanaka, Nakahara-ku, Kawasaki-shi, Kanagawa, Fujitsu Limited (72) Inventor Yuri Owada 1015, Kamiodanaka, Nakahara-ku, Kawasaki, Kanagawa Prefecture, Fujitsu Limited ( 72) Inventor Kazuo Asakawa 1015 Kamiodanaka, Nakahara-ku, Kawasaki City, Kanagawa Prefecture, Fujitsu Limited (72) Inventor, Shigenori Matsuoka, 1 Fujimachi, Hino City, Tokyo Fuji Facom Control Co., Ltd. (72) Inventor, Hiroyuki Okada Tokyo No. 1 Fuji-cho, Hino City, Fuji-Facom Control Co., Ltd.

Claims (3)

[Claims] 1. An actual weight change amount is obtained by multiplying the change amount of the weight obtained by the backpropagation method at the time of learning of the neural network for each of all the connections between the units forming the neural network. Learning holding constant holding means (1a, 1b, ...) For holding the multiplication constant of (1a, 1b, ...) And a weight changing amount determining means (2a, 2b ,. ..) are provided, and learning adjustment constant holding means (1a, 1b ,.
Learning to receive a learning adjustment constant setting instruction to one or more learning adjustment constant holding means of () and set the designated learning adjustment constant to the learning adjustment constant holding means corresponding to the instructed combination. A learning adjustment system for coupling in a neural network, characterized by comprising an adjustment constant setting means (3). 2. For each of all connections between units forming a neural network, whether or not the amount of change in weight obtained by the backpropagation method at the time of learning of the neural network is zero is "on".
Or a learning adjustment constant holding means (1a, 1b, ...) Holding an “off” flag and a weight change amount determining means (2a, 2b, ...) Which determines an actual weight change amount according to the learning adjustment constant. ,) And learning adjustment constant holding means (1a, 1b, ...
Learning to receive a learning adjustment constant setting instruction to one or more learning adjustment constant holding means of () and set the designated learning adjustment constant to the learning adjustment constant holding means corresponding to the instructed combination. A learning adjustment system for coupling in a neural network, characterized by comprising an adjustment constant setting means (3). 3. A means for setting an initial value of a weight to 0 for one or more connections among all the connections between units forming a neural network, and the learning adjustment corresponding to the one or more connections. The learning adjustment constant setting means (3) is used as a learning adjustment constant in the constant holding means, 0 in claim 1 and "off" in claim 2.
And a means for setting the learning adjustment constant according to the setting as a learning adjustment constant, which is 0 in claim 1 and "off" in claim 2.
3. The neural network according to claim 1 or 2, further comprising: means for holding the value, and the weight change amount determination means for the one or more connections continues to output 0 as the weight change amount during learning of the neural network. A learning adjustment method for connections in networks.