JPH03230285A - Excess neuron determination processing system - Google Patents

Abstract

PURPOSE:To increase an arithmetic processing speed and to reduce the number of hardwares by specifying unactivated excess neurons in network structure. CONSTITUTION:When a read distance from the zero value of an internal state value of input side internal connection is close to the zero value, the 1st excess neuron specifying means 33 determines that the neuron is an inactive excess neuron which does not respond to an input. When a read distance from the zero value of the internal state value of output side internal connection is close to the zero value, the neuron is determined as an inactive excess neuron which does not generate an output. An excess neuron output means 39 outputs the ID of the determined excess neuron. Consequently, which is the inactivated excess neuron in the network structure part 11 of a network constitution data processor 10 can be specified, the arithmetic processing speed can be increased and the number of hardwares can be reduced.

Description

[Detailed Description of the Invention] (Summary) The present invention relates to a network-structured data processing system that performs data processing according to a network structure formed by internal connections of neurons, and the present invention aims to enable determining inactive surplus neurons. Alternatively, a means of determining a neuron whose internal state value assigned to the internal connection of the output is close to zero as a surplus neuron, or an internal connection of the input and output among a group of neurons whose input and output edges have the same form. A method for determining, as surplus neurons, neurons other than one of the neuron groups whose internal state values are approximately the same, or determining neurons whose output values have small changes in response to the presentation of the input pattern group as surplus neurons. The neuron is configured to include means for determining the neuron.

[Field of Industrial Application] The present invention relates to a surplus neuron determination processing method for determining inactive surplus neurons in a network configuration data processing device that executes data processing according to a network structure constructed by internal connections of neurons. It is something.

Conventional sequential processing computers (Neumann type combinators) cannot adjust their data processing functions in response to changes in usage or environment. Adaptive data processing devices using a parallel distributed processing method using a data conversion function have been proposed. A data processing device with this network configuration uses input signals prepared for learning (
The weight values of the internal connections of the network structure are determined according to a predetermined learning algorithm so that the output signal (output pattern) from the network structure output in response to the presentation of the input pattern (input pattern) matches the teacher signal (teacher pattern). To go. It is assumed that the weight values are determined by this learning process.

Even if an input signal that was not specified is input, a “soft” data processing function will be realized in which a suitable output signal will be output from this network structure.

In a network configuration data processing device with such a configuration, in order to increase the calculation processing speed of the network structure and to reduce the amount of hardware in the network structure, it is necessary to remove unactivated redundant neurons in the network structure. It is necessary to prepare a means to identify which one it is.

[Conventional technology]

A conventional technique will be explained according to a network configuration data processing device having a hierarchical network configuration.

In a data processing device having a hierarchical network configuration, the hierarchical network is composed of a type of node called a basic unit (which is a type of neuron) and internal connections having weight values corresponding to internal state values. FIG. 12 shows the basic configuration of the basic unit 1. This basic unit 1 is a multi-car output system, and includes a multiplication processing section 2 that multiplies a plurality of inputs by weight values of respective internal connections; It includes an accumulation processing section 3 that adds up all the multiplication results, and a dark value processing section 4 that performs nonlinear dark value processing on this accumulated value and outputs one final output.

Assuming that hl is the first layer and iN is the second layer, this accumulation processing section 3 executes the calculation of the following equation (1), and the threshold processing section 4
Now, execute the calculation of equation (2) below.

X□=Σyi w +, (t) formula)' pi ”” 1 / (1+ exp(x pt
10t)) Formula (2) where, h: Unit number i of the h layer Unit number p of the 21st layer; Input signal pattern number θ, = Threshold value W+h of the 1st unit of the i layer F h Internal coupling between the i layers weight value X pi
j Sum of products of inputs from each unit of the h layer to the 1st unit of the i layer)'ph: 1 of the h layer for the input signal of the Pth pattern
Output from unit No. yp+: Output from No. 1 unit of i layer in response to input signal of P-th pattern.And in the hierarchical network configuration data processing function, a large number of basic units l with such a configuration accept input signal values as they are. The input unit 1, which is distributed and outputted, is used as an input layer and is connected hierarchically as shown in Fig. 13 to form a hierarchical network section, which converts the input pattern into the corresponding output pattern. It will demonstrate powerful data processing capabilities.

In a data processing device having a network configuration, it is necessary to obtain weight values of the network structure that define the data conversion function through a learning process.

As a learning processing method for weight values of a Plt layer network configuration data processing device, a learning processing method called a back propagation method is particularly attracting attention because of its high practicality.

In the back propagation method, the weight value Wi+ and the threshold value θ8 of the hierarchical network are adaptively and automatically adjusted and learned using error feedback. (1
) As is clear from equation (2), the weight value W1 and the threshold value θ
, must be performed at the same time, but this is a difficult task that interferes with each other. Therefore, as previously disclosed in Japanese Patent Application No. 1983-333484 (filed December 28, 1988, "Network Configuration Data Processing Bag N"), the applicant has proposed always “
By providing a unit that outputs the threshold value θi and assigns the threshold value θ1 as a weight value to the output, the threshold value θi
It was proposed that the threshold value W8 be incorporated into the weight value W8, and the threshold value W8 be treated as the weight value. By doing this, the above equations (1) and (2) become
pi ” 1 / (1+exp(-X pi))
(4) It will be represented by 2.

Next, a weight value learning processing method using the back propagation method will be explained according to the description format of equations (3) and (4). Here, this explanation will be given assuming that the hierarchical network section has a three-layered hierarchical network of layer h, layer 1, and layer jN as shown in FIG.

(3) By analogy with (4) 2, the following (5) (6)
The formula is obtained. That is,
3 layers of j from each unit of l! Sum of products y□ of inputs to the unit: Output from the third unit of the three layers for the input signal of the second pattern In the back propagation method, output pattern y1, and its output pattern y2. When the teacher pattern dpj (the teacher signal to the j-th unit of the j-layer with respect to the input signal of the second pattern) is determined, the output pattern Ypj and the teacher pattern d□ are determined.
Calculate the difference value (dp=-yp=) with
Then, ΔWH(t)=εΣαpj )' pi+ζΔWJi(
t-1) However, ε: Learning constant ζ: Momentum t: Update amount of weight values between 1FJ-j layers ΔW7+(
1) Calculate. Here, the reason for adding [ζΔW, (t-1)J, which is related to the update amount of the weight value determined in the previous update cycle, is to speed up the learning.

Next, the calculated α3. First, βpt
=ytt(1-yet)Σα□W,, (t-1) is calculated, and then ΔWib(t) = tΣβpi V ph+ζΔW,
According to k(t-1), the update amount ΔW of the weight value between layer h and jN, (1) is calculated.

Subsequently, the weight value Wjt(t) −WH(t−1)+ΔW j i (t
) Wib(t) = Wrh(t l)+ΔWrh(t
) by repeating the method to determine the output pattern ypj from the output layer that is output when the input pattern for learning is presented, and the teacher pattern that is the signal that the output pattern y and J should take. Processing is performed to learn the weight values W, , , W, , (including the threshold θ8.θ) with which d3, matches.

Then, the hierarchical network part consists of 8 layers - h layer - i layer - jl
When you have a hierarchical network with a four-layer structure called W,
First of all, γ□−Y ah(I V , h)Σβ,,w,,(t
-1), then ΔWb, (t)=εΣT ph y119+ζΔWh,
According to (t-t), the update amount ΔW of the weight value between the 8th layer and the h layer
hg(t), by determining the update amount ΔW of the weight value between the eyebrows on the previous stage side using the value found from the latter stage on the output side and the network output data, all the weight values can be calculated. It is processed to learn the threshold value.

[Problem to be solved by the invention]

However, conventionally, once the weight value of an internal connection is learned, data processing is performed using that weight value as is. In addition, when using a method of performing weight value learning processing on a large-scale computer and then porting the obtained weight values to actual hardware with a network structure, the weight values obtained by learning may be transferred as they are. It was being processed to enter data processing.

From now on, in the conventional technology, even if there are neurons that do not work effectively for data processing, data processing will be executed without taking this into consideration at all. The existence of such redundant neurons not only increases the amount of hardware, but also increases the calculation speed, without providing any advantage.

The present invention has been made in view of the above circumstances, and provides a means for identifying surplus neurons that are not activated in the network structure of a network structure data processing device. The objective is to provide a way to increase the processing speed of processing devices and reduce the amount of hardware.

[Means to solve the problem] FIG. 1 is a diagram showing the basic configuration of the present invention.

In the figure, 10 is a network configuration data processing device that executes adaptive data processing according to the network configuration, and 11 is a network configuration data processing device 10.
, which receives one or more inputs and an internal state value by which the inputs are to be multiplied to obtain a sum-of-products value, and converts this sum-of-products value by a predetermined function. 12 is an internal state value storage means included in the network configuration data processing device 10, which is configured by internal connections of neurons that calculate and output an output pattern corresponding to an input pattern. The structure unit 11 manages internal state values required when executing data conversion processing, and 20 is an internal state value learning device, which calculates the internal state values stored in the internal state value storage means 12 by learning processing. It is something.

Reference numeral 30 denotes a surplus neuron determining device, which includes a network structure information management means 31, an internal state II reading means 32, a first surplus neuron specifying means 33, a neuron group specifying means 34, and an identical function neuron specifying means 35. ,
The apparatus includes a second surplus neuron specifying means 36, an output range specifying means 37, a third surplus neuron specifying means 38, and a surplus neuron output means 39.

The network structure information management means 31 manages the network structure information of the network structure section 11, and the internal state value reading means 32 processes to read the internal state values stored in the internal state value storage means 12. The surplus neuron identifying means 33 determines whether either the distance from the zero value of the learned value of the internal state value assigned to the input or the distance from the zero value of the learned value of the internal state value assigned to the output is zero. The neuron group specifying means 34 identifies neurons having values close to the above values and processes the identified neurons to determine them as redundant neurons.
processes the network structure information management means 31 to identify a group of neurons whose input and output edges have the same form among the neurons of the network structure section 11;
The same function neuron specifying means 35 processes to specify, among the neuron groups specified by the neuron group specifying means 34, neuron groups whose corresponding internal state values assigned to inputs and outputs have approximately the same learned value, The second surplus neuron specifying means 36 processes neurons other than one of the neuron groups specified by the same function neuron specifying means 35 to determine them as surplus neurons, and the output range specifying means 37 processes Processing is performed to specify the maximum and minimum values of the output values output by each neuron in response to the presentation of the input pattern group, and the third surplus neuron specifying means 38 determines the maximum value specified by the output range specifying means 37. The surplus neuron output means 39 identifies neurons whose difference value between The surplus neuron specifying means 33, 36, and 38 of No. 3 perform processing to output the determined surplus neuron identifier.

[Operation] In the present invention, the internal state value learning device 20 presents a group of input patterns to be learned to the network structure section 11, and the network structure section 1 output according to this presentation.
The error value between the output pattern group from 1 and the teacher pattern group indicating the pattern that this output pattern group should take is calculated, and the internal state values of the internal state value storage means 12 are sequentially updated based on this error value. By doing so, the internal state value of the network structure unit 11 that realizes the desired data conversion process is learned and processed to be stored in the internal state value storage means 12.

In this way, when the internal state value of the network structure unit 11 that realizes the desired data conversion process is stored in the internal state value storage means 12, the internal state value reading means 32 manages the network structure information management means 31. One neuron is sequentially selected while referring to the information, and all internal state values assigned to the input-side internal connections of the selected neuron are read out from the internal state value storage means 12, and are assigned to the output-side internal connections. All internal state values are read out from the internal state value storage means 12.

Then, the first surplus neuron specifying means 33 determines that when the distance from the zero value of the read internal state value of the internal connection on the input side has a value close to zero value, the neuron is a non-responsive neuron that does not respond to the input. In addition to determining that the neuron is an active surplus neuron, if the distance from the zero value of the internal state value of the internal connection on the output side that has been read out has a value close to zero, the neuron is inactive and does not output any output. The surplus neuron is determined to be a surplus neuron, and the surplus neuron output means 39 outputs the identifier of the determined surplus neuron.

Further, when the internal state value of the network structure unit 11 that realizes the desired data conversion process is stored in the internal state value storage means 12, the neuron group specifying means 34 refers to the management information of the network structure information management means 31. This identifies a group of neurons in the network structure 11 whose input and output edges have the same shape. Subsequently, the internal state value reading means 32 reads out from the internal state value storage means 12 all internal state values assigned to the internal connections of inputs and outputs of each neuron in the specified neuron group. and,
The identical function neuron identifying means 35 identifies a neuron group in which each internal state value assigned to the input and output has approximately the same value among the neuron groups whose input and output edges have the same shape, according to the read internal state value. However, the second surplus neuron specifying means 36 determines that this group of neurons having approximately the same value can originally be substituted by one neuron, and that neurons other than one become inactive surplus neurons. Determine and surplus neuron output means 3
9 outputs the determined surplus neuron identifier.

Further, when the internal state value of the network structure section 11 that realizes the desired data conversion process is stored in the internal state value storage means 12, the output range specifying means 37 refers to the management information of the network structure information management means 31. By sequentially selecting one neuron and referring to the network output data of the network structure unit 11 that is output in response to the presentation of the input pattern group to be learned, the selected neuron can display the input pattern group to be learned. Specify the maximum and minimum values of the output value to be output for. Then, when the difference value between the specified maximum value and minimum value has a value close to zero, the third surplus neuron specifying means 38
The neuron is determined to be an inactive surplus neuron that is not performing any effective operation, and the surplus neuron output means 39 outputs the identifier of the determined surplus neuron.

In this way, according to the present invention, it becomes possible to specify whether there are redundant neurons that are not activated in the network structure section 11 of the network configuration data processing device 10.

〔Example〕

Hereinafter, the present invention will be described in detail according to an embodiment applied to a hierarchical network configuration data processing device.

FIG. 2 illustrates an embodiment of the present invention. In the figures, the same parts as explained in FIGS. 1, 12, and 13 are indicated by the same symbols. 10a is a hierarchical network configuration data processing device that performs adaptive data processing according to the hierarchical network configuration; lla is a hierarchical network structure unit having the structure as explained in FIG. 13;
20.a is a weight value storage means corresponding to the internal state value storage means 12 in FIG. Reference character a is a weight value learning device that executes learning of weight values of the hierarchical network structure section 11a according to the back propagation method, and 30a is a surplus unit determining device corresponding to the surplus neuron determining device 30 in FIG.

The weight value learning device 20a includes a learning signal storage means 21, a learning signal presentation means 22, a learning convergence determination means 23, and a weight value learning means 24. The learning signal storage means 21 stores learning signals used for learning the weight values, and the learning signal presentation means 22 reads out the learning signals from the learning signal storage means 21 and arranges the learning presentation signals in the hierarchy. The learning teacher signal is presented to the network structure unit 11a, and the other learning teacher signal forming the pair is processed to be presented to the learning convergence determining means 23 and the weight value learning means 24, and the learning convergence determining means 2
3 receives the output signal output from the hierarchical network structure section 11a and the learning teacher signal from the learning signal presentation means 22, and determines whether the error of the data processing function of the hierarchical network structure section 11a is within an allowable range. The weight value learning means 24 receives the learning teacher signal from the learning signal presenting means 22 and the network output data of the hierarchical network structure section 11a. , the update amount of the weight value is calculated according to the back propagation method, and the weight value is updated according to the update amount, thereby performing learning so that the weight value converges.

And surplus unit determination bag M 30 characteristic of the present invention
When learning of weight values is completed by the weight value learning device 20a, a is a first surplus unit specifying means which is constituted by a program means and processes to specify inactive basic units 1 in the hierarchical network structure section 11a. 301, 2nd
The surplus unit identifying means 302, the third surplus unit identifying means 303, and the fourth surplus unit identifying means 304 are provided, and these surplus unit identifying means 301
．． Judgment data storage means 305 that manages the judgment value T and distance function f(α,) required by 302, 303, and 304
and these surplus unit identification means 301, 302, 3
03 and 304 to an external device. In Figure 3, the first
4 is a flowchart executed by the second surplus unit identification means 302, FIG. 5 is a flowchart executed by the third surplus unit identification means 303, and FIG. In Figure 6,
A flowchart executed by the fourth surplus unit identifying means 304 is illustrated.

Next, in accordance with the flowcharts shown in FIGS. 3 to 6, the process of identifying surplus basic units 1 will be executed on the condition that learning of weight values is completed by the weight value learning device 20a. Explain in detail.

As shown in the flowchart of FIG. 3, the first surplus unit specifying means 301 selects one basic unit 1 whose activation or inactivity has not been determined in step 30, and then, in step 31, Learning value α+ of weight values assigned to all internal connections output from the basic unit l
(1≦i≦n) is read from the weight value storage means 12a.

Next, in step 32, the judgment value γ assigned to the basic unit 1 and the distance function r(α,) (which can be set for each basic unit or the number of stages in the hierarchical structure) Then, in step 33, the weight value read in step 31 (forming an n-dimensional space) is substituted into the read distance function f(α,). Calculate the distance K of the weight value from the origin. Here, the distance function r(α,
) can be defined in various ways, for example, f(
αI) = (Σα, J l/! and f (α+) = MAX l a, l However, l≦i
It is calculated according to the following functional expressions: ≦n and r(α,)=Σ α.

In this way, when the distance MK from the origin in the n-dimensional weight space of the weight value assigned to the internal connection on the output side of the selected basic unit 1 is determined, in the following step 34, this distance K is read out in step 32. It is determined whether or not the value is smaller than the determined judgment value γ. If it is determined that the basic unit 1 is smaller than the judgment value T, the process proceeds to step 35 and the selected basic unit 1 is determined to be a surplus unit, and if it is judged that the selected basic unit 1 is larger than the judgment value T, the process proceeds to step 36 to select the It is determined that the basic unit l is not a surplus unit, and in the next step 37, the result output means 306 is notified of the determination result. Then, when it is detected in step 38 that activation or inactivation has not been determined for all the basic units 1, the process returns to step 30, whereby all the basic units of the hierarchical network structure section 11a UNISO) Executes the active/inactive determination for l.

In this way, the first surplus unit specifying means 301
When the learning value of all internal state values of the internal connections on the output side of basic unit l has a value close to zero, the basic unit l is processed to be determined to be an inactive surplus unit that does not output output. It is something to do.

On the other hand, the second surplus unit specifying means 302
By executing the flowchart in Fig. 4, when the learning value of all internal state values of the internal connections at the input end of basic unit 1 has a value close to zero, that basic unit l becomes an inactive state that does not accept input. Processing is performed to determine that the unit is a surplus unit. The difference between the flowchart in FIG. 4 and the flowchart in FIG. 3 is that in step 31 of the flowchart in FIG. In contrast to reading α1, step 41 of the flowchart in FIG. Note that the learning value of the threshold value of basic unit 1 does not need to be included in the learning value of the weight value to be read.

As shown in the flowchart of FIG. 5, the third surplus unit specifying means 303 determines, in step 50, a group of basic units l having the same weight space, such as a group of basic units 1 of the second intermediate layer. Select one group from (groups that are internally connected to the same group of basic units). Next, in step 51, the judgment value T assigned to that group and the distance function f(α
,.

α°, ) (which can be set for each group) is read from the judgment data storage means 305. Next, in step 52, one basic unit l is selected as a reference unit from among the basic units 1 of the selected group, and in subsequent step 53, the selected reference unit is determined to be an activated non-redundant unit. .

Next, in step 54, the undetermined and unselected basic unit 1
Select one basic unit 1 from among the following, followed by step 5
5, the weight value αi (forms an n-dimensional space) of the reference unit read from the weight value storage means 12a and the weight value α°, (forms an n-dimensional space) of this selected basic unit l.
By substituting this into the read distance function f (α4.α°,), the distance RK between the reference unit and the basic unit 1 selected in step 54 is calculated. Here, although it is preferable to include the threshold value in the weight value, it may not be included since the degree of contribution is small.

Furthermore, the distance function f(αpositive, α“i) can be defined in various ways, for example, f(α8.α,)=[Σ
(α, −α,)g] I/! Yar ((rt, lr'i) = MAX I α1-r
x', 1 However, 1≦i≦n and f(α3.α,)=Σ α, −α.

It will be calculated according to the functional formula.

Once the distance K between the reference unit and the basic unit 1 selected in step 54 is determined, in the subsequent step 56 it is determined whether this distance K is smaller than the judgment value γ read out in step 51. If it is determined that the value is smaller than the judgment value T, the process proceeds to step 57, and the selected basic unit 1 has the same function as the reference unit (a function that can be omitted by increasing the weight value of the reference unit). )
On the other hand, when it is determined that the unit is larger than the judgment value T, nothing is determined and the unit is left undetermined. Then, in the following step 58, it is determined whether all of the undetermined basic units 1 have been selected, and if there are any unselected basic units, the process returns to step 54.

When it is determined in step 5 that all have been selected, in the following step 59, it is determined whether or not undetermined basic units 7) 1 remain among the basic units l of the group selected in step 50. If the selected reference unit remains, the process returns to step 52 and selects a reference unit from among the remaining ones that performs the same function as the selected reference unit. Processing is performed to check whether there is a surplus basic unit l. When it is determined in step 59 that there are no undetermined basic units 1 left, the process proceeds to step 60 and it is determined whether all groups of basic units 1 having the same weight space have been processed. , when it is detected that the processing has not been carried out, the process returns to step 50, so that the hierarchical network structure unit 11a
Activation or inactivation is determined for all basic units l of , and the result output means 306 is notified of the determination result in step 61 .

In this way, the third surplus unit identifying means 303 groups the basic units 1 of the hierarchical network structure 11a that perform the same function, and selects only one of the grouped basic units l. is determined to be not a surplus unit, and the remaining basic unit 1 is determined to be a surplus unit. The basic unit 1 that is determined to be surplus by this third surplus unit specifying means 303 can be omitted by setting the weight value of the representative non-surplus basic unit 1 to be larger accordingly. becomes.

As shown in the flowchart of FIG. 6, the fourth surplus unit specifying means 304 selects one basic unit 1 whose activation or inactivity has not been determined in step 70, and then, in step 71, The judgment value T assigned to the basic unit 7) 1 (which can be set for each basic unit l or the number of stages in the hierarchical structure) is read from the judgment data storage means 305. Then, in step 72,
Instructs the learning signal presentation means 22 to present a learning presentation signal to the hierarchical network structure section 11a, and also outputs the maximum and minimum values of the output values from the selected basic unit 1 in response to this instruction. Process to find.

Then, in the following step 73, it is determined whether the difference value between the maximum value and the minimum value obtained is smaller than the judgment value T read in step 7 step 71, and when it is judged that it is smaller than the judgment value γ, Proceeding to step 74, it is determined that the selected basic unit l is a surplus unit, and the judgment (
! When it is determined that it is larger than r, the process proceeds to step 75 and determines that the selected basic unit 1 is not a surplus unit, and in the next step 76, the result output means 306 is notified of the determination result.

Then, when it is detected in step 1'7 that activation or inactivation has not been determined for all basic units 1, the process returns to step 70.
The activation or inactivation determination is performed for all basic units 1 of the hierarchical network structure section 11a.

In this way, the fourth surplus unit identifying means 304 identifies the basic unit 1, which hardly changes its output value with respect to the input pattern input to the hierarchical network structure section 11a, as an inactive surplus unit that does not contribute to data processing. Processing is performed to determine that it is a unit.

Next, the surplus unit identification process will be specifically explained based on simulation. In this simulation, the hierarchical network structure section 11a has a hierarchical network structure consisting of an input layer of 2 units, a one-stage intermediate layer of 9 units, and an output layer of 1 unit, as shown in FIG. The device 20a uses the 20 learning signals shown in FIG. 8 and sets the learning constant ε to 5.0. Set momentum ζ to 0.4 and change weight value and darkness value from -0,5 to 0.
．． Initialize with a random number up to 5 and present the learning signal 5490 times (at this time, the difference between the learning teacher signal and the output signal is 0.02
5 or less).

FIG. 9(a) shows the learned value of the weight value obtained by this simulation, and FIG. 9(b) shows the learned value of the dark value obtained by this simulation. Here, in FIG. 9(a), "0.548750°.

9(b), "0,049658" indicates the dark value of the riO unit J. It shows.

When such a weight value is obtained, the first surplus unit specifying means 301 determines that the weight value assigned to the internal connection on the output side of the "i4 unit" of the intermediate layer is "-0,01451".
3°° and zero (close to direct), this basic unit l is identified as a surplus unit.Then, the weight value assigned to the internal connection on the output side of basic unit 1 of the other hidden layer is Since it is relatively large, it is identified as an activated unit that is not redundant.

FIG. 1O shows a plot of the weight values (forming a two-dimensional weight space) assigned to the internal connections on the input side of the basic unit l of the intermediate layer in FIG. 9(a).

When such a weight space is obtained, the second surplus unit specifying means 302 determines the intermediate layer's ri4 unit' and ri8 unit.
Since the distance from the origin of the weight value assigned to the internal connection on the input side of "Unit" is small, these basic units 1 are identified as surplus units. and,
Since the distance from the origin of the weight value assigned to the internal connection on the input side of the basic unit 1 of the other intermediate layer is relatively large, it is specified as an activated unit that is not redundant.

Further, the third surplus unit specifying means 303
As can be seen from Figure 0, when the dark value is not considered, the intermediate layer ri2 unit and ri6 unisoto perform roughly the same function, and the ri4 unit and ri3 unit perform roughly the same function. Therefore, ri5
``Unit'' and ``18 Unit J'' are identified as surplus units.

FIG. 11 illustrates the output value of each basic unit ()l of the intermediate layer that is output for each of the 20 learning signals when the learned weight values and darkness values are set.

As can be seen from Fig. 11, the middle layer ``12 units'', ``i4 unit J and ri6 unit'', and r7B
Since these basic units do not change the output value, the fourth surplus unit specifying means 304
will be identified as a surplus unit.

Although the illustrated embodiment has been described above, the present invention is not limited thereto.For example, the present invention is not limited to a hierarchical network configuration data processing device, but can be applied as is to other network configuration data processing devices. It can be done. Further, the neuron to which the present invention is applied is not limited to the basic unit that performs threshold conversion processing as described in the embodiments, but may be one that performs other function conversion processing.

〔Effect of the invention〕

As explained above, according to the present invention, in a network configuration data processing device, it becomes possible to identify which neurons are redundant and not activated in the network structure. There is no need to prepare an unnecessarily large amount of data. From now on, it will be possible to increase the arithmetic processing speed of the network configuration data processing device and to reduce the amount of hardware.

[Brief explanation of drawings]

FIG. 1 is a diagram showing the principle configuration of the present invention, FIG. 2 is an embodiment of the present invention, FIG. 3 is a flowchart executed by the first surplus unit specifying means, and FIG. 4 is a flow chart of the second surplus unit specifying means. FIG. 5 is a flowchart executed by the third surplus unit specifying means; FIG. 6 is a flowchart executed by the fourth surplus unit specifying means; FIG. 7 is an explanation of the hierarchical network structure used in the simulation. Figure 8 is an explanatory diagram of the learning signal used in the simulation, Figures 9, 10, and 11 are explanatory diagrams of the simulation results, Figure 12 is a basic configuration diagram of the basic unit, and Figure 13 is a hierarchical diagram. The network part is d杢1imfilQ. In the figure, 1 is a basic unit, 10 is a network configuration data processing device, loa is a hierarchical network configuration data processing device, 11 is a network structure section, 11a is a hierarchical network structure section, 12 is an internal state value storage means, and 12a is a weight value storage means; 20 is an internal state value learning device; 20a is a weight value learning device; 30 is a surplus neuron determining device; 30a
is a surplus unit determination device.

Claims (3)

[Claims] (1) Receiving one or more inputs and an internal state value by which the inputs are to be multiplied to obtain a sum of products value, and
A network structure unit (11) configured by internal connections of neurons that converts the product-sum value by a predetermined function to obtain an output value, and calculates an output pattern corresponding to the input pattern according to the network structure unit (11). a network configuration data processing device (10) that outputs a group of input patterns to be learned;
11) and calculate the error value between the output pattern group from the network structure unit (11) that is output according to the presentation and the teacher pattern group that indicates the pattern that the output pattern group should take, An internal state value learning device (2) that learns an internal state value for which the error value falls within an allowable range by sequentially updating the internal state value based on the error value.
0), either one of the distance from the zero value of the learned value of the internal state value assigned to the input, or the distance from the zero value of the learned value of the internal state value assigned to the output. Identify the neurons whose distance is close to zero, and
A surplus neuron determination processing method, comprising: surplus neuron specifying means (33) for determining the identified neuron as a surplus neuron. (2) receiving one or more inputs and an internal state value by which the inputs are to be multiplied to obtain a product-sum value;
A network structure unit (11) configured by internal connections of neurons that converts the product-sum value by a predetermined function to obtain an output value, and calculates an output pattern corresponding to the input pattern according to the network structure unit (11). a network configuration data processing device (10) that outputs a group of input patterns to be learned;
11) and calculate the error value between the output pattern group from the network structure unit (11) that is output according to the presentation and the teacher pattern group that indicates the pattern that the output pattern group should take, An internal state value learning device (2) that learns an internal state value for which the error value falls within an allowable range by sequentially updating the internal state value based on the error value.
0), a neuron group specifying means (34) for specifying a neuron group in which input and output edges of the neurons have the same shape;
), and identical function neuron identifying means (34) for identifying neuron groups whose learning values of corresponding internal state values assigned to inputs and outputs take approximately the same value among the neuron groups identified by the neuron group identifying means (34). 35), and surplus neuron identification means (36) for determining, as surplus neurons, neurons other than one of the neuron groups identified by the same function neuron identification means (35).
A surplus neuron determination processing method characterized by comprising: (3) receiving one or more inputs and an internal state value by which the inputs are to be multiplied to obtain a product-sum value;
A network structure unit (11) configured by internal connections of neurons that converts the product-sum value by a predetermined function to obtain an output value, and calculates an output pattern corresponding to the input pattern according to the network structure unit (11). a network configuration data processing device (10) that outputs a group of input patterns to be learned;
11) and calculate the error value between the output pattern group from the network structure unit (11) that is output according to the presentation and the teacher pattern group that indicates the pattern that the output pattern group should take, An internal state value learning device (2) that learns an internal state value for which the error value falls within an allowable range by sequentially updating the internal state value based on the error value.
0) in a network configuration data processing system comprising: an output range that specifies the maximum and minimum values of output values output by each neuron in response to presentation of a group of input patterns to be learned, according to learned internal state values; Specific means (3
7) and specifying the neuron whose difference value between the maximum value and the minimum value specified by the output range specifying means (37) is close to zero, and determining the specified neuron as a surplus neuron. Means for identifying surplus neurons (3)
6) A redundant neuron determination processing method.