JPH0451382A - Information processor - Google Patents

Abstract

PURPOSE:To facilitate a wiring between vector/matrix multipliers by arranging the vector/matrix multipliers having multiple input lines and output lines on a semiconductor chip so that the input lines face the output lines. CONSTITUTION:N1-number of the neuron output values of a first layer are inputted to the vector/matrix multiplier in parallel. N2-number of the neuron output values of a second layer, which are obtained as the output of VMM 1, are transmitted to the input line of the vector/matrix multiplier VMM 2 through n2-number of wirings. Then, n3-number of the neuron output values of a third layer, which are obtained from VMM 2 are outputted. Since an output part and an input part are arranged so that they face each other, they can be wired at a short distance. A trouble due to the increase of noise, the drop of a signal voltage by the resistance of the wiring or a signal delay by the wiring can be avoided.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to an information processing apparatus for realizing large-scale, high-speed parallel distributed processing, and particularly to an information processing apparatus suitable for neural network information processing.

[Prior Art] Parallel 11-part information processing using neural networks (hereinafter referred to as neural network information processing) called neurocomputing is described in Complex Systems 1 (1,987), pp. 145 to 168. (Se
jnowski, T. J.

and Ilosenberg, C.R01987,
Parallelncworks that 1ea
rn to pronounce English8ht,
ex+,, Complex Systems l, p
p, ]] 45-168 or Neural Network Information Processing (Sangyo Tosho, written by Hideki Aso), it is attracting attention in the fields of audio and image processing. In neural network information processing,
A large number of computing elements called neurons connected in a network perform sophisticated information processing by exchanging information through transmission lines called connections. Each neuron performs simple operations such as product or sum on information sent from other neurons (Neelon output value). It is possible to perform calculations within each neuron and even calculations in different neurons in parallel, so in principle it has the advantage of being able to process information at high speed. Also, Nature 323-9, (August 1986)
) pages 533 to 535 (Rumelbart,
D, E, .

ini, on, G, E,, and William
ms, R.J., 1986a.

Learning representations
by backpropagation errors
, Nature323-9. pp, 533536)
, or as described in Chapter 2 of Neural Network Information Processing (Sangyo Tosho, written by Hideki Aso),
Algorithms (learning) have also been proposed that set weight values for connections between neurons in order to perform desired information processing, so it is possible to perform various information processing depending on the purpose.

The operating principle of neural networks will be explained with reference to two typical types of networks, a hierarchical network and a Hopfield network. FIG. 2(a) shows the structure of a hierarchical network, and FIG. 3(a) shows the structure of a Hopfield type network. Both of these are composed of neurons and connections connecting neurons. Here we use the term neuron, or in some cases nodes, or 1fli
It is also sometimes called a calculation element. The direction of the connection arrow indicates the direction in which the neuron output value is transmitted. In a hierarchical network, as shown in FIG. 2(a), neurons are arranged in a plurality of layers, and neuron output values are transmitted only in the direction from the input layer to the output layer. On the other hand, in the Hopfield type network, as shown in FIG. 3(a), neuron output values are fed back to the same neuron, and neuron output values are transmitted in both directions between any two neurons.

Figures 2(b) and 3(b) illustrate the principles of calculations performed within neurons. Since the principle of calculation is the same for both networks, the hierarchical network will be explained using FIG. 2(b).

FIG. 2(b) is an enlarged view of the first niniron in the S+1th layer. In this neuron, the output values of neurons in the previous layer, that is, the 8th and 11th layers, are connected through connections.V.S11.・VI'-・,,,Vnss
is input. Here, ns indicates the number of neurons in the layer 8 and 1.3, and within the neuron, the input neuron output values V, S,.

1. V's, , , Vns s and the joint weight value Go""l
l.

+1. ”'JI, , , , '1'a II
Product with s V, ! ,T'A, ,,VISTSJ
+, , , Vns 5TSJ ns is calculated by the multiplier MT. Next, these products and 10sho +
The sum with 1 is calculated by the adder ADD. Here, O.J.
S+1 is a quantity called an offset and may be omitted in some cases. Furthermore, the result is input to a nonlinear function circuit to obtain the neuron output value VJS+1. The nonlinear function circuit has characteristics as shown in FIG. 2(C) or FIG. 2(d), and outputs an output g(x) in response to an input X. 1 Figure 2 (c) shows that input X or a certain threshold value x 1, l
] is an example of a nonlinear function that outputs a binary output g1 or g2 depending on whether or not it exceeds .

One nonlinear function circuit may have other characteristics as necessary. Further, depending on the case, linear characteristics may be provided.

The principle of the calculation described in item 1 is the same in the Hopfield network as shown in FIG. 3(l). However, in the Hopfield-type network, the output values of all neurons except itself are input to the neurons in the first layer 111j.3 As can be seen from Figures 2(a) and (b) In a hierarchical network, first, the output value of the neuron in the input layer is set, then the output value of the neuron in the hidden layer is updated one after another based on that, and finally the output value of the neuron in the output layer is updated. be done. On the other hand, in the Hopfield network as shown in FIG. 3(a), since there are no layers, each neuron can update its output value at an appropriate timing. In this Hopfield type network,
Appropriate initial values are given to all neuron output values, and the neuron output values continue to be updated until the neuron output values reach an equilibrium state. It is usually necessary to update the output values of all neurons several times before the output values of the neurons reach an equilibrium state. A network in which the output values of all neurons are updated simultaneously is called a synchronous Hopfield network, and a network in which each neuron updates its output value at its own timing is called an asynchronous Hopfield network.

A method using software and a method using hardware have been used to perform neural network operations such as those described above. In the method using software, the neuron calculations are performed by a program written in a computer language, so the structure of the renetwork with the number of neurons can be easily changed depending on the purpose.

However, since calculations are performed sequentially, increasing the number of neurons has the disadvantage that information processing time increases rapidly. For example, in a Hopfield network using n neurons, the product must be calculated n times to update the output value of one two-neuron. Therefore, in order to update the output values of all neurons at least once, it is necessary to calculate the product [)' times.

In other words, as the number of neurons increases, the amount of calculation increases to II
' increases with the order of '. As a result, if multiplication is performed sequentially, the information processing time also increases on the order of n2.

In the method using hardware, even if the calculations are performed sequentially, compared to the method using software, the time required for each calculation can be reduced [I+J], so that higher speed can be expected. Attempts have also been made to further increase speed by performing operations in parallel.

For example, I, E, E, E, Journal of Solid State Circuits, Volume 25, Number] (February 1990), pp. 207-214 (I F to] E, , 1011 r n )l lo
i' 5olid 5tate Circuit1. S, vol, 25. No, ].

(Feb,], 990) pp 207-21.
4), neural network operations can be performed at high speed by vector/matrix multipliers using two-dimensional arrays.

FIGS. 4(a) and 4(b) show the principle of the vector/matrix multiplier shown in FIG. 1 of Reference 1.

The vector/matrix multiplier VMM shown in FIG. 4(a) has a horizontal input line, a vertical summation line, and a resistor R1l that connects them.

Rmn, and current-voltage conversion amplifier apl.

ap2.1. , apm. When input voltages VI l, , , , Vln are simultaneously applied to the input lines, current flows into the summation line through the resistors R11, 0, , Rmn. At this time, the current Ij flowing through the summation line is expressed as J-Σ,VTi.(1/Rji) (]). Here, J-'+ 2+ , , m, and the sum is taken from i=1 to n. Output voltage VOI,.

,, VOm is the characteristic FI I, 0 of the amplifier apl, , , apm that performs current-voltage conversion provided at the end of the summation line.
．． , 21rn, VOj = aj
It is expressed as (Ij). As can be seen from equation (1,), the relationship between the input terminal and the output current is (b-1) in Figure 4 (1)).
It can be expressed as a product of a vector and a matrix as shown in the equation. For this reason, the circuit of FIG. 4(a) is called a vector/matrix multiplier.

'' - From formulas (1) and (2) and the formula shown in Figure 2(b), in Figure 4(a), the resistance Rji is set to a value inversely proportional to the coupling weight TJ''l, and the amplifier's After adjusting the input/output characteristics of the characteristics a1., 1..a, and m to the nonlinear function g in Fig. 2, the voltage proportional to the neuron output value of the S-th layer is set as the input voltage Vll, ..., Vln, lj ,
Then, the output voltage VOI, , , VO+n is the s+th
It can be seen that a voltage proportional to the neuron output value of the first layer can be obtained. In addition, the offset θJ! , ++,
This can be easily achieved by setting the characteristics of each amplifier to shift by a value corresponding to θJ5+1.

Although a hierarchical network has been described here, it can also be applied to a synchronous Hopfield network. In other words, the number n of input lines and the number m of summation lines are set equal,
After setting the resistance RJl to a value inversely proportional to the coupling weight 'Jゞ, the input voltage Vl l, , .

, Vln are set to values proportional to the neuron output values Vl, . . . , Vn, the output voltage VOI, .

An updated value of the output value of each neuron is obtained as VOm. Note that in the normal Hopfield type, no coupling to itself is provided, so in that case, the resistance Rij corresponding to the diagonal component of the matrix may be omitted.

From the above, if a vector/matrix multiplier using a two-dimensional array is used, the output value of neurons for one layer in a hierarchical neural network I network, or the updated value of the output values of all neurons in a Hopfield neural network. It can be seen that both can be obtained at the same time.

In order to simplify the explanation, we have cited an example in which the coupling weight is expressed as a fixed resistance, but there are also other examples, such as -1- listed in I, E, E, E, Journal of Solid State Circuits, Vol. 25. , number 1 (
February 1990) pages 207 to 214 (iEEE
.

, Journal of 5solidStatCC
1rcui1. s, v○J, 25°No, ], (
Feb. 1990) pp. 207214) describes a method of storing coupling weights as voltages in capacitors.

According to this method, the connection weights can be rewritten as necessary. In addition, various other configuration methods have been reported so far.

[Problems to be Solved by the Invention] As described above, the vector/matrix multiplier has the advantage of being able to simultaneously obtain the horn values of a large number of neurons. However, since conventional neural networks have been targeted at relatively small-scale networks, there has been no consideration given to the placement of wiring to make it easier to reduce the size of the network. Therefore, when attempting to scale up neural networks, the following problems arise: Became.

As an example, consider a case where a three-layer network is configured, each layer consisting of 1000 new rons. Since it is 3 layers, two vector/matrix multipliers are required1.If these are stored in separate packages, the output values of the 1,000 second layer neurons obtained by one vector/matrix multiplier must be transmitted to the other vector/matrix multiplier. Therefore, in order to perform parallel processing, at least 1000 or more wires must be provided between packages. However, it is usually not possible to provide 1000 bottles in one package.

Therefore, if each vector and matrix multiplier is housed in a separate chip, the number of bins becomes a bottleneck, making it difficult to construct a large-scale network. On the other hand, it is also conceivable to provide a number of vector/matrix multipliers in one chip.

For example, I, Varnish, Varnish, Sea, Sea, Digest 90, page 142.143.284 (ISSCC
, Digest: of゛I″Cchnical
PapCrs(FOb.

1990) pp142. 143. and 2
84) describes an example in which a three-layer network in which the first layer is 32 euros and the second and third layers are 16 euros is integrated on one chip. In this conventional example, since the number of neurons is small, the area other than the vector/matrix multiplier can be used relatively freely as the wiring area. Therefore, a plurality of vector/matrix multipliers can be easily integrated into one chip. However, the number 100. When integrating vector/matrix multipliers that calculate neuron output values in the order of several thousand on one chip, the number of
000 wires are required, making the layout extremely difficult. Even if this were possible, it is expected that problems would occur such as an increase in chip area, an increase in noise, a drop in signal voltage due to the resistance component of the wiring, or a signal delay due to the wiring. Another solution is I, Nis, Nis, Sea, Digest 90, No. 144.145, 285.
Page (ISSCC, Digest of'Techni
cal Papers (Feb.

] 990) pY) 144, 145,
+-1, rld 285), there is also a limitation on the number of wires between the vector and matrix multiplication nodes (128 in the I-2 example). However, since this would omit some of the connections between neurons, there is a risk that the capabilities of network i-work may not be fully utilized.

On the other hand, a Hopfield type network can, in principle, be constructed from a single vector/matrix multiplier by wiring the input section and output section of the vector/matrix multiplier. However, when the number of neurons is large, problems similar to the hierarchical type occur. That is, as the number of neurons increases, the area occupied by the wiring region increases because the wiring is routed from the input section to the output section. Furthermore, the wiring length becomes longer, which increases noise, and the resistance component of the wiring causes a lower voltage level.
・ ゛ or signal delay due to wiring becomes a problem. As described above, in a Hopfield type network, it is necessary to update the neuron output value many times before reaching a steady state. Therefore, if the signal delay per cycle is large, the time required to reach a steady state will increase considerably.

In order to realize a large-scale neural network information processing device using vector/matrix multipliers as described above, it was necessary to solve the wiring problem.

A twenty-first object of the present invention is to facilitate wiring between arithmetic units or especially vector/matrix multipliers in an information processing device that performs large-scale 112-column arithmetic processing, or in particular in a neural network information processing device. It is to make it.

[Means for Solving the Problems] An information processing apparatus of the present invention for achieving the objects set forth in
An information processing device having a plurality of input lines and a plurality of output lines, and configured using a plurality of arithmetic units that input information from the plurality of input lines and obtain outputs in parallel to the plurality of output lines. At least two 'el' J units are characterized in that the input ff1j of one arithmetic unit and the output part of the other arithmetic unit are arranged to face each other.

In this case, the -I-2 arithmetic unit may perform a sum-of-products operation necessary for neural network information processing.

[Function] As mentioned above, two arithmetic units or vector/matrix multipliers that perform continuous arithmetic processing are connected to the input section and the output ffl+
By arranging them so that they face each other, wiring between arithmetic units or vector/matrix multipliers becomes easier. Is this about it?
It is possible to constantly transmit many signals in parallel between arithmetic units or vector/matrix multipliers, making it possible to realize large-scale operations such as neural networks.

As mentioned above, in a large-scale neural network, the number of wires between vector/matrix multipliers can reach several thousand. This number of wires is much larger than that between the control section and the vector I/le/matrix multiplier. Therefore, the present invention aims to prioritize wiring between vector/matrix multipliers to facilitate downsizing, and the input section and output section of the vector/matrix multiplier are arranged opposite to each other.
3. This leads to a reduction in area and complexity in large-scale integration. Needless to say, this will lead to a decrease in IIQ! of delay.

U Embodiment FIG. 1 shows an embodiment of the present invention, which is an information processing device in which a hierarchical neural network consisting of three layers is integrated on a semiconductor chip L-3, and the number of neurons in the first layer is n.
1, the number of neurons in the second layer is n2, and the number of neurons in the third layer is n3. A vector/matrix multiplier VMMl is used to calculate the neuron output value of the second layer from the neuron output value of the first layer. An I-le matrix multiplier VMM2 is illustrated.

In this embodiment, first, n1 neuron output values of the first layer are input in parallel to the vector/matrix multiplier VMM.

The n2 second layer neuron output values obtained as the outputs of the vector I/matrix multiplier VMMIQ) are transmitted to the input line of the vector I/matrix multiplier VMM2 through n2 wires. Subsequently, the n', three neuron output values of the third layer obtained from the vector/matrix multiplier VMM2 are output. In this embodiment, vector/matrix multiplier V M
M l and VMM 2 are connected to the output of VMMI and V
M M 20') The input parts are arranged so as to face each other 4. Therefore, it is possible to easily wire between them over a short distance, which increases noise and increases the resistance component of the wiring. This makes it possible to avoid problems caused by low voltage levels or signal 8' delays due to wiring. Furthermore, since the area occupied by the wiring can be reduced, an increase in chip area can also be suppressed. Therefore, a three-layer neural network consisting of many neurons can be integrated on a semiconductor chip.

Although the above embodiment was for a three-layer neural network, the present invention can be easily extended to networks with any number of layers. This is a second embodiment of the invention.

In this embodiment, four vector/matrix multipliers are arranged so that their input sections and output sections face each other. In the figure, the number of wires nl, n2. n3. n/l and n5 are the first layer, the second layer, the third layer, the 711th layer, the 5th layer, and the 711th layer, respectively.
2 equal to the number of neurons in the layer, and in this embodiment also the vector/matrix multipliers VMM l and VMM2, VMM2 and V
M M : 3, it is possible to easily wire between VMM3 and VMM4 with a short wire. Therefore, the number of wires can be increased, and a neural network consisting of many neurons can be integrated into the conductor chip 1'. In this way, a network with an arbitrary number of layers can be realized by arranging the Begg matrix multipliers so that the input section and the output section face each other.

FIG. 6 is a third embodiment of the present invention, which is an example of a Hopfield type neural network. In this embodiment, four vector/matrix multipliers are arranged so that their input sections and output sections face each other, and the output line of the vector/matrix multiplier VMM4 is connected to the input line of VMMI.

Vector/matrix multipliers VMMI, VMM2. VMM3.
The same one is used for the VMM4, and it is connected with wiring equal to the total number of neurons n.

In this embodiment, first, the initial values of the output values of all n neurons are input to the vector/matrix multiplier VMM l in the -1 column. Next, the input signal and the vector/matrix multiplier VM
M], the neuron output value settles to a steady value while being transmitted between the four vector/matrix multipliers3.After that, the steady value of the neuron output value can be read out to the outside of the chip11. In the embodiment, the four vector/matrix multipliers are arranged so that the input section and the output section face each other, so that the wiring length between them can be shortened. For this reason,
Compared to the conventional case where a single vector/matrix multiplier is used and its input and output are connected with long wiring, problems such as noise, signal voltage drop due to wiring resistance components, and signal delay problems are reduced. Ru.

In the embodiments of FIGS. 1, 5, and 6 described above, the numbers of input lines and output lines of the VEG I-matrix multiplier that perform continuous processing are the same. Therefore, if the input line and the output line have the same wiring pitch, they can be directly connected in a straight line. Even if the pitches are different, wiring is easy because the input 1111 and the output section face each other.

In addition, in the embodiments shown in FIGS. 1, 5, and 6, as shown in FIG. A vector/matrix multiplier that can rewrite values can also be used.A vector/matrix multiplier that can rewrite connection weight values is, for example, I, E, E, E, Journal of Solid State Circuits, Vol. 25. , No. 1 (February 1,990), pp. 207-214 (IEEE.

Journal of 5solidState
Ci, rcuits, vow, 25
゜No,],, (Feb, 1990) pp207
21/I), it is possible to use a single information processing device for various purposes by using a Begg I-le-matrix multiplier like this, which can rewrite the connection weight values. can.

Next, an embodiment relating to the transmission and reception of the signal J (interface) between the information processing device according to the present invention integrated on a semiconductor chip and a device outside the chip will be described.

FIG. 7 is an embodiment showing a configuration for an interface with the outside of the chip, using FIG. 1 as an example.

In addition to the vector/matrix multipliers VMMI and VMM2, an input signal processing circuit 1 for interfacing with the outside of the chip;
The output signal processing circuit 2 is illustrated.

In this embodiment, n1 first layer neuron output values are input from outside the chip through the input signal processing circuit 1 to the vector/matrix multiplier VMMl. Further, the neuron output value of the third layer obtained as the output of the VMM 2 is outputted to the outside of the chip through the output signal processing circuit 2. The number of signal wires nin and noul between the outside of the chip and the input signal processing circuit 1 and between the output signal processing circuit 2 and the outside of the chip cannot usually be set very carefully due to restrictions on the number of bins of the package. Therefore, in this embodiment, n i n .

The numbers nout are made smaller than nl and n3, respectively, so that transmission and reception of signals 13 to and from outside the chip are performed on a time-sharing basis. Therefore, according to this embodiment, a large-scale neural network can be realized without being limited by the number of package bins.

In the example shown in FIG. 7 above, if the number of neurons is very large compared to n 1n, no t, it takes time to interface with the outside of the chip, and the advantage of speeding up by parallel processing inside the chip is lost. It may fade. In particular, this problem becomes serious when processing of the neuron output values of a small set is performed successively. In such a case, the input signal processing circuit 1 may be provided with double buffer memories for temporarily storing 01 neuron output values. First, 01 neuron output values are written from outside the chip into a first buffer memory in a time-division manner and transferred all at once to a second buffer memory.

While writing the next n 1 neuron output values to the first buffer memory in a time-division manner, input the 01 neuron output values from the second buffer memory to VMML in parallel to perform calculations. . In this way, it is possible to suppress an increase in processing time due to time-divisionally reading neuron output values from outside the chip. Of course, the output signal processing circuit 2 also has two buffer memories as needed.
It can be installed in a ton.

Although the embodiment shown in FIG. 7 uses a 73-layer neural network as an example, it is of course applicable to other hierarchical types such as those shown in FIGS. 5 and 6, or the Hopfield type.

The information processing device according to the present invention can perform the human output of signal 3 as an analog signal, or if the external device is a device that handles digital signals, it is more convenient to use the human output signal as a digital value. . Additionally, digital signals have the advantage of being more resistant to noise, making it easier to improve reliability.

Next, an embodiment of an input signal processing circuit 1 and an output signal processing circuit 2 for inputting and outputting digital signals will be described. Figure 8(a) shows that the output values of 01 neurons expressed as digital signals of Rpitz I are taken in from outside the chip in a time-division manner by nin bits, and are sent in parallel to the vector/matrix multiplier as 01 analog signals. This is an embodiment of the input belief processing circuit 1 for inputting data into the circuit 1. In this example, the output value of a neuron represented by a digital signal of r bits each is input from outside the chip in a time-sharing manner and is stored in the digital memory [) Ml. It is written and memorized. Digital memory 1] M1 heneuron output value from outside the chip, ()
The number of wires n i n to be inserted is determined by taking into account the number of bins of the package.

Next, the digital signal corresponding to the output value of the neuron read out from the digital memory DMI is sent to the DA converter DΔ1 for every r bits. DΔ29. .

The output values of n1 neurons input to DAnl and converted to analog values as its output are simultaneously vector
It is input to matrix multiplier VMM1.

According to this embodiment, since input and output are performed using digital signals, it is possible to easily establish a highly reliable interface with the outside of the chip. Also, the number of signal wires from outside the chip n i n is the number of output signals of the input signal processing circuit l
Since it can be installed independently from 1, a large-scale neural network can be constructed even in cases where the number of pins cannot be increased too much due to restrictions such as packaging.

If you want to defeat the point that the processing time increases because the neuron output values are written in a time-sharing manner, you can provide double buffer memories as described in 1);j. FIG. 8(b) shows the digital memory 1.0 as a buffer memory.
This is an example using ) MIA and l-) Ml l:3. Digital memory i) MIA and I) M
] 13, it is possible to transfer m1 pieces of information at a time. In this embodiment, the digital memory DM is accessed from outside the chip.
After writing the output values of 01 neurons into IA in several parts, they are collectively transferred to DM] and B.

- If we set m1 equal to the product of r and nl, then D
Output values of n1 neurons can be transferred to MIB at the same time. According to this embodiment, D M l 13 to I]△
While information is transferred to the vector/matrix multiplier through the converter and operations are performed, the digital memory DMIΔ
The output values of the 01 neurons necessary for the next information processing can be written in several parts. In this way, when continuously processing the output values of neurons in a detailed set, the time loss due to input from outside the chip can be kept to a minimum. The embodiment shown in FIG. 8 described above is suitable when the output value of a neuron is represented by r bits4. In some cases, the output value of a neuron may be represented by l bits of O and l. In such a case, I) Since the Δ converter becomes difficult, it is sufficient to input information directly from the digital memory to the vector/matrix multiplier as in the embodiment of FIG. 9.The embodiment of FIG. Since it does not use a DA converter, it has the advantage of being less susceptible to noise and other factors and making it easier to obtain stable performance.

Next, an embodiment of the output signal processing circuit 2 will be described.
6 FIG. 10 shows Δl) converter ΔI) l and switch wire SWI, SW2.1. ．． 3, which is an embodiment of the output t1 processing circuit 2 configured by 5Wn3, after 03 neuron output values are obtained by the vector/matrix multiplier, one of S to Vn3 is output from the switch S to V]. When one neuron is made conductive and the other is non-conductive, the analog transmission corresponding to the output value of one neuron is A1. ) is input to the converter AI) and outputs a neuron output value expressed as a digital value. A desired neuron output value can be obtained by changing the switch. In this embodiment, a desired neuron output value can be obtained as a digital value, which is convenient for interfacing with a digital LSI outside the chip. Note that it is also possible to easily obtain a plurality of neuron output values at a time by increasing the number of AD converters as necessary.

The embodiment of the output signal processing circuit 2 shown in FIG. 11 is a combination of 03 AD converters and a digital memory. In this embodiment, 03 neuron output values are simultaneously converted into digital values of S pi I and stored in the digital memory I.
) M 28-; Can be read. n outside the chip
The neuron output value is read out by dividing O 11 into several parts using a real signal line. In this way, after loading the neuron output value into the digital memory l)M2, the vector/matrix multiplier can start calculating the next neuron output value. Therefore, if this embodiment is used for the output signal processing circuit 2 and the circuit shown in FIG. 8(1) is used for the input signal processing circuit 1,
Number of lines n i n, n O IJ 1. It is possible to perform high-speed information processing without being limited by.

In the embodiments of the output signal processing circuit 2 described so far, an AD converter is used to convert the neuron output value into a digital value of SpiI. However, when the neuron output value is expressed as a 1-bit digital value, ie, O or 1, the AD converter can be omitted, as in the case of the input signal processing circuit.

Figure 12 shows an example in which a vector/matrix multiplier that can rewrite connection weight values is used to make it applicable to both hierarchical networks and Hopfield neural networks. If necessary, 3 layers of Nets I and Wa 2! (It can be used as a five-layer network or a Hopffield neural network.) In Fig. 12, lΔ, 11.3 is an input belief processing circuit, and 2Δ, 2B is an output signal processing circuit.

Further, 8 to IA and 5W13 are switches. .

First, we will explain the case where it is used as a three-layer network. According to this embodiment, two types of three-layer networks can be configured simultaneously on chip 1''.For convenience of explanation -
] 2. Let's call each network X and Y. 1. The connection weight between the first layer and the second layer of network Set the connection weight of VMM2 to switch SWA, S
The circuits IA and 2A and 2B and IB are electrically separated by WB. Furthermore, the connection weight between the first layer and the second layer of the network I/work Y is calculated by the vector/matrix multiplier V
In MM3, the 3,11 combined weight between the second layer and the third layer is set to V
Set it to MM4. In this state, the neuron output values of the first layer of networks X and Y are transmitted to the vector/matrix multiplier VMMI,
When input to VMM3, output signal is generated - Processing circuits 2A and 2B
Through this, it is possible to read the neuron output values of the third layer of the network X and Y.

Next, a case will be described in which the embodiment shown in FIG. 12 is used as a five-layer network. In this case, the coupling weight (b) between the first layer and the second layer of the network is assigned to the vector/matrix multiplier VMM I, the coupling weight between the second and third layer is assigned to VMM2, and the coupling weight between the second and third layer is assigned to the vector/matrix multiplier VMM The element 21 combination weight between
In M3, #i'i combination weight between the fourth layer and the fifth layer is VMM
4, the switch circuits SWA and SWB electrically disconnect and connect the circuits IA and 2A and the circuits 2+3 and 1B, respectively. In this state, the neuron output value of the first layer of the network is input to the signal processing circuit IA.
When inputted to the vector/matrix multiplier VMM1 through the output filter processing circuit 2A, the neuron output value of the fifth layer of the network can be read out.

Finally, a case where the embodiment shown in FIG. 12 is used as a synchronous Hopfield type network will be described. In this case, as in Fig. 6, the connection weight values of all neurons in the synchronous Hopfield type network are
Set each from MI to VMM4. Also, a switch circuit SW A.

By SWB, the above circuits IA, 2A and 2[3 and 1B
7. In this state, when the initial values of all the neuron output values of the network are input to the vector/matrix multiplier VMMl through the input signal processing circuit IA, all the neuron output values are input as the output signal of VMMl. The updated value of is obtained. Subsequently, that signal is input to vector/matrix multipliers VMM2, VMM3, VMM4 to continue updating all neuron output values. Furthermore, updating can be continued an arbitrary number of times by inputting the output value of the vector/matrix multiplier VMM4 to the vector/matrix multiplier VMMI through the input signal processing circuit 1A and continuing processing. After repeating updates for -1゛ minutes,
The neuron output value is read from the output signal processing circuit 213 or 2A. Note that it is also possible to increase the speed by providing multiple wiring lines so that the data can be directly transferred from the vector/matrix multiplier VMM4 to VMM1 and from VMM2 to VMM3 only through the switch without passing through the memory. The embodiment shown in FIG. 12 described above has the advantage that various networks such as hierarchical type and Hopfield type can be realized with one device.

FIG. 13 shows a signal processing circuit IA and an output signal processing circuit 2A when input/output is performed using digital signals in FIG.
This is an example showing the configuration of.

As shown in FIGS. 8(a) and (b) or FIG. 11, it is composed of a DA converter or an AD converter and a digital memory. The switch circuit SWA controls connection and disconnection of signal wiring from the digital memory DM2A to DMIA.

0 obtained as the output of the vector/matrix multiplier VMM4
Analog voltages corresponding to seven neuron output values are AD
The signals are input to converters ADI, . The written information is written to the digital memory I) MIA through the switch circuit SWA as required, or read out to the outside of the chip. When inputting the input value of the vector/matrix multiplier VMMl from outside the chip, leave the switch circuit SWA in a disconnected state and input the input value to the digital memory DMIA from outside the chip.
Digital information can be written through 1nA wires. Then DA converter DA 1. DA2. , ,,D
An I converts it into an analog signal and inputs it to the vector/matrix multiplier VMMl.

Note that the input signal processing circuit 1B and the output signal processing circuit 2B can also be configured in the same manner as in FIG. 13.

According to the embodiment shown in FIG. 13, input and output can be easily converted into digital signals in the embodiment shown in FIG. In addition, in the embodiment shown in FIG. 13, the digital memory in the output signal processing circuit is provided in duplicate in the same manner as in FIG. Information processing can also be performed. Furthermore, when the neuron output value is expressed by one bit of O or 1, the DA, A, and D converters can of course be omitted.

Although the present invention has been described using Example 1, the vector/matrix multiplier used in the present invention is not limited to those described above, and various types can be used.

Furthermore, in the present invention, if the pitches of the input lines and output lines of the vector/matrix multiplier are set d1 to be the same, the wiring between the vector/matrix multipliers can be linearly performed, so the wiring length can be reduced. It can be made shorter and the effects of noise or delay can be reduced.

However, even if the pitch of the input line and output line cannot be made the same, wiring can be done without increasing the wiring length by arranging the vector/matrix multiplier so that the input line and the output J line face each other. Of course, it can be done.

So far, a neural network information processing device using vector/matrix multipliers has been described, but the present invention is not limited to this and can be applied to other devices. In particular, when multiple parallel arithmetic units with a large number of input/output lines are integrated on a chip and signals are transmitted between them, parallel arithmetic units that perform continuous signal processing as in the present invention may be used. Wiring can be facilitated by arranging the input section and the output section facing each other.

[Effects of the Invention] As described above, according to the present invention, a vector/matrix multiplier having a large number of input lines and output lines is arranged on a semiconductor chip so that the input lines and output lines face each other. This facilitates wiring between the vector and matrix multipliers. Therefore, large-scale parallel computation becomes possible, and a neural network consisting of a large number of neurons can be constructed.

[Brief explanation of drawings]

Figure 1 shows an example in which a three-layer hierarchical neural network is configured on a semiconductor chip. FIGS. 2(a) and 2(b) are diagrams showing the principle of a hierarchical neural network. FIGS. 2(c) and 2(d) are diagrams showing examples of characteristics of the nonlinear function circuit in FIG. 2(b). FIGS. 3(a) and 3(b) are diagrams showing the principle of a Hopfield neural network. FIGS. 4(a) and 4(b) are diagrams showing the principle of a conventional Bertorl/matrix multiplier using resistive elements. FIG. 5 shows an example in which a five-layer hierarchical neural network is configured on a semiconductor chip. FIG. 6 shows an example in which a Hopfield neural network is configured on a semiconductor chip. FIG. 7 shows an example of an input/output signal processing circuit in which a three-layer hierarchical neural network is configured on one chip. FIG. 8(a) shows an example of the configuration of an input signal processing circuit. FIG. 8(b) is a second diagram showing the configuration of the input signal processing circuit.
Example. FIG. 9(a) is an embodiment suitable for representing the neuron output J value with one bit in the embodiment of FIG. 8(a). FIG. 9(b) is an embodiment suitable for representing the neuron output value in bits in the embodiment of FIG. 8(b). FIG. 10 shows one embodiment of the configuration of an output signal processing circuit. FIG. 11 is a second embodiment showing the configuration of an output signal processing circuit. Figure 12 shows how four vector/matrix multipliers can be combined to create various networks as needed, such as a 3-layer hierarchical neural network, a 5-layer hierarchical neural network, or a Hopfield neural network. An example of the present invention that can be used. Figure 13 shows an input signal processing circuit suitable for the configuration of Figure 12;
An example showing the configuration of an output signal processing circuit. Description of codes VMMI, VMM2°VMM3. VMM4・・・・・・・・・Vector・
Matrix multiplier], IA, IB... Input signal processing circuit 2.2A, 2B...
Output signal processing circuit DMI, DM] A. DMl, B, DM2゜DM2A, DM2B......Digital memory DA], , I) A2゜, , 1) Anl. ,DAI,DA2゜,,,DAn5・・・・・・・・・・・・・・・D
A converter AD1. , AD2゜, , , ADn3゜A101. AD2゜、、、ADn7・・・・・・・・・・・・・・・・・・
AD converter SWI, SW2゜,, SWn 3゜

Claims (2)

[Claims] 1. In an information processing device configured using a plurality of arithmetic units having a plurality of input lines and a plurality of output lines, inputting information from the plurality of input lines and obtaining outputs in parallel to the plurality of output lines, at least An information processing device characterized in that two arithmetic units are arranged such that an input part of one arithmetic unit and an output part of the other arithmetic unit face each other. 2. 2. The information processing apparatus according to claim 1, wherein said arithmetic unit performs a sum-of-products operation necessary for neural network information processing.