JPH03214373A - Neurocomputer - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

DETAILED DESCRIPTION OF THE INVENTION Field of Industrial Application The present invention relates to a neurocomputer, which is hardware for constructing a neural network used for pattern recognition, voice recognition, signal encoding, etc. be.

A conventional neural network is composed of neurons that convert a large number of inputs into a single output and output the same, and synapses that weight the output of a neuron in the previous stage and output it to a neuron in a subsequent stage. In such a neural network, an operation called learning is performed in which the weights in the synapses are changed so that the output for a certain input, such as OX, 10, becomes equal to the expected value corresponding to the input. Therefore, in order to realize learning, it is necessary to be able to freely change the weights. Conventionally, a multiplier circuit 2 made up of a MOS transistor 1 as shown in FIG.
The gate voltage of the MOS transistor 1 corresponding to the weight is held as a voltage in the gate capacitance of the MOS transistor 1 or in the capacitance 3 connected to the gate. During learning, change the amount of charge on the capacitor corresponding to the weight by an appropriate amount. This capacitance becomes a small value of several pF when integrating a neural network. 4 is an operational amplifier.

In addition, the multiplication circuit 2 used for synapses and the sigmoid function (f=1/(1+aXp(x)) used for neurons
1. Nlos transistors are used in the generation circuit (not shown) because they are compatible with conventional VLSI technology and are easy to integrate.

Problems to be Solved by the Invention In the above-mentioned configuration, the charge stored in the capacitance of transistor 1 is discharged within several tens of millimeters, so this neural dysfunction does not work constantly. As the weight charge is discharged, the output of the faulty work also changes, and the expected output for the input is not maintained. Furthermore, multipliers and sigmoid function generating circuits formed using MOS transistors have poor accuracy and require a long time to learn.

The present invention solves the above-described problems, and aims to provide a neurocomputer that constantly operates and has high learning efficiency.

Means for Solving the Problems In order to solve the above problems, the neurocomputer of the present invention includes a digital memory that holds weights, a digital-to-analog converter that converts this digital value into an analog value, and a converted analog value. and a synapse section that weights the sigmoid function outputted from the neuron using the analog value of the capacitor, and writes the analog value of the capacitor within one foot period in response to the output of the neuron. It is designed so that it can be changed.

Furthermore, in addition to the above configuration, the neurocomputer of the present invention includes bipolar transistors for calculation and M for input.
It includes a synapse section consisting of a multiplier made up of OS transistors, and a neuron having a sigmoid function generation circuit made up of biholer transistors.

Operation With the above configuration, the amount of weighted charge input to the synapse section and stored in the capacitor is kept constant by using a circuit having a digital memory that holds the weight and a D/A converter that converts the value into an analog value. In addition, since we use a multiplier made of bibolar transistors with a high degree of calculation and a sigmoid function generation circuit in the synapse and neurons, we can calculate the time change of the charge stored in the capacitor, that is, the weight. The weight storage capacity is reduced, the weight storage capacity is maintained steadily, and the calculation accuracy is reduced. Therefore, a neural network with high learning efficiency can be constructed.

EXAMPLE Hereinafter, an example of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a neurocomputer P according to an embodiment of the present invention. In Figure 1,
A hierarchical neural network consisting of three neurons 11 is configured, and the weights are determined by the learning control I control circuit 12.
RAM or EPROM, EEP
It is stored in a digital memory 131 such as a ROM. On the other hand, the weight in the digital memory 13 corresponding to the address selected by the address generation circuit l4 is D/A converted by the D/A converter 15, and then transferred to the address generation circuit 14.
transfer gate 1 via decoder 16 by
7 and give it as a charge to the capacitor 19 which is the capacitor of the corresponding synapse section 18 through the transfer gate 17. As shown in the conventional example, this charge is discharged, but before the amount of discharge is as small as about 10% or less, the weight is transferred again to the capacitor 19 through the gate 17 to prevent the charge from discharging. In this way, synapse part 1
The synapse part 18 is caused by the charge stored in the capacitor 19 of
The gate pressure of the MOS }-transistor becomes almost constant.

Furthermore, as shown in FIG. 2, the synapse sections 18 are arranged in a two-dimensional matrix. At this time, if the signal from the D/A converter 15 downstream of the digital memory 13 is converted into a time series, only one wire is required. Furthermore, by arranging the synapse portions 18 in a matrix, the number of address lines can be significantly reduced because only the row and column addresses need to be wired. In this way, the transfer gates 17a . 17b, capacity】9
The weight charge can be easily updated by giving a charge to the . This method is effective for circuit integration, since it is not necessary to increase the area of the capacitor section in order to extend the holding time of the weighted charges, and the number of wiring lines can be reduced.

In the initial state, the weights are given a random number, and the output for a certain input will deviate from the expected value. Training i
The weight is changed by the dJ C4 circuit 12. Replace the weight thus obtained with the value in the digital memory 13.

This learning cannot match the output with the expected value in one time, and the weights are corrected many times. Therefore, it is important that the time required to write data into the weight holding capacitor 19 is short, but according to our method, the writing time constant is determined by the on-resistance of the transfer gate 17 and the weight holding capacitor 19. It is determined by the product of 19, and one breath usually takes less than 10 seconds, which is extremely fast.

Next, FIG. 3 shows a circuit of a multiplier comprising a synapse section 18 composed of bipolar transistors and MOS transistors. Multipliers are used from subtraction-Gil
The weight holding capacitor 19 connected to the weight input line 20 and holding the input voltage is connected to the weight input line 20.
To prevent the discharge of charge from the input differential amplifier 2
1 uses a Jv]OS transistor with high input immunity. Further, 22 is a signal input terminal from the neuron 11, and is connected to the BIOS transistor. Furthermore, since a bipolar transistor is used in the multiplication circuit 23, the multiplication error can be suppressed to a few percent or less. 24 is Batsufa Anbu 2
This is the output terminal of the synapse section 18, which is connected to the output terminal of the synapse section 5.

FIG. 4 is a circuit diagram of a neuron, which has a sigmoid function generating circuit 26 made up of pievora transistors. The characteristics of a bipolar transistor differential amplifier are of the sigmoid function type (]/(]+sxp (qx/kT)
)).

Here, X represents the input voltage, q represents the elementary charge, k represents the Boltzmann constant, and T represents the temperature (0K). Emitsu criticism rl. r
2 is inserted in order to expand the input range to the sigmoid function generating circuit 26. This circuit is simple and very effective in increasing the number of units that can be integrated. 27 is a human input terminal of the neuron 11, and 28 is an output terminal. Well, 29 is an operational amplifier.

In this way, the circuits shown in FIGS. 3 and 4 both have low accuracy, with errors within 5%. When learning a hierarchical neural network, an optimization algorithm called backbroadcasting, which is a type of steepest descent method, is used to determine weight values. This calculates the partial weight of the error by multiplying the error between the output and the expected value by the weight of the sigmoid function. The weight is changed in proportion to the value of the partial differential so that the error becomes smaller. Normally, such calculations are performed on a computer, and the calculated weight values are fed to neural network hardware for processing. In this way, it is possible to create a large number of neural networks that have been trained in the same way as the calculated weights.

However, if the precision of the circuitry in the hardware of the Neuranore network is poor, if the shape of the sigmoid function used for calculation is different, or if the multiplication precision of the synapse part is poor, the calculated weight values may not match the hardware. The output changes and does not match the expected value.

Furthermore, if the hardware of the neural network can operate faster than a computer, the learning itself is also performed using hardware. In this case, input is given to the neural network, the output from the sononetwork is measured, and the computer calculates weights that will bring the output closer to the expected value.

Again, use these weights to measure the output of the neural network and repeat the process. This is repeated until the output converges to the expected value, but if the circuit accuracy is poor, it will take a very large number of times to converge.

As described above, the better the circuit accuracy, the better the learning efficiency of the neural network. As a precaution, I would like to design a circuit based on bipolar transistors with high circuit accuracy.

Effects of the Invention As described above, according to the present invention, weights can be held almost constantly in the weight holding capacitor of the capacitor, and a neural network with high learning efficiency can be constructed. .

[Brief explanation of drawings]

Fig. 1 is a block diagram showing the construction of a neurocomputer according to an embodiment of the present invention, Fig. 2 is a diagram showing the layout of the synapse section of the neurocomputer, and Fig. 3 is a circuit diagram of the synapse section of the neurocomputer. 4 is a circuit diagram of a neuron in the same neurocomputer, and FIG. 5 is a circuit diagram of a synapse section of a conventional neurocomputer. 11・゜・Neuron, 12-・・Learning control circuit, 13
--Digital memory, 14...Address generation circuit, l5...D/A converter, l6...Decoder-1
6m...X decoder, 16b...Y decoder, 17
．． 17m. 17b--Transfer gate, 18...Synapse section, 19...Capacitance, 21...
Input differential amplifier, 23...Multiplication circuit, 26...
Sigmoid function generator circuit.

Claims (1)

[Claims] 1. A digital memory that holds weights, a digital-to-analog converter that converts this digital value into an analog value, and a capacitor that holds the converted analog value;
A neurocomputer comprising a synapse section that weights a sigmoid function output from a neuron using an analog value of the capacitor, and configured to rewrite the analog value of the capacitor within a certain period of time in response to the output of the neuron. 2. Bipolar transistor for calculation and MOS for input
2. The neurocomputer according to claim 1, further comprising a synapse section made up of a multiplier made up of transistors, and a neuron having a sigmoid function generation circuit made up of bipolar transistors.