JPH0830572A - Neural network circuit and calculation method using the same - Google Patents
Neural network circuit and calculation method using the sameInfo
- Publication number
- JPH0830572A JPH0830572A JP6165641A JP16564194A JPH0830572A JP H0830572 A JPH0830572 A JP H0830572A JP 6165641 A JP6165641 A JP 6165641A JP 16564194 A JP16564194 A JP 16564194A JP H0830572 A JPH0830572 A JP H0830572A
- Authority
- JP
- Japan
- Prior art keywords
- value
- cumulative addition
- synapse
- calculation
- circuit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Complex Calculations (AREA)
Abstract
(57)【要約】
【目的】 シナプス演算及び累積加算の演算量を削減で
き、大規模で且つ高速動作可能なニューラルネットワー
ク回路及びこれを用いた演算方法を提供する。
【構成】 ２進ビット列で表される多数の入力値と該多
数の入力値に対応する多数のシナプス係数とのシナプス
演算を、シナプス演算及び累積回路２において一の入力
値の全ビットとこれに対応するシナプス係数の１ビット
との間で該シナプス係数の上位ビットから順次行って加
算し、これを各入力値及びシナプス係数毎に行って累積
加算し、判定回路６で該累積加算値が飽和領域に達した
か否かを判定し、飽和領域でなければ演算をシナプス係
数のより下位ビットまで行い、飽和領域になれば演算を
停止することにより、それ以降のシナプス演算を省略す
る。
(57) [Summary] [Object] To provide a large-scale neural network circuit capable of reducing the calculation amount of synapse calculation and cumulative addition and operating at high speed, and a calculation method using the same. A synapse calculation of a large number of input values represented by a binary bit string and a large number of synapse coefficients corresponding to the large number of input values is performed on all bits of one input value in the synapse calculation and accumulation circuit 2 and With respect to 1 bit of the corresponding synapse coefficient, the higher order bit of the synapse coefficient is sequentially added and added, and this is performed for each input value and synapse coefficient and cumulative addition is performed, and the cumulative addition value is saturated by the determination circuit 6. It is determined whether or not the region has been reached, and if it is not the saturation region, the calculation is performed up to the lower bits of the synapse coefficient, and when it becomes the saturation region, the calculation is stopped, and the subsequent synapse calculation is omitted.
Description
【０００１】[0001]
【産業上の利用分野】本発明は、シナプス演算回路、累
積回路及びしきい値回路からなるニューロン回路を多数
接続して構成されるニューラルネットワーク回路及びこ
れを用いた演算方法に関するものである。BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural network circuit formed by connecting a large number of neuron circuits each including a synapse operation circuit, an accumulator circuit and a threshold circuit, and an operation method using the neural network circuit.
【０００２】[0002]
【従来の技術】ニューラルネットワーク回路は、ニュー
ロン回路を単位として、これを多数接続して構成されて
いる。１個のニューロン回路は、図２に示すように多入
力（ｘ１，ｘ２，……ｘｎ）１出力（ｙ）の回路であ
る。複数の入力にはそれぞれ対応するシナプス係数（Ｗ
１，Ｗ２，……Ｗｎ）があり、入力値とシナプス係数と
の積、差の絶対値あるいは差の二乗等の演算を行った後
に累積加算が行われ、累積加算の結果の大きさにより出
力値が決定される。2. Description of the Related Art A neural network circuit is constructed by connecting a large number of neuron circuits as a unit. One neuron circuit is a multi-input (x1, x2, ... Xn) one output (y) circuit as shown in FIG. The synapse coefficient (W
1, W2, ... Wn), and the cumulative addition is performed after calculating the product of the input value and the synapse coefficient, the absolute value of the difference or the square of the difference, and output depending on the size of the cumulative addition result. The value is determined.
【０００３】出力値を最終的に決定するしきい値回路
は、図３に示すような伝達特性を備えている。図３(a)
のシグモイド関数形が最も汎用性が高いが、演算を簡単
化するために図３(b) の折れ線形あるいは図３(c) のス
テップ関数形のように単純化したものや、機能を高める
ために図３(d) のように非単調形にしたものも使用でき
る。なお、図中、Ｔｈ，Ｔｈ１，Ｔｈ２，Ｔｈ３，Ｔｈ
４はそれぞれ飽和領域境界値である。このニューロン回
路の接続によりニューラルネットワーク回路の構造が決
まる。The threshold circuit which finally determines the output value has a transfer characteristic as shown in FIG. Figure 3 (a)
The sigmoid function form of is the most versatile, but in order to simplify the operation, it is simplified such as the polygonal shape of Fig. 3 (b) or the step function form of Fig. 3 (c), or to enhance the function. The non-monotonic type shown in Fig. 3 (d) can also be used. In the figure, Th, Th1, Th2, Th3, Th
4 is a saturation region boundary value, respectively. The connection of the neuron circuits determines the structure of the neural network circuit.
【０００４】最も一般的に用いられるのは、図４に示す
ような３層構造のニューラルネットワーク回路である。
３層構造の場合には、第２層のニューロン回路の層を中
間層、第３層のニューロン回路の層を出力層と呼んでい
る。それぞれの入力端子からの信号は全てのニューロン
回路に並列に入力され、それぞれのニューロン回路は並
列に入力信号を処理する。入力信号が加えられると、特
定のニューロン回路が反応して認識等の処理が実現され
る。Most commonly used is a neural network circuit having a three-layer structure as shown in FIG.
In the case of the three-layer structure, the layer of the neuron circuit of the second layer is called the intermediate layer, and the layer of the neuron circuit of the third layer is called the output layer. The signals from the respective input terminals are input in parallel to all the neuron circuits, and the respective neuron circuits process the input signals in parallel. When an input signal is applied, a specific neuron circuit reacts to realize processing such as recognition.
【０００５】従来、前述したような機能を有する大規模
なニューラルネットワーク回路を具体化する場合、図５
に示すようにマイクロプロセッサとＲＡＭとを組み合わ
せて構成するのが一般的であった。本構成では、シナプ
ス演算、累積加算、しきい値処理をマイクロプロセッサ
で行い、シナプス係数はＲＡＭに格納する。そして、マ
イクロプロセッサで演算中、シナプス係数が必要となっ
た時点でＲＡＭにアクセスしてシナプス係数を読み出す
という処理を行っていた。Conventionally, when embodying a large-scale neural network circuit having the above-mentioned function, FIG.
As shown in (1), it is generally configured by combining a microprocessor and RAM. In this configuration, synapse calculation, cumulative addition, and threshold processing are performed by the microprocessor, and synapse coefficients are stored in RAM. Then, during the calculation by the microprocessor, when the synapse coefficient is required, the RAM is accessed and the synapse coefficient is read out.
【０００６】[0006]
【発明が解決しようとする課題】このような構成におい
ては、ＲＡＭを増設することにより実現可能なニューラ
ルネットワーク回路の規模を拡大でき、プログラムを替
えることにより演算のアルゴリズムを変更することも容
易である。In such a structure, the scale of the neural network circuit that can be realized can be expanded by adding RAM, and it is easy to change the operation algorithm by changing the program. .
【０００７】しかしながら、その反面、ＲＡＭへのシナ
プス係数のアクセス速度が動作速度を規定し、さらにニ
ューラルネットワーク回路の規模が大きくなるとシナプ
ス演算及び累積加算の演算量が増加し、高速動作を妨げ
る要因の１つとなっていた。これは前述した従来の構成
では、全てのシナプス結合において入力値の全ビットと
シナプス係数の全ビットとのシナプス演算、並びにシナ
プス演算の結果の累積加算を行っていたため、演算量が
膨大となり、それに伴うシナプス係数のＲＡＭへのアク
セスにも時間を要するためであった。However, on the other hand, the access speed of the synapse coefficient to the RAM defines the operation speed, and when the scale of the neural network circuit increases, the operation amount of synapse calculation and cumulative addition increases, which is a factor that hinders high-speed operation. It was one. This is because in the conventional configuration described above, the synapse operation of all the bits of the input value and all the bits of the synapse coefficient and the cumulative addition of the results of the synapse operation are performed in all the synapse connections, resulting in a huge amount of operation. This is because it takes time to access the RAM of the accompanying synapse coefficient.
【０００８】本発明の目的は、シナプス演算及び累積加
算の演算量を削減でき、大規模で且つ高速動作可能なニ
ューラルネットワーク回路及びこれを用いた演算方法を
提供することにある。An object of the present invention is to provide a neural network circuit that can reduce the amount of operations for synapse operations and cumulative additions, can operate at a large scale, and an operation method using the neural network circuit.
【０００９】[0009]
【課題を解決するための手段】前記目的を達成するた
め、本発明では、２進ビット列で表される入力値とシナ
プス係数とのシナプス演算を、従来のように全てのビッ
ト間で行うのではなく、入力値の全ビットとシナプス係
数の任意の１ビットとの間で行い、得られた部分積を加
算することにより実行することを特徴とする。また、演
算に使用するシナプス係数の任意の１ビットを上位のビ
ットから下位のビットへ順次変えていくことにより、概
算値から徐々に正確な結果が得られる演算過程をとるこ
とを特徴とする。また、累積値が演算の進行に伴って単
調増加あるいは単調減少する演算手順を用いることによ
り、累積結果がしきい値関数の飽和領域内に達した時点
で演算を停止することを特徴とする。In order to achieve the above object, according to the present invention, a synapse operation between an input value represented by a binary bit string and a synapse coefficient is not performed between all bits as in the conventional case. Instead, it is performed between all the bits of the input value and an arbitrary 1 bit of the synapse coefficient, and the obtained partial products are added to execute. Further, it is characterized in that an arbitrary one bit of the synapse coefficient used for the operation is sequentially changed from the upper bit to the lower bit, thereby taking an operation process in which an accurate result is gradually obtained from the approximate value. Further, by using a calculation procedure in which the cumulative value monotonically increases or decreases with the progress of the calculation, the calculation is stopped when the cumulative result reaches the saturation region of the threshold function.
【００１０】[0010]
【作用】本発明によれば、２進ビット列で表されるシナ
プス係数をＲＡＭから読み出す場合、従来のように全て
のビットを毎回読み出す必要がなく、上位ビットから必
要なビットを読み出すのみで良く、ＲＡＭへのアクセス
時間を大幅に削減できる。また、シナプス演算及び累積
加算についても、従来、行っていた演算を全て実行する
ことなく出力が決定した時点で以降の演算を停止するた
め、演算処理時間を大幅に短縮できる。According to the present invention, when the synapse coefficient represented by a binary bit string is read from the RAM, it is not necessary to read all the bits each time as in the conventional case, and only the necessary bits are read from the upper bits. The access time to the RAM can be significantly reduced. Also, regarding the synapse calculation and the cumulative addition, since the subsequent calculations are stopped when the output is determined without executing all the calculations that have been conventionally performed, the calculation processing time can be significantly shortened.
【００１１】[0011]
【実施例１】図１は本発明の第１の実施例を示すもの
で、図中、１はセレクタ、２はシナプス演算及び累積回
路、３はシナプス係数メモリ回路、４はメモリ読み出し
回路、５はしきい値回路、６は判定回路、７はクロック
発生回路、８は制御回路である。なお、判定回路６は比
較器６１，６２、飽和領域境界値１用レジスタ６３及び
飽和領域境界値２用レジスタ６４からなっている。First Embodiment FIG. 1 shows a first embodiment of the present invention, in which 1 is a selector, 2 is a synapse calculation and accumulator circuit, 3 is a synapse coefficient memory circuit, 4 is a memory read circuit, and 5 is a memory read circuit. Is a threshold circuit, 6 is a determination circuit, 7 is a clock generation circuit, and 8 is a control circuit. The determination circuit 6 includes comparators 61 and 62, a saturated region boundary value 1 register 63 and a saturated region boundary value 2 register 64.
【００１２】飽和領域境界値１及び２の値は、図３に示
したしきい値回路の伝達特性において飽和領域と境界領
域との境界を表しており、小さい方の値が飽和領域境界
値１、大きい方が飽和領域境界値２である。しきい値回
路の伝達特性が図３(d) のように過渡領域が複数ある場
合には、さらに飽和領域境界値３用レジスタ及び飽和領
域境界値４用レジスタ並びにこれらに対応する比較器が
必要となる。ここでは説明を簡便にするため、しきい値
関数の伝達特性が過渡領域を１つのみ有する場合につい
て説明する。The saturated region boundary values 1 and 2 represent the boundary between the saturated region and the boundary region in the transfer characteristic of the threshold circuit shown in FIG. 3, and the smaller value is the saturated region boundary value 1. , The larger is the saturated region boundary value 2. When the transfer characteristic of the threshold circuit has a plurality of transient regions as shown in Fig. 3 (d), a saturated region boundary value 3 register, a saturated region boundary value 4 register, and a comparator corresponding to these are required. Becomes Here, in order to simplify the description, a case where the transfer characteristic of the threshold function has only one transient region will be described.
【００１３】本実施例において、シナプス演算及び累積
回路２の入力にはセレクタ１を通して入力端子が接続さ
れ、また、メモリ読み出し回路４を介してシナプス係数
メモリ回路３が接続されている。また、シナプス演算及
び累積回路２の出力はしきい値回路５及び判定回路６の
比較器６１，６２の入力に接続されている。In the present embodiment, an input terminal is connected to the input of the synapse calculation and accumulation circuit 2 through the selector 1 and a synapse coefficient memory circuit 3 is connected through the memory read circuit 4. The output of the synapse calculation and accumulation circuit 2 is connected to the inputs of the threshold circuit 5 and the comparators 61 and 62 of the determination circuit 6.
【００１４】しきい値回路５の出力はニューロン回路の
出力端子に接続されている。比較器６１の他方の入力に
は飽和領域境界値１用レジスタ６３が、比較器６２の他
方の入力には飽和領域境界値２用レジスタ６４がそれぞ
れ接続されている。比較器６１，６２の出力はそれぞれ
制御回路８の入力に接続されている。制御回路８の出力
はセレクタ１及びメモリ読み出し回路４に接続されてい
る。The output of the threshold circuit 5 is connected to the output terminal of the neuron circuit. The saturated region boundary value 1 register 63 is connected to the other input of the comparator 61, and the saturated region boundary value 2 register 64 is connected to the other input of the comparator 62. The outputs of the comparators 61 and 62 are connected to the inputs of the control circuit 8, respectively. The output of the control circuit 8 is connected to the selector 1 and the memory reading circuit 4.
【００１５】図６は本発明の特徴部分である、シナプス
演算及び累積回路２の詳細を示すもので、図中、２１は
「２」の補数変換回路、２２はシフタ、２３は加算器、
２４はレジスタＡ（単調増加演算の累積加算値格納用レ
ジスタ）、２５はレジスタＢ（単調減少演算の累積加算
値格納用レジスタ）、２６はスイッチ、２７はレジスタ
Ｃ（出力用累積加算値格納用レジスタ）、２８はＡＮＤ
回路、２９は制御回路である。FIG. 6 shows details of the synapse calculation and accumulator circuit 2 which is a characteristic part of the present invention. In the figure, 21 is a "2" complement conversion circuit, 22 is a shifter, and 23 is an adder.
Reference numeral 24 is a register A (a register for storing a cumulative addition value of a monotone increase operation), 25 is a register B (a register for storing a cumulative addition value of a monotone decrease operation), 26 is a switch, and 27 is a register C (for storing a cumulative addition value for output). Register), 28 is AND
A circuit, 29 is a control circuit.
【００１６】補数変換には「２」の補数で表現する場
合、「１」の補数で表現する場合等があるが、ここでは
「２」の補数で表現する場合を例にとって説明する。従
って、本実施例において「２」の補数変換回路を「１」
の補数変換回路で置き換えることも可能である。本実施
例において、「２」の補数変換回路２１に入力値の入力
端子が接続されており、「２」の補数変換回路２１の出
力はシフタ２２の入力に接続されている。シフタ２２の
出力は加算器２３の一方の入力に接続されている。加算
器２３のＳＵＭ出力はレジスタＡ２４とレジスタＢ２５
の入力に接続されている。レジスタＡ２４及びレジスタ
Ｂ２５の出力はそれぞれスイッチ２６の入力に接続され
ている。スイッチ２６の出力はＡＮＤ回路２８を介して
加算器２３の他方の入力に接続されるとともにレジスタ
Ｃ２７の入力に接続されている。The complement conversion may be represented by a "2" complement, a "1" complement, or the like. Here, the case of a "2" complement will be described as an example. Therefore, in this embodiment, the complement conversion circuit of "2" is set to "1".
It is also possible to replace with the complement conversion circuit of. In this embodiment, the input terminal of the input value is connected to the “2” complement conversion circuit 21, and the output of the “2” complement conversion circuit 21 is connected to the input of the shifter 22. The output of the shifter 22 is connected to one input of the adder 23. The SUM output of the adder 23 is the register A24 and the register B25.
Connected to the input of. The outputs of the register A24 and the register B25 are connected to the inputs of the switch 26, respectively. The output of the switch 26 is connected to the other input of the adder 23 via the AND circuit 28 and also to the input of the register C27.
【００１７】また、制御回路２９には「シナプス係数の
下位からｉビット目を用いた演算を実行中であるという
信号」、「シナプス係数の下位からｉビット目のビット
値」及び「入力値の符号」の入力端子が接続されてお
り、該制御回路２９から出力される制御信号Ｓ１が
「２」の補数変換回路２１に、制御信号Ｓ２がシフタ２
２に、制御信号Ｓ３がレジスタＡ２４に、制御信号Ｓ４
がレジスタＢ２５に、制御信号Ｓ５がスイッチ２６に、
制御信号Ｓ６がＡＮＤ回路２８の一方の入力に、制御信
号Ｓ７がレジスタＣ２７に接続されている。Further, the control circuit 29 has a "signal indicating that an operation using the i-th bit from the lower order of the synapse coefficient is being executed", a "bit value of the i-th bit from the lower order of the synapse coefficient" and an "input value". The control signal S1 output from the control circuit 29 is connected to the complement conversion circuit 21 of "2", and the control signal S2 is input to the shifter 2
2, the control signal S3 is sent to the register A24, and the control signal S4 is sent.
To the register B25, the control signal S5 to the switch 26,
The control signal S6 is connected to one input of the AND circuit 28, and the control signal S7 is connected to the register C27.
【００１８】本実施例においては、図７に示すように、
入力値の全ビットとシナプス係数の１ビットとの部分積
を計算し、その後、それらの結果を加算することにより
シナプス演算を実行する。本実施例における回路動作を
以下に述べる。In this embodiment, as shown in FIG.
A synapse operation is executed by calculating a partial product of all bits of the input value and one bit of the synapse coefficient and then adding the results. The circuit operation in this embodiment will be described below.
【００１９】(1) シナプス係数の上位ビットからの演算
を行うため、演算対象となる入力値の全ビットが「２」
の補数変換回路２１に、シナプス係数の必要なビットが
制御回路２９に適宜転送される。(1) Since the calculation is performed from the higher bits of the synapse coefficient, all the bits of the input value to be calculated are "2".
The necessary bits of the synapse coefficient are appropriately transferred to the control circuit 29 to the complement conversion circuit 21 of FIG.
【００２０】(2) 「シナプス係数の下位からｉビット目
を用いた演算を実行中であるという信号」、「シナプス
係数の下位からｉビット目のビット値」及び「入力値の
符号」から、入力値に対して「２」の補数変換を施すか
否かを制御回路２９が決定し、制御信号Ｓ１により
「２」の補数変換回路２１を制御する。なお、本明細書
（特許請求の範囲を含む。）中、ｉビット目とは、最初
のビットにつきｉ＝０として数えるものとする。(2) From the “signal indicating that the operation using the i-th bit from the lower order of the synapse coefficient is being executed”, the “bit value of the i-th bit from the lower order of the synapse coefficient” and the “sign of the input value”, The control circuit 29 determines whether or not to apply the "2" complement conversion to the input value, and controls the "2" complement conversion circuit 21 by the control signal S1. In the present specification (including claims), the i-th bit is counted as i = 0 for the first bit.
【００２１】(3) 「シナプス係数の下位からｉビット目
を用いた演算を実行中であるという信号」から制御回路
２９で何ビットシフトさせるかを決定し、制御信号Ｓ２
によりシフタ２２におけるシフト量を制御する。(3) Determine how many bits are to be shifted by the control circuit 29 from the "signal indicating that the operation using the i-th bit from the lower order of the synapse coefficient is being executed", and control signal S2
The shift amount in the shifter 22 is controlled by.
【００２２】(4) 「シナプス係数の下位からｉビット目
を用いた演算を実行中であるという信号」、「シナプス
係数の下位からｉビット目のビット値」及び「入力値の
符号」から、加算器２３においてシフタ２２の出力値と
レジスタＡ２４の値あるいはレジスタＢ２５の値のいず
れとを加算するかを制御回路２９において決定し、制御
信号Ｓ５によりスイッチ２６を介してレジスタＡ２４及
びレジスタＢ２５の出力のうちの一方が選択される。(4) From the "signal indicating that the operation using the i-th bit from the lower order of the synapse coefficient is being executed", the "bit value of the i-th bit from the lower order of the synapse coefficient" and the "sign of the input value", In the adder 23, the control circuit 29 determines whether to add the output value of the shifter 22 and the value of the register A24 or the value of the register B25, and the output of the register A24 and the register B25 via the switch 26 by the control signal S5. One of them is selected.
【００２３】(5) 制御回路２９より供給される制御信号
Ｓ６により、シフタ２２の出力にレジスタＡ２４又はレ
ジスタＢ２５のうちの選択された出力を加算するか否か
を制御する。(5) The control signal S6 supplied from the control circuit 29 controls whether or not the output selected by the register A24 or the register B25 is added to the output of the shifter 22.
【００２４】(6) 加算器２３においてシフタ２２の出力
値とＡＮＤ回路２８の出力との加算を行う。(6) The adder 23 adds the output value of the shifter 22 and the output of the AND circuit 28.
【００２５】(7) 「シナプス係数の下位からｉビット目
を用いた演算を実行中であるという信号」、「シナプス
係数の下位からｉビット目のビット値」及び「入力値の
符号」から、加算の結果をレジスタＡ２４に格納するか
レジスタＢ２５に格納するかを制御回路２９において決
定し、制御信号Ｓ３及び制御信号Ｓ４をトリガとしてレ
ジスタＡ２４又はレジスタＢ２５に加算結果を取り込
む。(7) From the “signal indicating that the operation using the i-th bit from the lower order of the synapse coefficient is being executed”, the “bit value of the i-th bit from the lower order of the synapse coefficient” and the “sign of the input value”, The control circuit 29 determines whether to store the addition result in the register A24 or the register B25, and the control signal S3 and the control signal S4 are used as triggers to load the addition result into the register A24 or the register B25.
【００２６】以上説明した回路における初期値設定法及
びビット演算法の詳細を以下に説明する。Details of the initial value setting method and the bit operation method in the circuit described above will be described below.
【００２７】(1) データは入力値、シナプス係数ともに
「２」の補数表現で演算回路に与えられる。ここで、各
値の語長（ビット数）をＬ、絶対値部分の下位からのビ
ット数をｉとする。(1) The data, both the input value and the synapse coefficient, are given to the arithmetic circuit in the complementary expression of "2". Here, the word length (the number of bits) of each value is L, and the number of bits from the lower part of the absolute value part is i.
【００２８】(2) 入力値とシナプス係数の符号ビット
（「２」の補数表現の最上位ビット）より、以下に示す
ように単調増加演算及び単調減少演算の累積加算値の初
期値を設定する。(2) From the input value and the sign bit of the synapse coefficient (the most significant bit of the complement expression of "2"), the initial value of the cumulative addition value of the monotone increasing operation and the monotone decreasing operation is set as shown below. .
【００２９】(a) 入力値、シナプス係数とも正の時：Ｌ
桁の「２」の補数で表現される正のシナプス係数の最小
値は“０”であり、Ｌ桁の「２」の補数で表現される正
のシナプス係数の最大値は“２(L-1) −１”である。従
って、シナプス演算の結果とり得る最小値は“０”、最
大値は“入力値×｛２(L-1) −１｝”である。そこで、
単調増加演算の累積加算値の初期値をシナプス演算の結
果とり得る最小値に設定し、単調減少演算の累積加算値
の初期値を−（シナプス演算の結果とり得る最大値）に
設定する。(A) When both input value and synapse coefficient are positive: L
The minimum value of the positive synaptic coefficient expressed by the digit “2” complement is “0”, and the maximum value of the positive synaptic coefficient expressed by the L digit “2” complement is “2 (L- 1) -1 ". Therefore, the minimum value that can be obtained as a result of the synapse calculation is "0", and the maximum value is "input value x {2 (L-1) -1}". Therefore,
The initial value of the cumulative addition value of the monotonic increase calculation is set to the minimum value that can be the result of the synapse calculation, and the initial value of the cumulative addition value of the monotonic decrease calculation is set to- (the maximum value that can be the result of the synapse calculation).
【００３０】図８を用いて、入力値とシナプス係数が
“４ビットの「２」の補数”で表現された場合について
説明する。この時、シナプス係数のとり得る最小値は
“０”であり、最大値は“７”である。従って、シナプ
ス演算の結果とり得る最小値は“０”、最大値は“３
５”となる。そこで、単調増加演算の累積加算値の初期
値は“０”に、単調減少演算の累積加算値の初期値は
“−３５”に設定する。A case in which the input value and the synapse coefficient are represented by "4 bits complement of" 2 "" will be described with reference to Fig. 8. At this time, the minimum value that the synapse coefficient can take is "0". , The maximum value is “7.” Therefore, the minimum value that can be obtained as a result of the synapse operation is “0” and the maximum value is “3”.
Therefore, the initial value of the cumulative addition value of the monotone increase calculation is set to "0", and the initial value of the cumulative addition value of the monotone decrease calculation is set to "-35".
【００３１】なお、単調減少演算については、単調増加
の演算となるように変換する。このため、単調減少演算
の累積加算値の初期値は「２」の補数をとって“−３
５”となるのである。他の場合も同様である。Note that the monotonic decrease calculation is converted to be a monotonic increase calculation. Therefore, the initial value of the cumulative addition value of the monotonic decrease calculation is the complement of "2" and is "-3".
5 ". The same applies to other cases.
【００３２】(b) 入力値は負、シナプス係数は正の時：
Ｌ桁の「２」の補数で表現される正のシナプス係数の最
小値は“０”であり、Ｌ桁の「２」の補数で表現される
正のシナプス係数の最大値は“２(L-1) −１”である。
従って、シナプス演算の結果とり得る最小値は“入力値
×｛２(L-1) −１｝”、最大値は“０”である。そこ
で、単調増加演算の累積加算値の初期値をシナプス演算
の結果とり得る最小値に設定し、単調減少演算の累積加
算値の初期値を−（シナプス演算の結果とり得る最大
値）に設定する。(B) When the input value is negative and the synapse coefficient is positive:
The minimum value of the positive synapse coefficient expressed by the L digit "2" complement is "0", and the maximum value of the positive synapse coefficient expressed by the L digit "2" complement is "2 (L -1) -1 ".
Therefore, the minimum value that can be obtained as a result of the synapse calculation is "input value x {2 (L-1) -1}", and the maximum value is "0". Therefore, the initial value of the cumulative addition value of the monotone increase operation is set to the minimum value that can be the result of the synapse operation, and the initial value of the cumulative addition value of the monotonic decrease operation is set to- (the maximum value that can be the result of the synapse operation). .
【００３３】図９を用いて、入力値とシナプス係数が
“４ビットの「２」の補数”で表現された場合について
説明する。この時、シナプス係数のとり得る最小値は
“０”であり、最大値は“７”である。従って、シナプ
ス演算の結果とり得る最小値は“−３５”、最大値は
“０”となる。そこで、単調増加演算の累積加算値の初
期値は“−３５”に、単調減少演算の累積加算値の初期
値は“０”に設定する。A case in which the input value and the synapse coefficient are represented by "4 bits complement of" 2 "" will be described with reference to Fig. 9. At this time, the minimum value that the synapse coefficient can take is "0". , The maximum value is “7.” Therefore, the minimum value that can be obtained as a result of the synapse operation is “−35”, and the maximum value is “0.” Therefore, the initial value of the cumulative addition value of the monotonic increase operation is “−”. 35 "and the initial value of the cumulative addition value of the monotonic decrease calculation is set to" 0 ".
【００３４】(c) 入力値は正、シナプス係数は負の時：
Ｌ桁の「２」の補数で表現される負のシナプス係数の最
小値は“−（２(L-1)）”であり、Ｌ桁の「２」の補数
で表現される負のシナプス係数の最大値は“−１”であ
る。従って、シナプス演算の結果とり得る最小値は“入
力値×｛−（２(L-1) ｝”、最大値は“−（入力値）”
である。そこで、単調増加演算の累積加算値の初期値を
シナプス演算の結果とり得る最小値に設定し、単調減少
演算の累積加算値の初期値を−（シナプス演算の結果と
り得る最大値）に設定する。(C) When the input value is positive and the synapse coefficient is negative:
The minimum value of the negative synapse coefficient expressed by the L digit "2" complement is "-(2 (L-1) )", and the negative synapse coefficient expressed by the L digit "2" complement is The maximum value of is "-1". Therefore, the minimum value that can be obtained as a result of synapse calculation is "input value x {-(2 (L-1) }", and the maximum value is "-(input value)".
Is. Therefore, the initial value of the cumulative addition value of the monotone increase operation is set to the minimum value that can be the result of the synapse operation, and the initial value of the cumulative addition value of the monotonic decrease operation is set to- (the maximum value that can be the result of the synapse operation). .
【００３５】図１０を用いて、入力値とシナプス係数が
“４ビットの「２」の補数”で表現された場合について
説明する。この時、シナプス係数のとり得る最小値は
“−８”であり、最大値は“−１”である。従って、シ
ナプス演算の結果とり得る最小値は“−４０”、最大値
は“−５”となる。そこで、単調増加演算の累積加算値
の初期値は“−４０”に、単調減少演算の累積加算値の
初期値は“５”に設定する。The case where the input value and the synapse coefficient are represented by "4 bits of complement of" 2 "" will be described with reference to Fig. 10. At this time, the minimum value that the synapse coefficient can take is "-8". Therefore, the maximum value is “−1.” Therefore, the minimum value that can be obtained as a result of the synapse operation is “−40”, and the maximum value is “−5.” Therefore, the initial value of the cumulative addition value of the monotonic increase operation Is set to "-40" and the initial value of the cumulative addition value of the monotonic decrease calculation is set to "5".
【００３６】(d) 入力値、シナプス係数ともに負の時：
Ｌ桁の「２」の補数で表現される負のシナプス係数の最
小値は“−（２(L-1)）”であり、Ｌ桁の「２」の補数
で表現される負のシナプス係数の最大値は“−１”であ
る。従って、シナプス演算の結果とり得る最小値は“−
（入力値）”、最大値は“入力値×｛−（２
(L-1) ）｝”である。そこで、単調増加演算の累積加算
値の初期値をシナプス演算の結果とり得る最小値に設定
し、単調減少演算の累積加算値の初期値を−（シナプス
演算の結果とり得る最大値）に設定する。(D) When both input value and synapse coefficient are negative:
The minimum value of the negative synapse coefficient expressed by the L digit "2" complement is "-(2 (L-1) )", and the negative synapse coefficient expressed by the L digit "2" complement is The maximum value of is "-1". Therefore, the minimum value that can be taken as a result of the synapse calculation is "-
(Input value), maximum value is "input value x {-(2
(L-1) )} ”. Therefore, the initial value of the cumulative addition value of the monotone increase operation is set to the minimum value that can be taken as a result of the synapse operation, and the initial value of the cumulative addition value of the monotone decrease operation is-(synapse Maximum value that can be obtained as a result of calculation).
【００３７】図１１を用いて、入力値とシナプス係数が
“４ビットの「２」の補数”で表現された場合について
説明する。この時、シナプス係数のとり得る最小値は
“−８”であり、最大値は“−１”である。従って、シ
ナプス演算の結果、とり得る最小値は“５”、最大値は
“４０”となる。そこで、単調増加演算の累積加算値の
初期値は“５”に、単調減少演算の累積加算値の初期値
は“−４０”に設定する。A case where the input value and the synapse coefficient are represented by "4 bits complement of" 2 "" will be described with reference to Fig. 11. At this time, the minimum value that the synapse coefficient can take is "-8". Therefore, the maximum value is “−1.” Therefore, as a result of the synapse operation, the minimum value that can be taken is “5” and the maximum value is “40.” Therefore, the initial value of the cumulative addition value of the monotonic increase operation is The initial value of the cumulative addition value of the monotonic decrease calculation is set to "-5" and "-40".
【００３８】(3) 入力値とシナプス係数の下位からｉビ
ット目のビット値より、以下のようにビット演算を行
う。(3) A bit operation is performed as follows from the input value and the bit value of the i-th bit from the lower order of the synapse coefficient.
【００３９】(a) 入力値が正でシナプス係数のビット値
が“１”の時：単調増加演算の累積加算値に“入力値×
２i ”を加算する。(A) When the input value is positive and the bit value of the synapse coefficient is "1": "input value x
2 i ”is added.
【００４０】(b) 入力値が正でシナプス係数のビット値
が“０”の時：単調減少演算の累積加算値に“入力値×
２i ”を加算する。(B) When the input value is positive and the bit value of the synapse coefficient is "0": "input value x
2 i ”is added.
【００４１】(c) 入力値が負でシナプス係数のビット値
が“１”の時：単調減少演算の累積加算値に“入力値の
「２」の補数×２i ”を加算する。[0041] (c) when the input value is the bit value of the synaptic coefficients in the negative is "1": the accumulated value of the monotonically decreasing operation adds "complement × 2 i" 2 "of the input values".
【００４２】(d) 入力値が負でシナプス係数のビット値
が“０”の時：単調増加演算の累積加算値に“入力値の
「２」の補数×２i ”を加算する。(D) When the input value is negative and the bit value of the synapse coefficient is “0”: “2's complement of input value × 2 i ” is added to the cumulative addition value of the monotone increasing operation.
【００４３】以上述べた処理により、図１２及び１３に
示すように、単調増加演算及び単調減少演算の累積加算
値は、演算が進むにつれて初期値から最終値に向かって
徐々に収束していく。By the above-described processing, as shown in FIGS. 12 and 13, the cumulative addition values of the monotonous increase calculation and the monotone decrease calculation gradually converge from the initial value toward the final value as the calculation progresses.
【００４４】この際、単調増加演算の累積加算値は、図
１２に示すように単調増加しながら最終値に収束するこ
とにより、飽和領域Ａに到達したか否かの判定を行う。
一方、単調減少演算の累積加算値は、図１３に示すよう
に単調減少しながら最終値に収束することにより、飽和
領域Ｂに到達したか否かの判定を行う。そして、累積加
算値が飽和領域Ａ又はＢに到達した時点で以降の演算を
省略する。At this time, the cumulative addition value of the monotonic increase calculation converges to the final value while monotonically increasing as shown in FIG. 12, thereby determining whether or not it has reached the saturation region A.
On the other hand, the cumulative addition value of the monotonic decrease calculation converges to the final value while monotonically decreasing as shown in FIG. 13 to determine whether or not it has reached the saturation region B. Then, when the cumulative addition value reaches the saturation region A or B, the subsequent calculation is omitted.
【００４５】[0045]
【実施例２】図１４は本発明の第２の実施例を示すもの
で、ここでは第１の実施例のシナプス演算及び累積回路
においてＡＮＤ回路を付加し、オフセット処理をするこ
とにより、累積加算値が飽和領域に到達したか否かの判
定を容易に実現できるようになした例を示す。即ち、図
中、３０はＡＮＤ回路であり、その入力にはレジスタＡ
２４及びレジスタＢ２５の最上位ビット値が入力されて
いる。[Embodiment 2] FIG. 14 shows a second embodiment of the present invention. In this embodiment, an AND circuit is added in the synapse calculation and accumulator circuit of the first embodiment and offset processing is performed to perform cumulative addition. An example in which it is possible to easily realize whether or not the value has reached the saturation region is shown. That is, in the figure, 30 is an AND circuit, whose input is the register A
24 and the most significant bit value of the register B25 are input.
【００４６】累積加算値が図１２又は１３に示す飽和領
域Ａ、過渡領域、飽和領域Ｂのどの領域にあるかを判定
するためには、現在の累積加算値と飽和領域境界値Ａ及
び飽和領域境界値Ｂとを比較すれば良い。ここで、飽和
領域境界値Ａとは飽和領域Ａと過渡領域との境界の値、
飽和領域境界値Ｂとは飽和領域Ｂと過渡領域との境界の
値である。しかし、この判定を行うためには、累積加算
値と飽和領域境界値Ａ及び累積加算値と飽和領域境界値
Ｂの大小関係を求めるために比較器が２つ必要となる。
今回、これを省略するために以下に述べる処理をとりい
れた。In order to determine which of the saturated area A, the transient area and the saturated area B shown in FIG. 12 or 13, the current cumulative added value, the saturated area boundary value A and the saturated area are used. Compare with the boundary value B. Here, the saturated region boundary value A is the value of the boundary between the saturated region A and the transient region,
The saturated region boundary value B is a value at the boundary between the saturated region B and the transient region. However, in order to make this determination, two comparators are required to obtain the magnitude relationship between the cumulative addition value and the saturation area boundary value A and between the cumulative addition value and the saturation area boundary value B.
This time, in order to omit this, the following processing was included.
【００４７】(1) 図１５に示すように、飽和領域Ａを判
定する単調増加演算の累積加算値の初期値から予めＳＨ
（飽和領域境界値Ａの値）を減じておき、単調減少演算
の累積加算値の初期値に予めＳＬ（飽和領域境界値Ｂの
値）を加算しておく。(1) As shown in FIG. 15, from the initial value of the cumulative addition value of the monotonically increasing calculation for determining the saturation area A, SH is calculated in advance.
(The value of the saturation region boundary value A) is subtracted, and SL (the value of the saturation region boundary value B) is added in advance to the initial value of the cumulative addition value of the monotonic decrease calculation.
【００４８】(2) このままでは、累積加算値が最終的に
正しい値を示さないので、しきい値処理を行う前に(1)
で加えた値、即ちオフセット値を取り除く。(2) If the value is left as it is, the cumulative addition value does not finally show a correct value. Therefore, before performing the threshold processing, (1)
The value added in step 2, that is, the offset value is removed.
【００４９】以上の処理を行うことにより、単調増加演
算の累積加算値が負から正に変化すれば飽和領域Ａ、単
調減少演算の累積加算値が負から正に変化すれば（実際
には符号を反転させて演算を行うため）飽和領域Ｂ、単
調増加演算の累積加算値、単調減少演算の累積加算値と
も負のままであれば過渡領域であると判定することがで
きる。これにより演算停止の判定を容易に行うことがで
きる。By performing the above processing, if the cumulative addition value of the monotonic increase calculation changes from negative to positive, the saturation region A, and if the cumulative addition value of the monotone decrease calculation changes from negative to positive (actually, the sign When the saturation region B, the cumulative addition value of the monotonic increase calculation, and the cumulative addition value of the monotonic decrease calculation remain negative, it can be determined to be the transient region. As a result, it is possible to easily determine whether to stop the calculation.
【００５０】そして、いずれかの累積加算値の値が正
（最上位ビットが“０”）に変化した時点で累積加算の
最終結果が判明するので以降の演算を省略し、演算の省
略及びシナプス係数メモリ回路へのアクセスの省略を実
現する。Then, when the value of any of the cumulative addition values changes to a positive value (the most significant bit is "0"), the final result of the cumulative addition is known. The omission of access to the coefficient memory circuit is realized.
【００５１】また、以上述べた演算方法において、単調
増加演算又は単調減少演算の初期値、オフセット値、ビ
ット演算時の加算値の符号を反転させた演算を行えば、
いずれかの累積加算値の値が負（最上位ビットが
“１”）に変化した時点で累積加算の最終結果が判明す
るので以降の演算を省略し、演算の省略及びシナプス係
数メモリ回路へのアクセスの省略を実現することができ
る。Further, in the above-mentioned arithmetic method, if the sign of the initial value of the monotonous increase operation or the monotone decrease operation, the offset value, and the added value at the time of bit operation is inverted,
Since the final result of cumulative addition is known when the value of any of the cumulative addition values changes to negative (most significant bit is “1”), the subsequent calculation is omitted, and the calculation is omitted and the synapse coefficient memory circuit Omission of access can be realized.
【００５２】本発明を採用した場合の演算の省略効果及
びシナプス係数メモリ回路へのアクセスの低減効果をシ
ミュレーションにより評価した結果を図１６及び図１７
に示す。FIGS. 16 and 17 show the results of evaluation by simulation of the effect of omitting operations and the effect of reducing access to the synapse coefficient memory circuit when the present invention is adopted.
Shown in
【００５３】図１６は入力数が１７個、入力値、シナプ
ス係数とも１６ビットの「２」の補数表現、飽和領域境
界値Ａ＝０．５、飽和領域境界値Ｂ＝−０．５の場合の
１ニューロンに対する全シナプス結合の演算を行った場
合について、最終的な累積加算値と演算実施率（本発明
を使用した場合の演算回数の本発明を使用しない場合の
演算回数に対する割合）との関係を示したものである。
この図より、過渡領域においては厳密な累積加算値が必
要となるため演算省略の効果はないが、最終的な累積加
算値が飽和領域にある場合には、飽和領域境界値から離
れると演算省略の効果が顕著に現われることが分かる。FIG. 16 shows a case where the number of inputs is 17, the input value and the synapse coefficient are 16-bit "2" complement representations, the saturation region boundary value A = 0.5, and the saturation region boundary value B = -0.5. In the case where the calculation of all synapse connections for one neuron is performed, the final cumulative addition value and the calculation execution rate (ratio of the number of calculations when the present invention is used to the number of calculations when the present invention is not used) It shows the relationship.
From this figure, there is no effect of omitting the calculation because the strict cumulative addition value is required in the transient region, but if the final cumulative addition value is in the saturation region, the calculation is skipped if it deviates from the saturation region boundary value. It can be seen that the effect of appears significantly.
【００５４】また、図１７は同様の演算における、最終
的な累積加算値とシナプス係数を上位何ビットアクセス
したかの関係を示したものである。この図より、最終的
な累積加算値が飽和領域境界値から離れるとシナプス係
数メモリ回路へのアクセスの低減効果が顕著に現われる
ことが分かる。FIG. 17 shows the relationship between the final cumulative addition value and the number of upper bits of the synapse coefficient accessed in the same operation. From this figure, it can be seen that the effect of reducing access to the synapse coefficient memory circuit appears remarkably when the final cumulative addition value deviates from the saturation region boundary value.
【００５５】具体的に演算省略の効果を見積ってみる。
最終的な累積加算値が飽和領域になる確率を９０％、過
渡領域になる確率を１０％とし、最終的な累積加算値が
飽和領域になる時の平均演算実施率を図１６より２０％
とすれば、演算量を約２８％に省略することができる。The effect of omission of calculation will be specifically estimated.
The probability that the final cumulative added value will be in the saturated region is 90%, the probability that it will be in the transient region is 10%, and the average calculation execution rate when the final cumulative added value is in the saturated region is 20% from FIG.
Then, the calculation amount can be reduced to about 28%.
【００５６】また、同様にシナプス係数メモリ回路への
アクセスも、最終的な累積加算値が飽和領域になる確率
を９０％、過渡領域になる確率を１０％とし、最終的な
累積加算値が飽和領域になる時の平均アクセスビット数
を図１７より４ビットとすれば、シナプス係数メモリ回
路へのアクセスを約３１％に低減できる。Similarly, in the access to the synapse coefficient memory circuit, the probability that the final cumulative addition value is in the saturation region is 90%, the probability that it is in the transient region is 10%, and the final cumulative addition value is saturated. If the average number of access bits when becoming a region is 4 bits as shown in FIG. 17, the access to the synapse coefficient memory circuit can be reduced to about 31%.
【００５７】[0057]
【発明の効果】以上説明したように本発明によれば、演
算量の削減及びシナプス係数メモリ回路へのアクセス量
の削減が可能となり、シナプス演算の高速化及び演算に
要する消費電力の低減が可能となる。これにより、大規
模で且つ高速動作可能なディジタルニューラルネットワ
ーク回路を構成することができる。As described above, according to the present invention, it is possible to reduce the amount of calculation and the amount of access to the synapse coefficient memory circuit, and it is possible to speed up the synapse calculation and reduce the power consumption required for the calculation. Becomes This makes it possible to construct a large-scale digital neural network circuit that can operate at high speed.
【図１】本発明の第１の実施例を示す構成図FIG. 1 is a configuration diagram showing a first embodiment of the present invention.
【図２】ニューロン回路の構成図FIG. 2 is a block diagram of a neuron circuit
【図３】しきい値回路の伝達特性の説明図FIG. 3 is an explanatory diagram of transfer characteristics of a threshold circuit.
【図４】３層構造のニューラルネットワーク回路の構成
図FIG. 4 is a block diagram of a neural network circuit having a three-layer structure.
【図５】従来のニューラルネットワーク回路の具体的な
構成図FIG. 5 is a specific configuration diagram of a conventional neural network circuit.
【図６】シナプス演算及び累積回路の詳細を示す構成図FIG. 6 is a configuration diagram showing details of a synapse calculation and accumulation circuit.
【図７】本発明におけるシナプス演算法の説明図FIG. 7 is an explanatory diagram of a synapse calculation method according to the present invention.
【図８】第１の実施例における演算過程の説明図FIG. 8 is an explanatory diagram of a calculation process in the first embodiment.
【図９】第１の実施例における演算過程の説明図FIG. 9 is an explanatory diagram of a calculation process in the first embodiment.
【図１０】第１の実施例における演算過程の説明図FIG. 10 is an explanatory diagram of a calculation process in the first embodiment.
【図１１】第１の実施例における演算過程の説明図FIG. 11 is an explanatory diagram of a calculation process in the first embodiment.
【図１２】飽和領域の判定動作の説明図FIG. 12 is an explanatory diagram of a saturated region determination operation.
【図１３】飽和領域の判定動作の説明図FIG. 13 is an explanatory diagram of a saturated region determination operation.
【図１４】本発明の第２の実施例を示す構成図FIG. 14 is a configuration diagram showing a second embodiment of the present invention.
【図１５】第２の実施例における飽和領域判定原理の説
明図FIG. 15 is an explanatory diagram of a saturated region determination principle in the second embodiment.
【図１６】最終的な積和演算結果と演算実施率との関係
を示す説明図FIG. 16 is an explanatory diagram showing the relationship between the final product-sum operation result and the operation execution rate.
【図１７】最終的な積和演算結果と演算に使用したシナ
プス係数のビット数との関係を示す説明図FIG. 17 is an explanatory diagram showing the relationship between the final product-sum operation result and the number of bits of the synapse coefficient used for the operation.
１…セレクタ、２…シナプス演算及び累積回路、３…シ
ナプス係数メモリ回路、４…メモリ読み出し回路、５…
しきい値回路、６…判定回路、７…クロック発生回路、
８…制御回路、２１…「２」の補数変換回路、２２…シ
フタ、２３…加算器、２４…レジスタＡ、２５…レジス
タＢ、２６…スイッチ、２７…レジスタＣ、２８，３０
…ＡＮＤ回路、２９…制御回路。1 ... Selector, 2 ... Synapse calculation and accumulation circuit, 3 ... Synapse coefficient memory circuit, 4 ... Memory reading circuit, 5 ...
Threshold circuit, 6 ... Judgment circuit, 7 ... Clock generation circuit,
8 ... Control circuit, 21 ... "2" complement conversion circuit, 22 ... Shifter, 23 ... Adder, 24 ... Register A, 25 ... Register B, 26 ... Switch, 27 ... Register C, 28, 30
... AND circuit, 29 ... Control circuit.
Claims (6)
該多数の入力値に対応する多数のシナプス係数との間で
それぞれシナプス演算を行い、それらの結果を累積加算
し、さらにしきい値処理を行って出力値を決定するニュ
ーラルネットワーク回路において、 一の入力値と該一の入力値に対応するシナプス係数との
シナプス演算を、入力値の全ビットとシナプス係数の任
意の１ビットとの間で行い、得られた部分積を加算し、
これを各入力値及びシナプス係数毎に行って累積加算す
るシナプス演算及び累積回路と、 シナプス係数を格納したシナプス係数メモリ回路と、 シナプス係数メモリ回路から任意のシナプス係数の任意
のビットをシナプス演算及び累積回路に転送するメモリ
読み出し回路と、 シナプス演算及び累積回路の出力に所定のしきい値処理
を実行するしきい値回路と、 シナプス演算及び累積回路の出力がしきい値回路の伝達
特性の飽和領域に到達したか否かを判定する判定回路
と、 シナプス演算をシナプス係数の上位ビットから実行さ
せ、累積加算値が飽和領域に達した時点で停止させる制
御回路とを備えたことを特徴とするニューラルネットワ
ーク回路。1. A synapse operation is performed between a large number of input values represented by a binary bit string and a large number of synapse coefficients corresponding to the large number of input values, and the results are cumulatively added, and a threshold value is further added. In a neural network circuit that performs value processing to determine an output value, a synapse operation between one input value and a synapse coefficient corresponding to the one input value is performed by using all bits of the input value and any one bit of the synapse coefficient. And add the partial products obtained,
This is performed for each input value and synapse coefficient and cumulative addition is performed. A synapse operation and accumulation circuit is performed. Memory read circuit that transfers to accumulator circuit, threshold circuit that executes predetermined threshold processing on synapse operation and output of accumulator circuit, and output of synapse operation and accumulator circuit saturates transfer characteristic of threshold circuit It is characterized by comprising a determination circuit for determining whether or not the region has been reached, and a control circuit for executing a synapse operation from the upper bits of the synapse coefficient and stopping when the cumulative addition value reaches the saturation region. Neural network circuit.
シフトさせるシフタと、シフタの出力と累積加算値との
加算を行う加算器と、累積加算値格納用のレジスタとに
より、入力値とシナプス係数とのシナプス演算を行うシ
ナプス演算及び累積回路を用い、 シナプス演算の積演算過程で必要な入力値の全ビットと
シナプス係数の任意の１ビットとの部分積の加算及び累
積加算に必要な加算を共通の加算器で実行可能としたこ
とを特徴とする請求項１記載のニューラルネットワーク
回路。2. An input value is provided by a shifter that shifts all bits of the input value by an arbitrary number of bits, an adder that adds the output of the shifter and the cumulative addition value, and a register for storing the cumulative addition value. Using the synapse operation and accumulator circuit that performs synapse operation with the synapse coefficient, it is necessary to add partial products of all bits of the input value required for the product operation process of synapse operation and any one bit of synapse coefficient and to perform cumulative addition. The neural network circuit according to claim 1, wherein the addition can be executed by a common adder.
回路を用いた演算方法において、 単調増加演算及び単調減少演算の累積加算値を計算し、
判定回路において累積加算値としきい値回路の伝達特性
の飽和領域及び過渡領域の境界値とを比較することによ
り、２つ以上の飽和領域の判定を行う際、 シナプス演算及び累積回路を、入力値の全ビットを補数
に変換することが可能な補数変換回路と、補数変換回路
の出力に対して任意のビット数だけシフトさせるシフタ
と、シフタの出力を単調増加演算の累積加算値あるいは
単調減少演算の累積加算値に加算する加算器と、単調増
加演算の累積加算値格納用のレジスタ及び単調減少演算
の累積加算値格納用のレジスタとにより構成し、 単調増加演算の累積加算値の初期値をシナプス演算の結
果とり得る最小値に設定し、単調減少演算の累積加算値
の初期値をシナプス演算の結果とり得る最大値の符号を
反転させた値に設定し、 入力値の全ビットとシナプス係数の下位からｉビット目
との演算を行う際は、入力値が正の時には“入力値×２
i ”を、シナプス係数のビット値が“１”の時には単調
増加演算の累積加算値に加算し、シナプス係数のビット
値が“０”の時には単調減少演算の累積加算値に加算
し、入力値が負の時には“入力値に極性反転を施した値
×２i ”を、シナプス係数のビット値が“１”の時には
単調減少演算の累積加算値に加算し、シナプス係数のビ
ット値が“０”の時には単調増加演算の累積加算値に加
算することにより、単調増加演算及び累積減少演算の累
積加算値を計算することを特徴とするニューラルネット
ワーク回路を用いた演算方法。3. A calculation method using a neural network circuit according to claim 2, wherein cumulative addition values of a monotone increase calculation and a monotone decrease calculation are calculated,
The judgment circuit compares the cumulative addition value with the boundary values of the saturation region and the transient region of the transfer characteristic of the threshold circuit to determine the two or more saturation regions. Complement conversion circuit that can convert all bits of to the complement, shifter that shifts the output of complement conversion circuit by any number of bits, and cumulative addition value or monotonic decrease operation of shifter output It is configured with an adder that adds to the cumulative addition value of, the register for storing the cumulative addition value of the monotonic increase operation, and the register for storing the cumulative addition value of the monotonic decrease operation. Set it to the minimum value that can be taken as a result of synapse operation, and set the initial value of the cumulative addition value of monotonic decrease operation to the value that is the inverted sign of the maximum value that can be taken as a result of synapse operation. Tsu Doo and when performing the computation of the i-th bit from the lower synaptic coefficients, when the input value is positive "input value × 2
i ”is added to the cumulative addition value of the monotone increasing operation when the bit value of the synapse coefficient is“ 1 ”, and is added to the cumulative addition value of the monotonic decreasing operation when the bit value of the synapse coefficient is“ 0 ” Is negative, the value obtained by reversing the polarity of the input value x 2 i is added to the cumulative addition value of the monotonic decrease operation when the bit value of the synapse coefficient is "1", and the bit value of the synapse coefficient is "0". In the case of ", a calculation method using a neural network circuit is characterized in that the cumulative addition value of the monotonic increase calculation and the cumulative decrease calculation is calculated by adding to the cumulative addition value of the monotone increase calculation.
回路を用いた演算方法において、 単調増加演算の累積加算値の初期値を「（請求項３記載
の単調増加演算の累積加算値の初期値）−（過渡領域の
上限の値）」に設定し、 単調減少演算の累積加算値の初期値を「（請求項３記載
の単調減少演算の累積加算値の初期値）＋（過渡領域の
下限の値）」に設定することにより、 単調増加演算の累積加算値又は単調減少演算の累積加算
値の最上位ビットのいずれかの値が“１”から“０”に
変化した場合に飽和領域であると判定して演算を停止さ
せることを特徴とするニューラルネットワーク回路を用
いた演算方法。4. The calculation method using the neural network circuit according to claim 3, wherein the initial value of the cumulative addition value of the monotone increase operation is “(the initial value of the cumulative addition value of the monotone increase operation of claim 3)- (The upper limit value of the transient region) ", and the initial value of the cumulative addition value of the monotone decrease calculation is" (the initial value of the cumulative addition value of the monotone decrease calculation according to claim 3) + (the lower limit value of the transient region ) ”, It means that when either the most significant bit of the cumulative addition value of the monotonic increase calculation or the cumulative addition value of the monotonic decrease operation changes from“ 1 ”to“ 0 ”, it means that it is in the saturated region. A calculation method using a neural network circuit, which is characterized by stopping the calculation.
回路を用いた演算方法において、 単調増加演算及び単調減少演算の累積加算値を計算し、
判定回路において累積加算値としきい値回路の伝達特性
の飽和領域及び過渡領域の境界値とを比較することによ
り、２つ以上の飽和領域の判定を行う際、 シナプス演算及び累積回路を、入力値の全ビットを補数
に変換することが可能な補数変換回路と、補数変換回路
の出力に対して任意のビット数だけシフトさせるシフタ
と、シフタの出力を単調増加演算の累積加算値あるいは
単調減少演算の累積加算値に加算する加算器と、単調増
加演算の累積加算値格納用のレジスタ及び単調減少演算
の累積加算値格納用のレジスタとにより構成し、 単調増加演算の累積加算値の初期値をシナプス演算の結
果とり得る最小値の符号を反転させた値に設定し、単調
減少演算の累積加算値の初期値をシナプス演算の結果と
り得る最大値に設定し、 入力値の全ビットとシナプス係数の下位からｉビット目
との演算を行う際は、入力値が正の時には“入力値に極
性反転を施した値×２i ”を、シナプス係数のビット値
が“１”の時には単調増加演算の累積加算値に加算し、
シナプス係数のビット値が”０”の時には単調減少演算
の累積加算値に加算し、入力値が負の時には“入力値×
２i ”を、シナプス係数のビット値が“１”の時には単
調減少演算の累積加算値に加算し、シナプス係数のビッ
ト値が“０”の時には単調増加演算の累積加算値に加算
することにより、単調増加演算及び単調減少演算の累積
加算値を計算することを特徴とするニューラルネットワ
ーク回路を用いた演算方法。5. A calculation method using a neural network circuit according to claim 2, wherein cumulative addition values of a monotone increase calculation and a monotone decrease calculation are calculated,
The judgment circuit compares the cumulative addition value with the boundary values of the saturation region and the transient region of the transfer characteristic of the threshold circuit to determine the two or more saturation regions. Complement conversion circuit that can convert all bits of to the complement, shifter that shifts the output of complement conversion circuit by any number of bits, and cumulative addition value or monotonic decrease operation of shifter output It is configured with an adder that adds to the cumulative addition value of, the register for storing the cumulative addition value of the monotonic increase operation, and the register for storing the cumulative addition value of the monotonic decrease operation. Set the sign of the minimum value that can be obtained as a result of synapse operation to the inverted value, set the initial value of the cumulative addition value of monotonic decrease operation to the maximum value that can be obtained as a result of synapse operation, and When performing a calculation from the lower Tsu preparative synaptic coefficient and i-th bit, when the input value is positive the "value × 2 i subjected to polarity inversion in the input value", the bit values of the synaptic coefficients "1" When is added to the cumulative addition value of the monotonically increasing operation,
When the bit value of the synapse coefficient is "0", it is added to the cumulative addition value of the monotonic decrease operation, and when the input value is negative, "input value x
2 i ”is added to the cumulative addition value of the monotone decrease operation when the bit value of the synapse coefficient is“ 1 ”, and is added to the cumulative addition value of the monotone increase operation when the bit value of the synapse coefficient is“ 0 ”. , A calculation method using a neural network circuit, which calculates a cumulative addition value of a monotone increase calculation and a monotone decrease calculation.
回路を用いた演算方法において、 単調増加演算の累積加算値の初期値を「（請求項５記載
の単調増加演算の累積加算値の初期値）＋（過渡領域の
上限の値）」に設定し、 単調減少演算の累積加算値の初期値を「（請求項５記載
の単調減少演算の累積加算値の初期値）−（過渡領域の
下限の値）」に設定することにより、 単調増加演算の累積加算値又は単調減少演算の累積加算
値の最上位ビットのいずれかの値が“０”から“１”に
変化した場合に飽和領域であると判定して演算を停止さ
せることを特徴とするニューラルネットワーク回路を用
いた演算方法。6. The calculation method using the neural network circuit according to claim 5, wherein the initial value of the cumulative addition value of the monotonic increase calculation is “(the initial value of the cumulative addition value of the monotone increase calculation of claim 5) + (The upper limit value of the transient region) ", and the initial value of the cumulative addition value of the monotone decrease calculation is" (the initial value of the cumulative addition value of the monotone decrease calculation according to claim 5)-(the lower limit value of the transient region. ) ”, It means that when either the most significant bit of the cumulative addition value of the monotonic increase operation or the cumulative addition value of the monotonic decrease operation changes from“ 0 ”to“ 1 ”, it means that it is in the saturated region. A calculation method using a neural network circuit, which is characterized by stopping the calculation.
Priority Applications (6)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP6165641A JPH0830572A (en) | 1994-07-18 | 1994-07-18 | Neural network circuit and calculation method using the same |
| US08/373,440 US5630024A (en) | 1994-01-19 | 1995-01-17 | Method and apparatus for processing using neural network with reduced calculation amount |
| KR1019950000739A KR0168975B1 (en) | 1994-01-19 | 1995-01-18 | Neural network, neural network system and calculation method using same |
| EP95100620A EP0664516B1 (en) | 1994-01-19 | 1995-01-18 | Neural network with reduced calculation amount |
| DE69531418T DE69531418T2 (en) | 1994-01-19 | 1995-01-18 | Neural network with reduced amount of calculations |
| EP00113124A EP1033660A1 (en) | 1994-01-19 | 1995-01-18 | Method and apparatus for processing using neural network with reduced calculation amount |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP6165641A JPH0830572A (en) | 1994-07-18 | 1994-07-18 | Neural network circuit and calculation method using the same |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| JPH0830572A true JPH0830572A (en) | 1996-02-02 |
Family
ID=15816227
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| JP6165641A Pending JPH0830572A (en) | 1994-01-19 | 1994-07-18 | Neural network circuit and calculation method using the same |
Country Status (1)
| Country | Link |
|---|---|
| JP (1) | JPH0830572A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH10187648A (en) * | 1996-12-19 | 1998-07-21 | Nec Corp | Neural unit operation system and its device |
| JP2018120433A (en) * | 2017-01-25 | 2018-08-02 | 株式会社東芝 | Product-sum operator, network unit and network device |
| JP2023026404A (en) * | 2021-08-13 | 2023-02-24 | 台湾積體電路製造股▲ふん▼有限公司 | Cumulative sum of bit unit having skip logic |
-
1994
- 1994-07-18 JP JP6165641A patent/JPH0830572A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH10187648A (en) * | 1996-12-19 | 1998-07-21 | Nec Corp | Neural unit operation system and its device |
| JP2018120433A (en) * | 2017-01-25 | 2018-08-02 | 株式会社東芝 | Product-sum operator, network unit and network device |
| US10853721B2 (en) | 2017-01-25 | 2020-12-01 | Kabushiki Kaisha Toshiba | Multiplier accumulator, network unit, and network apparatus |
| JP2023026404A (en) * | 2021-08-13 | 2023-02-24 | 台湾積體電路製造股▲ふん▼有限公司 | Cumulative sum of bit unit having skip logic |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102285965B1 (en) | Method and electronic device for convolution calculation in neutral network | |
| US11222254B2 (en) | Optimized neuron circuit, and architecture and method for executing neural networks | |
| CN112162723B (en) | Quantum subtraction method, device, electronic device and storage medium | |
| CN108364064B (en) | Method, device and system for operating neural network | |
| US5095457A (en) | Digital multiplier employing CMOS transistors | |
| JP4620943B2 (en) | Product-sum operation circuit and method thereof | |
| JP3634023B2 (en) | An adaptive weight adjustment circuit for neural networks | |
| CN113850389B (en) | Quantum circuit construction method and device | |
| CN107862374A (en) | Processing with Neural Network system and processing method based on streamline | |
| KR950001601B1 (en) | Neural network circuit | |
| US5630024A (en) | Method and apparatus for processing using neural network with reduced calculation amount | |
| JP2019139338A (en) | Information processor, information processing method and program | |
| CN112561050A (en) | Neural network model training method and device | |
| JPWO1999066423A1 (en) | Data Processing Unit | |
| JP2022164559A (en) | Memory device and method for operating memory device | |
| JPH0830572A (en) | Neural network circuit and calculation method using the same | |
| CN114239818B (en) | Memory computing architecture neural network accelerator based on TCAM and LUT | |
| JPWO1999031573A1 (en) | Semiconductor circuit for arithmetic processing and arithmetic processing method | |
| JPH07210533A (en) | Neural network circuit and calculation method using the same | |
| JP2020013455A (en) | Information processing device performing convolution arithmetic processing in layer of convolution neural network | |
| TWI844108B (en) | Integrated circuit and operation method | |
| JPH09101944A (en) | Neural network circuit | |
| KR102339896B1 (en) | Processing element, method of operation thereof, and accelerator including the same | |
| CN114548415A (en) | Quantum FAST corner detection method based on NEQR expression | |
| CN116992932A (en) | A parametric LSTM acceleration system for off-chip data block transmission and its design method |