WO2018024232A1 - 用于执行神经网络运算的装置及方法 - Google Patents

用于执行神经网络运算的装置及方法 Download PDF

Info

Publication number
WO2018024232A1
WO2018024232A1 PCT/CN2017/095810 CN2017095810W WO2018024232A1 WO 2018024232 A1 WO2018024232 A1 WO 2018024232A1 CN 2017095810 W CN2017095810 W CN 2017095810W WO 2018024232 A1 WO2018024232 A1 WO 2018024232A1
Authority
WO
WIPO (PCT)
Prior art keywords
neural network
unit
data
network processing
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2017/095810
Other languages
English (en)
French (fr)
Inventor
陈云霁
刘少礼
韩栋
陈天石
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Cambricon Information Technology Co Ltd
Original Assignee
Shanghai Cambricon Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Cambricon Information Technology Co Ltd filed Critical Shanghai Cambricon Information Technology Co Ltd
Priority to EP17836414.7A priority Critical patent/EP3496007B1/en
Publication of WO2018024232A1 publication Critical patent/WO2018024232A1/zh
Priority to US16/268,468 priority patent/US11120331B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
    • G06F15/163—Interprocessor communication
    • G06F15/173—Interprocessor communication using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
    • G06F15/17306—Intercommunication techniques
    • G06F15/17325—Synchronisation; Hardware support therefor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0499—Feedforward networks

Definitions

  • the present disclosure belongs to the field of neural network operations, and in particular, to an apparatus and method for performing neural network operations.
  • One known method of supporting multi-core multi-layer neural network operations is to use a general purpose processor.
  • the method supports the above algorithm by executing general purpose instructions using a general purpose register file and generic functions.
  • One of the disadvantages of this method is that the performance of a single general-purpose processor is low and cannot meet the performance requirements of the usual multi-core multi-layer artificial neural network operation.
  • communication between general-purpose processors becomes a performance bottleneck.
  • the general-purpose processor needs to decode the artificial neural network into a long column operation and a sequence of fetch instructions, and the processor front-end decoding brings a large power consumption overhead.
  • Another known method of supporting multi-core multi-layer artificial neural network operations is to use a graphics processing unit (GPU).
  • the method supports the above algorithm by executing a generic SIMD instruction using a general purpose register file and a generic stream processing unit. Since the GPU is a device dedicated to performing graphic image operations and scientific calculations, without the special support for artificial neural network operations, a large amount of front-end decoding work is still required to perform multi-layer artificial neural network operations, which brings a lot of overhead. .
  • the GPU has only a small on-chip cache, and the model data (weight) of the multi-layer artificial neural network needs to be repeatedly transferred from off-chip, and the off-chip bandwidth becomes the main performance bottleneck.
  • the present disclosure provides an apparatus and method for performing neural network operations, which can implement operations of one or more layers of multi-core multi-layer artificial neural networks in a low-cost manner, and has high computational performance.
  • the present disclosure provides an apparatus and method for performing neural network operations, the apparatus comprising an on-chip interconnecting module and a plurality of neural network processing modules communicatively coupled to the on-chip interconnecting unit,
  • the network processing module can read and write data from other neural network processing modules through the on-chip interconnect module.
  • each layer of neural network operation is divided, and then processed by multiple neural network processing modules to obtain the respective operation result data, and multiple neural network processing units will also perform respective operations.
  • the resulting data is exchanged for data.
  • each neural network processing module only calculates part of the output data, and each neural network processing module needs to be processed from other neural networks when performing the next layer of neural network operations.
  • the data of the module therefore, each neural network processing needs to send the calculated operation result data to the corresponding neural network processing module, so as to calculate the neural network of the next layer.
  • the neural network processing module can read and write data from other neural network processing modules through the on-chip interconnect module, and can also read and write data from the local.
  • the neural network processing module includes a neural network processing unit and a high speed storage unit; the neural network processing unit is configured to read and write data, and the high speed storage unit is configured to store local data.
  • the apparatus for performing neural network operations further includes an external storage module, and the neural network processing module is further capable of reading and writing data from the external storage module through the on-chip interconnecting unit.
  • the neural network processing unit includes an instruction queue, a cache unit, an IO reading unit, and a neural network operation unit, wherein:
  • the instruction queue stores an operation instruction
  • the IO reading unit reads data from the outside of the neural network processing unit according to the operation instruction, and buffers the read data into the cache unit, and the neural network operation unit according to the operation instruction Reading the cached data in the cache unit, and performing a neural network operation to obtain operation result data;
  • the neural network processing unit further includes a synchronization relationship unit, and the instruction queue further stores a data delivery instruction.
  • the synchronization relationship unit corresponding to the neural network operation unit executes the data.
  • the instruction is sent to send a data delivery signal to the synchronization relationship unit corresponding to the other neural network operation unit.
  • the instruction queue further stores a data dependency instruction, and after the neural network operation unit receives the data sent by the other neural network operation unit, the corresponding synchronization relationship unit executes the data dependency instruction to detect whether the data is received. Delivery signal, if yes, continue Execute the instruction in the execution instruction queue, otherwise block the instruction queue.
  • the instruction queue further stores a data synchronization instruction
  • the synchronization relationship unit in the neural network processing unit sends a synchronization signal to the synchronization relationship unit in the other neural network processing unit by executing the data synchronization instruction to force multiple nerves.
  • the network processing unit does the synchronization operation.
  • the on-chip interconnecting module includes a first-level interconnecting module and a plurality of second-level interconnecting modules communicably connected to the first-level interconnecting module, and the first-level interconnecting module is further communicably connected to the external storage module, and the plurality of second-level interconnecting modules and The plurality of neural network processing modules are in one-to-one correspondence, wherein each of the secondary interconnecting modules is respectively communicatively coupled to the neural network processing unit and the high speed storage unit in the corresponding neural network processing module.
  • the present disclosure also provides a method for performing a single layer neural network operation, comprising:
  • each of the plurality of neural network processing modules directly reads data from the local, and/or reads data from other neural network processing modules through the on-chip interconnect module, wherein the plurality of neural network processing modules and The on-chip interconnect unit communication connection;
  • each neural network processing module performs partial operations of the single layer neural network according to the read data, and obtains respective operation result data;
  • Each neural network processing module locally stores the respective operation result data and/or writes the respective operation result data into other neural network processing modules through the on-chip interconnect module.
  • each neural network processing module sends a data delivery signal to other specific neural network processing modules after writing the respective operation results to other neural network processing modules.
  • the present disclosure also provides a method for performing a multi-layer neural network operation. For each layer of neural network operation, the above steps S1-S3 are performed, and the operation result data obtained by each layer of the neural network processing module is used for the next step. Layer neural network operation.
  • a single-layer neural network is allowed to divide tasks, execute on multiple neural network processing modules, and use dedicated instructions to allow When performing a multi-layer neural network, the calculated data can be transmitted between multiple neural network processors, so that multi-core multi-core neural network operations can be realized.
  • FIG. 1 is a schematic structural diagram of an apparatus for performing a neural network operation provided by the present disclosure
  • FIG. 2 is a schematic structural diagram of a neural network processing module in the present disclosure
  • FIG. 3 is a schematic structural diagram of an external storage module in the present disclosure
  • FIG. 4 is a schematic structural diagram of a neural network processing unit in the present disclosure
  • FIG. 5 is a schematic structural diagram of an on-chip interconnecting unit in the present disclosure.
  • FIG. 6 is a flow diagram of an embodiment of the present disclosure performing a full connectivity layer operation.
  • the apparatus for performing artificial neural network operation can be applied to the following (including but not limited to) scenarios: data processing, robot, computer, printer, scanner, telephone, tablet, smart terminal, mobile phone, driving record Instruments, navigators, sensors, cameras, cloud servers, cameras, cameras, projectors, watches, headsets, mobile storage, wearable devices and other electronic products; aircraft, ships, vehicles and other types of transportation; television, air conditioning, Microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, range hoods and other household appliances; and various types of medical equipment including nuclear magnetic resonance instruments, B-ultrasounds, electrocardiographs, etc.
  • FIG. 1 is a schematic structural diagram of an apparatus for performing a neural network operation provided by the present disclosure.
  • the apparatus includes a plurality of neural network processing modules 10 and an on-chip interconnecting module 20, and a plurality of neural network processing modules 10 and The on-chip interconnect unit 20 is communicatively coupled, wherein:
  • the neural network processing module 10 can read and write data from other neural network processing modules 10 through the on-chip interconnect module 30, and can also read and write data from the local.
  • each neural network processing module 10 performs a corresponding operation as a core, and the data required for the operation can be directly obtained directly from the local, or through the on-chip interconnecting module 20 and other neural network processing modules 10 Communication to read data required for operations from other neural network processing modules 10.
  • the corresponding operation is performed to obtain the respective operation result data.
  • each neural network processing module 10 can summarize the respective operation result data to A neural network processing module 10 performs accumulation to obtain final result data.
  • the data required for the operation in the next layer may be used by the other neural network processing module 10, so that after the operation of the layer neural network is completed, Each neural network processing module 10 performs data interaction to prepare for the next layer of neural network operations.
  • the neural network processing module 10 includes a neural network processing unit 11 and a high speed storage unit 12; when the neural network processing module 10 performs neural network operations, the neural network The network processing unit 11 reads data directly from its corresponding high speed storage unit 12, and/or reads data from the neural network processing unit 11 in other neural network processing modules 10 via the on-chip interconnecting unit 20, and/or through on-chip
  • the interconnecting unit 20 reads data from the high speed storage unit 12 in the other neural network processing module 10; the neural network processing unit 11 in each neural network processing module 10 performs neural network operations based on the read data to obtain respective operation results.
  • the neural network processing unit 11 directly writes the operation result data into the corresponding high-speed storage unit 12, and/or writes the operation result data to the other neural network processing module 10 through the on-chip interconnecting unit 20.
  • the operation result data in the neural network processing unit 11 and/or through the on-chip interconnect unit 20 It is written to the high speed storage unit 12 in the other neural network processing module 10.
  • the neural network processing unit 11 can directly acquire data from its corresponding high-speed storage unit, and can also acquire data of other locations through the on-chip interconnecting module 20, thereby avoiding repeatedly reading data into the memory and reducing the memory access bandwidth.
  • the apparatus for performing neural network operations provided by the present disclosure further includes an outer
  • the storage module 30 is communicatively coupled to the on-chip interconnecting unit 20, and the neural network processing module 10 is also capable of reading and writing data from the external storage module through the on-chip interconnecting unit, and the external storage module 30 can be used to import new data from the outside to the device.
  • the final execution result data executed by the device can also be written to the external storage module 30 for external export.
  • the external storage module 30 can be implemented by hardware (including but not limited to FPGA, CGRA, application specific integrated circuit ASIC, analog circuit or memristor, etc.).
  • the neural network processing unit 11 includes an instruction queue 111, a neural network operation unit 112, an IO reading unit 113, a cache unit 114, and a synchronization relationship.
  • Unit 115 The instruction queue 111 stores a plurality of types of instructions, and the neural network processing unit 11 performs different operations in accordance with different instructions. The following table describes the various types of instructions:
  • the instruction includes the instruction name and multiple opcodes:
  • the data delivery instruction, the instruction name is ACK, wherein each operation code respectively indicates whether to send a data delivery signal (ACK signal) to the neural network processing unit 11, and the neural network processing unit 11 writes data to the other neural network processing unit 11 And executing a data delivery instruction to send a data delivery signal to the corresponding neural network processing unit 11 to indicate that the data has been transmitted into place;
  • the data depends on the instruction, the instruction name is FENCE, wherein each operation code indicates whether to check the ACK signal from the neural network processing unit 11; the neural network processing unit 11 executes the data dependency instruction to detect whether all of its dependent data has arrived at the neural network processing unit.
  • Data synchronization instruction the instruction name is SYNC, wherein each operation code represents the neural network Whether the network processing unit participates in the synchronous operation, the neural network processing unit 11 executes a data synchronization instruction to force the plurality of neural network processing units 11 to perform a synchronous operation, that is, when the plurality of neural networks execute the current instruction, the neural network processing units Can execute subsequent instructions;
  • the operation instruction, the instruction name is COMPUTE, wherein the first operation code represents a specific calculation task, such as MLP, CONV, POOL, etc., and the remaining operation codes are used to indicate the address and size of the input and output data, and the configuration information of the neural network calculation instruction. .
  • the input/output command has an instruction name of IO, wherein the operation code respectively represents information of the start address, the end address, and the data size of the transport data, and the neural network processing unit 11 executes the input/output command to communicate data with the remaining modules.
  • the IO reading unit reads data from the outside of the neural network processing unit 11 (such as the high speed storage unit 12, other neural network processing unit 11, etc.) according to the operation instruction in the instruction queue 111, and caches the read data to In the cache unit 114, the neural network operation unit 112 reads the buffered data from the cache unit 114 according to the operation instruction, and performs a neural network operation to obtain corresponding operation result data;
  • the neural network processing unit 11 such as the high speed storage unit 12, other neural network processing unit 11, etc.
  • the neural network operation unit 112 writes the operation result data into the cache unit 114, and when it is necessary to transmit the operation result data to the outside (other neural network processing unit 11, etc.), the IO reading unit 113 reads from the cache unit 114. The operation result data is taken, and the operation result data is written to the outside of the neural network processing unit 11.
  • the on-chip interconnect 20 module includes a primary interconnecting module 21 and a plurality of secondary interconnecting modules 22 communicatively coupled to the primary interconnecting module.
  • the module 21 is also in communication with the external storage module 30.
  • the plurality of secondary interconnection modules 22 are in one-to-one correspondence with the plurality of neural network processing modules 10, wherein each secondary interconnection module 22 and the neural network in the corresponding neural network processing module respectively
  • the processing unit 11 and the high speed storage unit 12 are communicatively coupled.
  • the second-level interconnection module 22 is connected to the neural network processing unit 11 by one port, one port is connected to the high-speed storage unit 12 corresponding to the neural network processing unit, and the other port is connected to the first-level interconnection module 21, and the first-level interconnection module 21 is connected.
  • the secondary interconnect module 22 is coupled to the external storage module 30 to ensure data paths between the modules. In this way, it is possible to ensure communication between the respective neural network processing units 11 and the high speed storage unit 12 and the external storage module 30, and occupy a small area overhead.
  • the on-chip interconnect 20 module may further include one or more three-level interconnect modules communicatively coupled to each of the two-level interconnect modules 22, and may further include one or more four-level interconnect modules communicatively coupled to each of the three-level interconnect modules. , ..., one or more n-level interconnect modules communicatively coupled to each n-1 level interconnect module, where n is a positive integer greater than or equal to 3.
  • Each of the n-level interconnection modules is communicatively coupled to the neural network processing unit 11 and the high-speed storage unit 12 in the respective neural network processing modules.
  • a single layer neural network operation can be performed, including:
  • each neural network processing module 10 reads data directly from the local according to the calculation instruction stored in its own instruction queue 11 according to the address indicated by the operation code in the instruction, and/or from other neural networks through the on-chip interconnect module 20. Reading data in the processing module 10;
  • each neural network processing module 10 performs partial operations of a single layer neural network according to the read data to obtain respective operation result data;
  • each neural network processing module 10 stores the respective operation result data locally and/or writes the respective operation result data to the other neural network processing module 10 through the on-chip interconnecting module 20.
  • each neural network processing module 10 reads from the new address according to the new operation instruction. New data is taken for calculation, and computing tasks are distributed among multiple cores (ie, multiple neural network processing modules 10) in accordance with new instructions. For each layer of neural network operation, the above steps S1-S3 are performed, and the operation result data obtained by each layer of the neural network processing module 10 is used for the next layer of neural network operation.
  • FIG. 6 is a flow chart of an embodiment of performing a layer of fully connected layer operation in the present disclosure, and its execution process is as shown in FIG. 6:
  • Step 1 According to the operation instruction COMPUTE, each neural network processing unit 11 reads data from the corresponding high-speed storage unit 12, and separately calculates a partial operation of the fully connected layer. Result data.
  • the instruction queue 111 sends an operation instruction COMPUTE to the neural network operation unit 112 and the IO reading unit 113, and the neural network operation unit 112 determines that a layer is to be executed according to the instruction name in the operation instruction COMPUTE.
  • Fully connected layer operation Specifically, the IO reading unit 113 reads the operation required data from its corresponding high speed storage unit 12 according to the address in the operation instruction COMPUTE, and stores the read data in the cache unit 114.
  • the neural network operation unit 112 reads the corresponding data from the cache unit 114, and then executes the operation instruction COMPUTE based on the read data to perform a partial operation of the fully connected layer, and obtains the partial operation result data of the fully connected layer as the output data.
  • Step 2 According to the input/output command IO, each neural network processing unit 11 sends the partial operation result data calculated by itself to the corresponding neural network processing unit 11 through the on-chip interconnecting module 20. Since each neural network processing unit 11 only calculates part of the operation result data, it needs to send the partial output data to the corresponding neural network processing unit 11 for the addition operation.
  • the neural network operation unit 112 stores the calculated partial operation result data in the cache unit 114, and after the instruction queue 111 sends the input/output instruction IO to the IO reading unit 113, the IO reading unit 113 performs The instruction IO is output to read the partial operation result data stored in the cache unit 114 and transmit it to the external corresponding neural network processing unit 11.
  • each neural network processing unit 11 may send partial operation result data to a corresponding neural network processing unit 11 or may be sent to multiple corresponding neural network processing units 11, that is, Each of the neural network processing units 11 may also receive partial operation result data transmitted by one neural network processing unit 11, and may also receive partial operation result data transmitted by the plurality of neural network processing units 11.
  • Step 3 After each neural network processing unit 11 sends the partial operation result data calculated by itself to the corresponding neural network processing unit 11, the data delivery instruction ACK needs to be executed to send data to the corresponding neural network processing unit 11. Delivery signal. Each neural network processing unit 11 needs to send data to the neural network processing unit 11 that receives the data it receives. Signal to indicate its data dependencies.
  • Step 4 According to the data dependency instruction FENCE, each neural network processing unit 11 detects whether the transmission data delivery signal reaches the corresponding neural network processing unit 11, and if not, waits for the corresponding data delivery signal to reach the corresponding neural network. Processing unit 11. For each of the neural network processing units 11 to be added, only when it receives the data delivery signals sent by all other neural network processing units 11, it indicates that all the required input data arrives, thereby performing the addition operation. .
  • Step 5 According to the operation instruction COMPUTE, each neural network processing unit 11 collects part of the operation result data of the other neural network processing unit 11, and then combines the partial operation result data obtained by the self operation to perform the addition operation to obtain the final operation result. data.
  • Step 6 According to the input/output command IO, each neural network processing unit 11 writes the calculated final operation result data as output data into the external storage module 30.
  • the execution process of writing the final operation result data into the external storage module 30 is similar to that of step 2, and will not be described herein.
  • the device and the instruction set provided by the present disclosure solve the problems of insufficient performance of the CPU and the GPU and large overhead of the front-end decoding, and can effectively support the multi-layer artificial neural network operation, and simultaneously target the multi-core multi-layer artificial neural network.
  • the operation uses dedicated on-chip storage, which fully exploits the reusability of neurons and weight data, avoids repeatedly reading these data into memory, reduces the memory access bandwidth, and avoids the problem that memory bandwidth becomes the bottleneck of multi-layer artificial neural network computing performance. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Computer Hardware Design (AREA)
  • Advance Control (AREA)
  • Image Analysis (AREA)
  • Multi Processors (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)
  • Design And Manufacture Of Integrated Circuits (AREA)

Abstract

一种用于执行神经网络运算的装置及方法，该装置包括片上互联模块和与该片上互联单元通信连接的多个神经网络处理模块，神经网络处理模块能够通过片上互联模块从其它神经网络处理模块中读写数据。在多核多层人工神经网络运算中，要将每一层神经网络运算进行划分，进而由多个神经网络处理模块进行运算，得到各自的运算结果数据，多个神经网络处理单元还将各自的运算结果数据进行数据交换。

Description

用于执行神经网络运算的装置及方法 技术领域
本公开属于神经网络运算领域，尤其涉及一种用于执行神经网络运算的装置及方法。
背景技术
一种支持多核多层神经网络运算的已知方法是使用通用处理器。该方法通过使用通用寄存器堆和通用功能部件执行通用指令来支持上述算法。该方法的缺点之一是单个通用处理器的运算性能较低，无法满足通常的多核多层人工神经网络运算的性能需求。而多个通用处理器并行执行时，通用处理器之间相互通信又成为了性能瓶颈。另外，通用处理器需要把人工神经网络运算译码成一长列运算及访存指令序列，处理器前端译码带来了较大的功耗开销
另一种支持多核多层人工神经网络运算的已知方法是使用图形处理器(GPU)。该方法通过使用通用寄存器堆和通用流处理单元执行通用SIMD指令来支持上述算法。由于GPU是专门用来执行图形图像运算以及科学计算的设备，没有对人工神经网络运算的专门支持，仍然需要大量的前端译码工作才能执行多层人工神经网络运算，带来了大量的额外开销。另外GPU只有较小的片上缓存，多层人工神经网络的模型数据(权值)需要反复从片外搬运，片外带宽成为了主要性能瓶颈。
发明内容
有鉴于此，本公开提供一种用于执行神经网络运算的装置及方法，能以低开销的方式实现一层或多层多核多层人工神经网络的运算，并且运算性能高效。
本公开提供一种用于执行神经网络运算的装置及方法，装置包括片上互联模块和与该片上互联单元通信连接的多个神经网络处理模块，神 经网络处理模块能够通过片上互联模块从其它神经网络处理模块中读写数据。在多核多层人工神经网络运算中，要将每一层神经网络运算进行划分，进而由多个神经网络处理模块进行运算，得到各自的运算结果数据，多个神经网络处理单元还将各自的运算结果数据进行数据交换。例如每进行一层神经网络计算之后，每个神经网络处理模块只计算得到了部分输出数据，而在执行下一层神经网络运算时，每个神经网络处理模块还会需求来自于其他神经网络处理模块的数据，因此每个神经网络处理的需要将自己计算得到的运算结果数据发送给对应的神经网络处理模块，用以使之计算下一层的神经网络。
用于执行神经网络运算的装置中，神经网络处理模块能够通过片上互联模块从其它神经网络处理模块中读写数据，还可从本地读写数据。
进一步，神经网络处理模块包括神经网络处理单元和高速存储单元；神经网络处理单元用于读写数据，高速存储单元用于存储本地数据。
进一步，用于执行神经网络运算的装置还包括外部存储模块，神经网络处理模块还能够通过片上互联单元从外部存储模块中读写数据。
进一步，神经网络处理单元包括指令队列、高速缓存单元、IO读取单元和神经网络运算单元，其中：
进一步，指令队列存储有运算指令，IO读取单元根据运算指令从该神经网络处理单元的外部读取数据，并将读取的数据缓存至高速缓存单元中，神经网络运算单元根据该运算指令从高速缓存单元中读取所缓存的数据，并执行神经网络运算，得到运算结果数据；
进一步，神经网络处理单元还包括同步关系单元，指令队列还存储有数据送达指令，神经网络运算单元向其它神经网络运算单元发送数据后，所述神经网络运算单元所对应的同步关系单元执行数据送达指令，以向其它神经网络运算单元所对应的同步关系单元发送一数据送达信号。
进一步，指令队列还存储有数据依赖指令，所述神经网络运算单元收到其其它神经网络运算单元所发送的数据后，其对应的同步关系单元执行所述数据依赖指令以检测是否是收到数据送达信号，若是，则继续 执行执行指令队列中的指令，否则阻塞指令队列。
进一步，指令队列还存储有数据同步指令，神经网络处理单元中的同步关系单元通过执行所述数据同步指令，以向其它神经网络处理单元中的同步关系单元发送一同步信号，以强制多个神经网络处理单元做同步操作。
进一步，片上互联模块包括一级互联模块和与该一级互联模块通信连接的多个二级互联模块，一级互联模块还与所述外部存储模块通信连接，所述多个二级互联模块与多个神经网络处理模块一一对应，其中，每个二级互联模块分别与相应神经网络处理模块中的神经网络处理单元和高速存储单元通信连接。
本公开还提供一种用于执行单层神经网络运算的方法，包括：
S1，多个神经网络处理模块中的每个神经网络处理模块直接从本地读取数据，和/或通过片上互联模块从其它神经网络处理模块中读取数据，其中，多个神经网络处理模块与该片上互联单元通信连接；
S2，每个神经网络处理模块根据读取的数据进行单层神经网络的部分运算，得到各自的运算结果数据；
S3，每个神经网络处理模块将各自的运算结果数据进行本地存储和/或通过片上互联模块将各自的运算结果数据写入至其他神经网络处理模块中。
进一步，步骤S3中，每个神经网络处理模块将各自的运算结果写入至其它神经网络处理模块中后，向其它特定的神经网络处理模块发送一数据送达信号。
本公开还提供一种用于执行多层神经网络运算的方法，对于每一层神经网络运算，执行上述步骤S1-S3，并将该层各神经网络处理模块得到的运算结果数据用于下一层神经网络运算。
本公开所提供的用于执行神经网络运算的装置及方法，具有以下优点：
1、由于采用多核神经网络处理模块，允许单层的神经网络将任务划分，在多个神经网络处理模块上执行，并且采用了专用指令，允许在 执行多层神经网络的时候，可以在多个神经网络处理器之间相互传输计算得到的数据，故能实现多层多核的神经网络运算。
2、由于采用多核神经网络处理模块，解决了在执行多核多层神经网络处理运算的时候，单个处理器处理性能不足的问题，具有显著加速多核多层神经网络运算的效果。
3、由于采用了专用的数据指令，有效的解决了在执行多核多层神经网络时，其多个处理器之间需要交互大量数据的问题，具有显著加速多核多层神经网络运算的效果。
附图说明
图1是本公开提供的用于执行神经网络运算的装置的结构示意图；
图2是本公开中神经网络处理模块的结构示意图；
图3是本公开中外部存储模块的结构示意图；
图4是本公开中神经网络处理单元的结构示意图；
图5是本公开中片上互联单元的结构示意图；
图6是本公开执行一层全连接层运算实施例的流程图。
具体实施方式
本公开提供的用于执行人工神经网络运算的装置可以应用于以下(包括但不限于)场景中：数据处理、机器人、电脑、打印机、扫描仪、电话、平板电脑、智能终端、手机、行车记录仪、导航仪、传感器、摄像头、云端服务器、相机、摄像机、投影仪、手表、耳机、移动存储、可穿戴设备等各类电子产品；飞机、轮船、车辆等各类交通工具；电视、空调、微波炉、冰箱、电饭煲、加湿器、洗衣机、电灯、燃气灶、油烟机等各类家用电器；以及包括核磁共振仪、B超、心电图仪等各类医疗设备。
图1是本公开提供的用于执行神经网络运算的装置的结构示意图，如图1所示，装置包括多个神经网络处理模块10及一个片上互联模块20，多个神经网络处理模块10与该片上互联单元20通信连接，其中：
神经网络处理模块10能够通过片上互联模块30从其它神经网络处理模块10中读写数据，还可从本地读写数据。当要执行神经网络运算时，每个神经网络处理模块10作为一个核执行相应的运算，其运算所需的数据可直接从本地直接获取，也可通过片上互联模块20与其他神经网络处理模块10通信，以从其他神经网络处理模块10处读取运算所需的数据。各个神经网络处理模块10读取运算所需的数据后，执行相应的运算，得到各自的运算结果数据，在单层神经网络运算中，各个神经网络处理模块10可将各自的运算结果数据汇总至一个神经网络处理模块10中进行累加，以得到最终结果数据。在多层神经网络运算中，当层各个神经网络处理模块10计算得到运算结果数据，可能在下一层作为运算所需的数据被其他神经网络处理模块10使用，这样在当层神经网络运算完毕后，各个神经网络处理模块10会进行数据交互，以准备进行下一层神经网络运算。
图2是本公开中神经网络处理模块的结构示意图，如图2所示，神经网络处理模块10包括神经网络处理单元11和高速存储单元12；神经网络处理模块10在进行神经网络运算时，神经网络处理单元11直接从与其对应的高速储存单元12中读取数据，和/或通过片上互联单元20从其它神经网络处理模块10中的神经网络处理单元11中读取数据，和/或通过片上互联单元20从其它神经网络处理模块10中的高速存储单元12中读取数据；每个神经网络处理模块10中的神经网络处理单元11根据读取的数据进行神经网络运算，得到各自的运算结果数据；在完成运算后，神经网络处理单元11将运算结果数据直接写入至与其对应的高速储存单元12中，和/或通过片上互联单元20将运算结果数据写入至其它神经网络处理模块10中的神经网络处理单元11中，和/或通过片上互联单元20将运算结果数据写入至其它神经网络处理模块10中的高速存储单元12中。总之，神经网络处理单元11可直接从其对应的高速储存单元获取数据，也可以通过片上互联模块20获取其它位置的数据，这样避免了反复向内存读取数据，降低了内存访问带宽。
如图3所示，本公开提供的用于执行神经网络运算的装置还包括外 部存储模块30，其与片上互联单元20通信连接，神经网络处理模块10还能够通过片上互联单元从外部存储模块中读写数据，利用外部存储模块30，可以从外界向装置中导入新的数据，装置执行的最终执行结果数据也可以写入至外部存储模块30，以供外部导出。其中，外部存储模块30可以通过硬件来实现(包括但不限于FPGA、CGRA、专用集成电路ASIC、模拟电路或忆阻器等)。
图4是本公开中神经网络处理单元11的结构示意图，如图4所示，神经网络处理单元11包括指令队列111、神经网络运算单元112、IO读取单元113、高速缓存单元114和同步关系单元115。指令队列111存储有多种类型的指令，神经网络处理单元11根据不同的指令执行不同的操作。下表为各类指令的描述：
指令名称 操作码1 操作码2 操作码3 操作码4 操作码5 ……
ACK 0/1 0/1 0/1 0/1 0/1 ……
FENCE 0/1 0/1 0/1 0/1 0/1 ……
SYNC 0/1 0/1 0/1 0/1 0/1 ……
COMPUTE MLP addr1 size1 addr2 size2 ……
IO src dest size      
指令包括指令名称以及多个操作码：
数据送达指令，指令名称为ACK，其中各个操作码分别表示是否向该神经网络处理单元11发送数据送达信号(ACK信号)，神经网络处理单元11向其他神经网络处理单元11写入数据后，执行数据送达指令以发送数据送达信号给对应的神经网络处理单元11，以表明数据已经传输到位；
数据依赖指令，指令名称为FENCE，其中各操作码表示是否检查来自该神经网络处理单元11的ACK信号；神经网络处理单元11执行数据依赖指令以检测其所有依赖的数据是否已到达本神经网络处理单元。
数据同步指令，指令名称为SYNC，其中各个操作码表示该神经网 络处理单元是否参与同步操作，神经网络处理单元11执行数据同步指令用以强制多个神经网络处理单元11做同步操作，即当多个神经网络都执行到当前指令后，这些神经网络处理单元才可以执行之后的指令；
运算指令，指令名称为COMPUTE，其中第一个操作码表示具体的计算任务，如MLP，CONV，POOL等，其余操作码用来表示输入输出数据的地址和大小，以及神经网络计算指令的配置信息。
输入输出指令，指令名称为IO，其中的操作码分别表示搬运数据的起始地址，结束地址以及数据大小的信息，神经网络处理单元11执行输入输出指令以与其余模块之间进行通信数据。
IO读取单元根据113根据指令队列111中的运算指令从该神经网络处理单元11的外部(如高速存储单元12、其他神经网络处理单元11等)读取数据，并将读取的数据缓存至高速缓存单元114中，神经网络运算单元112根据该运算指令从高速缓存单元114中读取所缓存的数据，并执行神经网络运算，得到相应的运算结果数据；
神经网络运算单元112将运算结果数据写入至高速缓存单元114中，当需要将运算结果数据发送中外部(其他神经网络处理单元11等)时，IO读取单元113从高速缓存单元114中读取运算结果数据，并将运算结果数据写入到该神经网络处理单元11的外部。
图5是本公开中片上互联单元的结构示意图，如图5所示，片上互联20模块包括一级互联模块21和与该一级互联模块通信连接的多个二级互联模块22，一级互联模块21还与外部存储模块30通信连接，多个二级互联模块22与多个神经网络处理模块10一一对应，其中，每个二级互联模块22分别与相应神经网络处理模块中的神经网络处理单元11和高速存储单元12通信连接。具体的，二级的互联模块22一个端口连接神经网络处理单元11，一个端口连接该神经网络处理单元对应的高速存储单元12，另一个端口连接一级互联模块21，一级互联模块21将多个二级互联模块22和外部存储模块30连接，用以保证这些模块之间的数据通路。这样，可以在保证各个神经网络处理单元11以及高速存储单元12和外部存储模块30之间相互通信，并且占用较小的面积开销。
另外，片上互联20模块还可以包括与每个二级互联模块22通信连接的一个或多个三级互联模块，还可以包括与每个三级互联模块通信连接的一个或多个四级互联模块，…，与每个n-1级互联模块通信连接的一个或多个n级互联模块，其中n为大于等于3的正整数。每个n级互联模块分别与相应神经网络处理模块中的神经网络处理单元11和高速存储单元12通信连接。
采用本公开以上所描述的装置，可执行单层神经网络运算，包括：
S1，每个神经网络处理模块10根据其自身指令队列11中存储的计算指令，根据指令中操作码所指示的地址，直接从本地读取数据，和/或通过片上互联模块20从其它神经网络处理模块10中读取数据；
S2，每个神经网络处理模块10根据读取的数据进行单层神经网络的部分运算，得到各自的运算结果数据；
S3，每个神经网络处理模块10将各自的运算结果数据进行本地存储和/或通过片上互联模块20将各自的运算结果数据写入至其他神经网络处理模块10中。
对于多层神经网络运算，其实现过程与单层神经网络类似，当上一层人工神经网络执行完毕后，在下一层运算时，每个神经网络处理模块10根据新的运算指令从新的地址读取新的数据进行计算，并且依据新的指令在多核(即多个神经网络处理模块10)之间分配计算任务。对于每一层神经网络运算，执行上述步骤S1-S3，并将该层各神经网络处理模块10得到的运算结果数据用于下一层神经网络运算。
为使本公开的目的、技术方案和优点更加清楚明白，以下结合具体实施例，并参照附图，对本公开进一步详细说明。应当说明的是，虽然下面所述实施例仅讨论了全连接层的运算流程图，但是不局限在全连接层，也可以是所有的人工神经网络算法的运算流程图。
图6是本公开执行一层全连接层运算实施例的流程图，其执行过程如图6所示：
步骤1：依据运算指令COMPUTE，每个神经网络处理单元11从对应的高速存储单元12中读取数据，分别计算得到全连接层的部分运算 结果数据。
在每个神经网络处理单元11中，指令队列111将运算指令COMPUTE发送至神经网络运算单元112和IO读取单元113，神经网络运算单元112根据运算指令COMPUTE中的指令名称，确定将要执行一层全连接层运算，具体的，IO读取单元113根据运算指令COMPUTE中的地址从其对应的高速存储单元12中读取运算所需数据，并将读取的数据存储于高速缓存单元114中，神经网络运算单元112从高速缓存单元114中读取相应的数据，然后根据读取的数据执行运算指令COMPUTE，以进行全连接层的部分运算，得到全连接层的部分运算结果数据作为输出数据。
步骤2：依据输入输出指令IO，每个神经网络处理单元11将自己计算的到的部分运算结果数据通过片上互联模块20发送给相应的神经网络处理单元11。由于每个神经网络处理单元11只计算出部分运算结果数据，因此其需要将该部分输出数据发送给相应的神经网络处理单元11进行加和运算。
具体的，步骤1中神经网络运算单元112将计算得到的部分运算结果数据存储于高速缓存单元114中，指令队列111将输入输出指令IO发送给IO读取单元113后，IO读取单元113执行输出指令IO，以将存储于高速缓存单元114中的部分运算结果数据读取，并发送至外部的相应的神经网络处理单元11。这里需要说明的是，每个神经网络处理单元11可能会将部分运算结果数据发送至一个对应的神经网络处理单元11中，也可能发送至多个对应的神经网络处理单元11中，也就是说，每个神经网络处理单元11也可能收到一个神经网络处理单元11发送的部分运算结果数据，也可能收到多个神经网络处理单元11发送的部分运算结果数据。
步骤3：每个神经网络处理单元11将自己计算的到的部分运算结果数据发送给相应的神经网络处理单元11后，需要执行数据送达指令ACK，以向对应的神经网络处理单元11发送数据送达信号。每个神经网络处理单元11需要向接受其发送数据的神经网络处理单元11发送数据送达 信号，用以表明其数据依赖关系。
步骤4：依据数据依赖指令FENCE，每个神经网络处理单元11检测其发送数据送达信号是否到达相应的神经网络处理单元11，如果没有到达，则等待对应的数据送达信号到达相应的神经网络处理单元11。对于每个将要进行加和运算神经网络处理单元11，只有其收到所有其他神经网络处理单元11所发送的数据送达信号时，才表明其所需要的输入数据全部到达，从而执行加和运算。
步骤5：依据运算指令COMPUTE，每个的神经网络处理单元11汇集其他神经网络处理单元11的部分运算结果数据后，联合上自身运算所得的部分运算结果数据进行加和运算，得到最终的运算结果数据。
步骤6：依据输入输出指令IO，每个神经网络处理单元11将计算得到的最终的运算结果数据作为输出数据写入外部存储模块30中。在每个神经网络处理单元11中，将最终的运算结果数据写入外部存储模块30中的执行过程与步骤2类似，在此就不再赘述。
综上所述，本公开提供的装置和指令集，解决了CPU和GPU运算性能不足、前端译码开销大的问题，能有效支持多层人工神经网络运算，同时，针对多核多层人工神经网络运算采用专用片上存储，充分挖掘了神经元和权值数据的重用性，避免了反复向内存读取这些数据，降低了内存访问带宽，避免了内存带宽成为多层人工神经网络运算性能瓶颈的问题。
前面的附图中所描绘的进程或方法可通过包括硬件(例如，电路、专用逻辑等)、固件、软件(例如，被具体化在非瞬态计算机可读介质上的软件)，或两者的组合的处理逻辑来执行。虽然上文按照某些顺序操作描述了进程或方法，但是，应该理解，所描述的某些操作能以不同顺序来执行。此外，可并行地而非顺序地执行一些操作。
以上所述的具体实施例，对本公开的目的、技术方案和有益效果进行了进一步详细说明，应理解的是，以上所述仅为本公开的具体实施例而已，并不用于限制本公开，凡在本公开的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本公开的保护范围之内。

Claims (12)

  1. 一种用于执行神经网络运算的装置，其特征在于，包括片上互联模块和与该片上互联单元通信连接的多个神经网络处理模块，其中：
    所述神经网络处理模块能够通过所述片上互联模块从其它神经网络处理模块中读写数据。
  2. 根据权利要求1所述的用于执行多核多层神经网络运算的装置，其特征在于，所述神经网络处理模块还可从本地读写数据。
  3. 根据权利要求2所述的用于执行神经网络运算的装置，其特征在于，所述神经网络处理模块包括神经网络处理单元和高速存储单元；
    所述神经网络处理单元用于读写数据，所述高速存储单元用于存储本地数据。
  4. 根据权利要求1-3任意一项所述的用于执行神经网络运算的装置，其特征在于，还包括外部存储模块，所述神经网络处理模块还能够通过所述片上互联单元从所述外部存储模块中读写数据。
  5. 根据权利要求2所述的用于执行神经网络运算的装置，其特征在于，所述神经网络处理单元包括指令队列、高速缓存单元、IO读取单元和神经网络运算单元，其中：
    所述指令队列存储有运算指令，所述IO读取单元根据所述运算指令从该神经网络处理单元的外部读取数据，并将读取的数据缓存至所述高速缓存单元中，所述神经网络运算单元根据该运算指令从所述高速缓存单元中读取所缓存的数据，并执行神经网络运算，得到运算结果数据；
    所述神经网络运算单元将所述运算结果数据写入至所述高速缓存单元中，所述IO读取单元从所述高速缓存单元中读取所述运算结果数据，并将所述运算结果数据写入到该神经网络处理单元的外部。
  6. 根据权利要求5所述的用于执行神经网络运算的装置，其特征在于，所述神经网络处理单元还包括同步关系单元，所述指令队列还存储有数据送达指令，所述神经网络运算单元向其它神经网络运算单元发送数据后，所述神经网络运算单元所对应的同步关系单元执行数据送达 指令，以向其它神经网络运算单元所对应的同步关系单元发送一数据送达信号。
  7. 根据权利要求6所述的用于执行神经网络运算的装置，其特征在于，所述指令队列还存储有数据依赖指令，所述神经网络运算单元收到其其它神经网络运算单元所发送的数据后，其对应的同步关系单元执行所述数据依赖指令以检测是否是收到数据送达信号，若是，则继续执行执行指令队列中的指令，否则阻塞指令队列。
  8. 根据权利要求6所述的用于执行神经网络运算的装置，其特征在于，所述指令队列还存储有数据同步指令，神经网络处理单元中的同步关系单元通过执行所述数据同步指令，以向其它神经网络处理单元中的同步关系单元发送一同步信号，以强制多个神经网络处理单元做同步操作。
  9. 根据权利要求3所述的用于执行神经网络运算的装置，其特征在于，所述片上互联模块包括一级互联模块和与该一级互联模块通信连接的多个二级互联模块，所述一级互联模块还与所述外部存储模块通信连接，所述多个二级互联模块与所述多个神经网络处理模块一一对应，其中，每个二级互联模块分别与相应神经网络处理模块中的神经网络处理单元和高速存储单元通信连接。
  10. 一种用于执行单层神经网络运算的方法，其特征在于，包括：
    S1，多个神经网络处理模块中的每个神经网络处理模块直接从本地读取数据，和/或通过片上互联模块从其它神经网络处理模块中读取数据，其中，多个神经网络处理模块与该片上互联单元通信连接；
    S2，每个神经网络处理模块根据读取的数据进行单层神经网络的部分运算，得到各自的运算结果数据；
    S3，每个神经网络处理模块将各自的运算结果数据进行本地存储和/或通过所述片上互联模块将各自的运算结果数据写入至其他神经网络处理模块中。
  11. 根据权利要求10所述的用于执行单层神经网络运算的方法，其特征在于，所述步骤S3中，每个神经网络处理模块将各自的运算结 果写入至其它神经网络处理模块中后，向其它特定的神经网络处理模块发送一数据送达信号。
  12. 一种用于执行多层神经网络运算的方法，其特征在于，对于每一层神经网络运算，执行如权利要求10所述的方法，并将该层各神经网络处理模块得到的运算结果数据用于下一层神经网络运算。
PCT/CN2017/095810 2016-08-05 2017-08-03 用于执行神经网络运算的装置及方法 Ceased WO2018024232A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP17836414.7A EP3496007B1 (en) 2016-08-05 2017-08-03 Device and method for executing neural network operation
US16/268,468 US11120331B2 (en) 2016-08-05 2019-02-05 Device and method for executing neural network operation

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610635286.X 2016-08-05
CN201610635286.XA CN107688853B (zh) 2016-08-05 2016-08-05 一种用于执行神经网络运算的装置及方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/268,468 Continuation-In-Part US11120331B2 (en) 2016-08-05 2019-02-05 Device and method for executing neural network operation

Publications (1)

Publication Number Publication Date
WO2018024232A1 true WO2018024232A1 (zh) 2018-02-08

Family

ID=61073469

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/095810 Ceased WO2018024232A1 (zh) 2016-08-05 2017-08-03 用于执行神经网络运算的装置及方法

Country Status (5)

Country Link
US (1) US11120331B2 (zh)
EP (1) EP3496007B1 (zh)
CN (2) CN111310893B (zh)
TW (1) TWI767925B (zh)
WO (1) WO2018024232A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108289177A (zh) * 2018-02-13 2018-07-17 北京旷视科技有限公司 信息交互方法、装置及系统
CN109919310A (zh) * 2019-01-15 2019-06-21 中国科学院信息工程研究所 一种面向深度学习训练任务的gpu内存优化方法及系统
WO2019202425A1 (en) * 2018-04-20 2019-10-24 International Business Machines Corporation Time, space, and energy efficient neural inference via parallelism and on-chip memory
CN111382857A (zh) * 2018-12-29 2020-07-07 上海寒武纪信息科技有限公司 任务处理装置、神经网络处理器芯片、组合装置以及电子设备

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019165940A1 (zh) * 2018-02-27 2019-09-06 上海寒武纪信息科技有限公司 集成电路芯片装置、板卡及相关产品
CN110197266B (zh) * 2018-02-27 2020-08-04 上海寒武纪信息科技有限公司 集成电路芯片装置及相关产品
CN111767996B (zh) * 2018-02-27 2024-03-05 上海寒武纪信息科技有限公司 集成电路芯片装置及相关产品
CN110297779A (zh) * 2018-03-23 2019-10-01 余晓鹏 一种内存难解性算法的解决方法
US11663461B2 (en) 2018-07-05 2023-05-30 International Business Machines Corporation Instruction distribution in an array of neural network cores
CN110825440B (zh) * 2018-08-10 2023-04-14 昆仑芯(北京)科技有限公司 指令执行方法和装置
CN109542830B (zh) * 2018-11-21 2022-03-01 北京灵汐科技有限公司 一种数据处理系统及数据处理方法
CN111258770B (zh) * 2018-11-30 2023-10-10 上海寒武纪信息科技有限公司 数据处理方法、处理器、数据处理装置及存储介质
CN111258645B (zh) * 2018-11-30 2022-12-09 上海寒武纪信息科技有限公司 数据处理方法、处理器、数据处理装置及存储介质
CN111258636B (zh) * 2018-11-30 2022-10-04 上海寒武纪信息科技有限公司 数据处理方法、处理器、数据处理装置及存储介质
CN111258635B (zh) * 2018-11-30 2022-12-09 上海寒武纪信息科技有限公司 数据处理方法、处理器、数据处理装置及存储介质
CN111258647B (zh) * 2018-11-30 2022-12-09 上海寒武纪信息科技有限公司 数据处理方法、处理器、数据处理装置及存储介质
CN111258652B (zh) * 2018-11-30 2022-12-09 上海寒武纪信息科技有限公司 数据处理方法、处理器、数据处理装置及存储介质
CN111258637B (zh) * 2018-11-30 2022-08-05 上海寒武纪信息科技有限公司 数据处理方法、处理器、数据处理装置及存储介质
US11126912B2 (en) * 2018-12-11 2021-09-21 Mipsology SAS Realigning streams of neuron outputs in artificial neural network computations
CN111767078B (zh) * 2019-04-02 2024-08-06 上海寒武纪信息科技有限公司 数据运行方法、装置和相关产品
KR102780228B1 (ko) * 2019-10-10 2025-03-14 삼성전자 주식회사 무선 통신 시스템에서 인공 지능을 활용한 신호 송수신 방법 및 장치
CN111488969B (zh) * 2020-04-03 2024-01-19 北京集朗半导体科技有限公司 基于神经网络加速器的执行优化方法及装置
CN111752689B (zh) * 2020-06-22 2023-08-25 深圳鲲云信息科技有限公司 一种基于数据流的神经网络多引擎同步计算系统
US20240054012A1 (en) * 2020-12-31 2024-02-15 Shanghai Cambricon Information Technology Co., Ltd. Inter-chip communication circuit, method and system
CN115081601A (zh) * 2021-03-11 2022-09-20 安徽寒武纪信息科技有限公司 同步Winograd卷积的计算装置与方法
CN113673701B (zh) * 2021-08-24 2024-09-06 安谋科技（中国）有限公司 神经网络模型的运行方法、可读介质和电子设备
CN114356494B (zh) * 2021-12-08 2025-10-10 深圳云天励飞技术股份有限公司 一种神经网络模拟器的数据处理方法、装置和终端
WO2023128009A1 (ko) * 2021-12-30 2023-07-06 리벨리온 주식회사 뉴럴 프로세싱 장치 및 그의 동기화 방법
CN115526302B (zh) * 2022-08-19 2023-07-25 北京应用物理与计算数学研究所 基于异构多核处理器的多层神经网络计算方法及装置
CN115952835A (zh) * 2023-01-19 2023-04-11 安谋科技（中国）有限公司 数据处理方法、可读介质和电子设备

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101882238A (zh) * 2010-07-15 2010-11-10 长安大学 基于sopc的小波神经网络处理器
CN102193518A (zh) * 2011-05-13 2011-09-21 南京理工大学 基于基底神经节的fpga仿生智能控制芯片
CN105488565A (zh) * 2015-11-17 2016-04-13 中国科学院计算技术研究所 加速深度神经网络算法的加速芯片的运算装置及方法

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5204938A (en) * 1989-05-30 1993-04-20 Loral Aerospace Corp. Method of implementing a neural network on a digital computer
WO1991018351A1 (en) * 1990-05-22 1991-11-28 International Business Machines Corporation Pyramid learning architecture neurocomputer
US7388862B2 (en) * 2003-06-19 2008-06-17 Cisco Technology, Inc. Technique for notifying EIGRP neighbors when destroying adjacencies in a computer network
EP2434420A3 (en) * 2003-08-01 2012-07-25 Dna Twopointo Inc. Systems and methods for biopolymer engineering
CN101527010B (zh) * 2008-03-06 2011-12-07 上海理工大学 人工神经网络算法的硬件实现方法及其系统
CN101534165B (zh) * 2009-03-31 2013-03-13 江南大学 一种混沌神经网络保密通信电路
JP5387147B2 (ja) * 2009-06-03 2014-01-15 日本電気株式会社 病理画像診断システム、病理画像処理方法、病理画像診断プログラム
US8346883B2 (en) * 2010-05-19 2013-01-01 International Business Machines Corporation Effecting hardware acceleration of broadcast operations in a parallel computer
US8935513B2 (en) * 2012-02-08 2015-01-13 International Business Machines Corporation Processor performance improvement for instruction sequences that include barrier instructions
US9087301B2 (en) * 2012-12-21 2015-07-21 International Business Machines Corporation Hardware architecture for simulating a neural network of neurons
US9542643B2 (en) * 2013-05-21 2017-01-10 Qualcomm Incorporated Efficient hardware implementation of spiking networks
GB201310859D0 (en) * 2013-06-18 2013-07-31 Cambridge Entpr Ltd Rational method for solubilising proteins
US9754204B2 (en) * 2013-08-05 2017-09-05 Board Of Trustees Of The University Of Alabama Systems, methods and devices for vector control of permanent magnet synchronous machines using artificial neural networks
CN104899640B (zh) * 2014-07-21 2019-09-10 徐志强 神经网络的模拟装置及方法
CN105373829B (zh) * 2014-09-02 2018-05-04 北京大学 一种全连接神经网络结构
US11049006B2 (en) * 2014-09-12 2021-06-29 Microsoft Technology Licensing, Llc Computing system for training neural networks
US20160342887A1 (en) * 2015-05-21 2016-11-24 minds.ai inc. Scalable neural network system
CN104978601B (zh) * 2015-06-26 2017-08-25 深圳市腾讯计算机系统有限公司 神经网络模型训练系统和方法
US10878320B2 (en) * 2015-07-22 2020-12-29 Qualcomm Incorporated Transfer learning in neural networks
CN107545303B (zh) * 2016-01-20 2021-09-07 中科寒武纪科技股份有限公司 用于稀疏人工神经网络的计算装置和运算方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101882238A (zh) * 2010-07-15 2010-11-10 长安大学 基于sopc的小波神经网络处理器
CN102193518A (zh) * 2011-05-13 2011-09-21 南京理工大学 基于基底神经节的fpga仿生智能控制芯片
CN105488565A (zh) * 2015-11-17 2016-04-13 中国科学院计算技术研究所 加速深度神经网络算法的加速芯片的运算装置及方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3496007A4 *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108289177A (zh) * 2018-02-13 2018-07-17 北京旷视科技有限公司 信息交互方法、装置及系统
CN108289177B (zh) * 2018-02-13 2020-10-16 北京旷视科技有限公司 信息交互方法、装置及系统
WO2019202425A1 (en) * 2018-04-20 2019-10-24 International Business Machines Corporation Time, space, and energy efficient neural inference via parallelism and on-chip memory
GB2586556A (en) * 2018-04-20 2021-02-24 Ibm Time, space, and energy efficient neural inference via parallelism and on-chip memory
GB2586556B (en) * 2018-04-20 2021-08-11 Ibm Time, space, and energy efficient neural inference via parallelism and on-chip memory
CN111382857A (zh) * 2018-12-29 2020-07-07 上海寒武纪信息科技有限公司 任务处理装置、神经网络处理器芯片、组合装置以及电子设备
CN111382857B (zh) * 2018-12-29 2023-07-18 上海寒武纪信息科技有限公司 任务处理装置、神经网络处理器芯片、组合装置以及电子设备
CN109919310A (zh) * 2019-01-15 2019-06-21 中国科学院信息工程研究所 一种面向深度学习训练任务的gpu内存优化方法及系统
CN109919310B (zh) * 2019-01-15 2021-05-18 中国科学院信息工程研究所 一种面向深度学习训练任务的gpu内存优化方法及系统

Also Published As

Publication number Publication date
US20190171932A1 (en) 2019-06-06
CN111310893A (zh) 2020-06-19
US11120331B2 (en) 2021-09-14
CN111310893B (zh) 2023-11-21
TW201805858A (zh) 2018-02-16
EP3496007A1 (en) 2019-06-12
CN107688853A (zh) 2018-02-13
TWI767925B (zh) 2022-06-21
CN107688853B (zh) 2020-01-10
EP3496007B1 (en) 2022-10-05
EP3496007A4 (en) 2020-03-25

Similar Documents

Publication Publication Date Title
WO2018024232A1 (zh) 用于执行神经网络运算的装置及方法
CN106991477B (zh) 一种人工神经网络压缩编码装置和方法
CN111857819B (zh) 一种用于执行矩阵加/减运算的装置和方法
WO2017185387A1 (zh) 一种用于执行全连接层神经网络正向运算的装置和方法
CN111651203B (zh) 一种用于执行向量四则运算的装置和方法
CN108320018B (zh) 一种人工神经网络运算的装置及方法
CN111488976B (zh) 神经网络计算装置、神经网络计算方法及相关产品
EP3451238A1 (en) Apparatus and method for executing pooling operation
CN111651204B (zh) 一种用于执行向量最大值最小值运算的装置和方法
WO2018058427A1 (zh) 神经网络运算装置及方法
CN111651206A (zh) 一种用于执行向量外积运算的装置和方法
WO2017185404A1 (zh) 一种用于执行向量逻辑运算的装置及方法
CN114840339A (zh) Gpu服务器、数据计算方法及电子设备
CN111860772B (zh) 一种用于执行人工神经网络pooling运算的装置和方法
CN111368986A (zh) 一种神经网络计算装置和方法
CN111368967A (zh) 一种神经网络计算装置和方法
CN111381802B (zh) 数据比较器、数据处理方法、芯片及电子设备
CN113918220A (zh) 流水线控制方法、运算模块及相关产品
CN111368990A (zh) 一种神经网络计算装置和方法
CN112394991A (zh) 浮点转半精度浮点指令处理装置、方法及相关产品
CN111258641A (zh) 运算方法、装置及相关产品
CN112394992A (zh) 半精度浮点转八位整形指令处理装置、方法及相关产品
CN112394990A (zh) 浮点转半精度浮点指令处理装置、方法及相关产品
CN112394989A (zh) 无符号转半精度浮点指令处理装置、方法及相关产品
CN112394993A (zh) 半精度浮点转短整形指令处理装置、方法及相关产品

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17836414

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2017836414

Country of ref document: EP

Effective date: 20190305