WO2019127363A1 - Procédé de codage de poids pour réseau neuronal, appareil informatique et système matériel - Google Patents

Procédé de codage de poids pour réseau neuronal, appareil informatique et système matériel Download PDF

Info

Publication number
WO2019127363A1
WO2019127363A1 PCT/CN2017/119821 CN2017119821W WO2019127363A1 WO 2019127363 A1 WO2019127363 A1 WO 2019127363A1 CN 2017119821 W CN2017119821 W CN 2017119821W WO 2019127363 A1 WO2019127363 A1 WO 2019127363A1
Authority
WO
WIPO (PCT)
Prior art keywords
weight
matrix
splicing
analog circuit
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2017/119821
Other languages
English (en)
Chinese (zh)
Inventor
张悠慧
季宇
张优扬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tsinghua University
Original Assignee
Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tsinghua University filed Critical Tsinghua University
Priority to PCT/CN2017/119821 priority Critical patent/WO2019127363A1/fr
Priority to CN201780042640.0A priority patent/CN109791626B/zh
Publication of WO2019127363A1 publication Critical patent/WO2019127363A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means

Definitions

  • the present invention relates generally to the field of neural network technologies, and more particularly to a weight coding method, a computing device, and a hardware system for a neural network.
  • the neural network has made breakthroughs in computing, and has achieved high accuracy in many fields such as image recognition, speech recognition, and natural language processing.
  • neural networks require massive computing resources.
  • the general-purpose processor has been difficult to meet the computational needs of deep learning, and designing a dedicated chip has become an important development direction.
  • memristor provides an efficient solution for neural network chip design.
  • Memristor has the advantages of high density, non-volatile, low power consumption, cost-effective, easy 3D, etc.
  • the characteristics of adjustable resistance can be used as the programmable weight, and the advantage of the combination of the calculation and the calculation can be used as the high-speed multiplier.
  • the neural network components are all neurons, which are connected to each other by a large number of neurons.
  • the connections between neurons can be thought of as directed edges with weights, the outputs of the neurons are weighted by the connections between the neurons, and then passed to the connected neurons, and all the neurons receive The inputs are added together for further processing, producing the output of the neurons.
  • Neural network modeling usually consists of several neurons as a layer, and layers are connected to each other to construct.
  • Figure 1 shows a chain of neural networks. Each circle in the figure represents a neuron, each The arrows indicate the connections between the neurons, each of which has a weight, and the structure of the actual neural network is not limited to a chain-like network structure.
  • the core computation of the neural network is a matrix vector multiplication operation.
  • the output produced by the layer L n containing n neurons can be represented by a vector V n of length n, which is fully associated with the layer L m containing m neurons, and the connection weights It can be expressed as a matrix M n ⁇ m , the matrix size is n rows and m columns, and each matrix element represents the weight of one connection.
  • the vector input to L m after weighting is M n ⁇ m V n , and such matrix vector multiplication is the core calculation of the neural network.
  • the neural network acceleration chip also has the main design goal of accelerating matrix multiplication.
  • the memristor array is just right for the above work.
  • V is a set of input voltage
  • the voltage is multiplied by the memristor conductance G and superimposed output current
  • the output current is multiplied by the grounding resistance Rs to obtain the output voltage V'.
  • the whole process is realized under the analog circuit, and has a fast speed and a small area.
  • chip computing based on memristor also has the disadvantages of low precision, large disturbance, large cost of digital-to-analog/analog conversion, and limited matrix size.
  • the memristor can perform matrix vector multiplication operations efficiently, since the memristor chip matrix vector multiplication is implemented in an analog circuit, noise and disturbance are inevitably brought about, so compared with the neural network, the memristor The calculation results are not accurate.
  • the use of a memristor indicates that the weight has a certain error. As shown in Figure 3, the weights of different levels will overlap. In order to avoid overlap, the existing methods generally use a number of low-precision memristor splicing to represent a high-precision weight, and in the case where each memristor has a low precision, the weight data can be considered accurate. Taking a 2-bit memristor to represent a 4-bit weight as an example, a 2-bit memristor is used to indicate a lower weight of 2 bits and the other represents a high 2 bits.
  • ISAAC existing ISAAC technology first trains a neural network with floating point numbers and then "writes" the weighted data to the memristor.
  • ISAAC uses four 2-bit memristor devices to represent an 8-bit weight, which allows more resources to be used to improve matrix operation accuracy.
  • ISAAC uses splicing methods to represent weights, which is relatively inefficient and requires a lot of resources. For example, if you represent one weight, you need 4 memristor devices.
  • the existing PRIME technology first trains a neural network with floating point numbers, then uses two 3-bit precision input voltages to represent a 6-bit input, and two 4-bit memristor devices to represent an 8-bit.
  • the weights are weighted, and the positive and negative weights are represented by two sets of arrays.
  • PRIME uses positive and negative addition and high and low splicing methods to represent weights, and also requires a lot of resources. That is, to represent one weight, four memristor devices are needed.
  • the present invention has been made in view of the above circumstances.
  • a non-splicing weight training method for a neural network comprising: a weight-spotting step of converting each matrix element of a weight matrix into a first number having a predetermined number of bits; a step of introducing a noise having a predetermined standard deviation into the first number to obtain a second number; and a training step of training the weight matrix represented by the second number, training to convergence, and obtaining a training result, wherein The training result will be used as the final weight matrix, each matrix element being written one by one into a single analog circuit device corresponding to a matrix element, wherein a single matrix is represented by a single analog circuit device rather than a splicing of multiple analog circuit devices. element.
  • the first number conversion in the weight setting step, can be performed by a linear relationship or a logarithmic relationship.
  • the noise may be a read/write error of an analog circuit, and obey a normal distribution law.
  • the analog circuit device may be a memristor, a capacitor comparator or a voltage comparator.
  • the first number may be a fixed point number and the second number may be a floating point number.
  • a non-splicing weight coding method for a neural network comprising the steps of: writing each matrix element of a weight matrix one by one into a single analog circuit device corresponding to a matrix element, A single matrix element is represented by a splicing of a single analog circuit device rather than a plurality of analog circuit devices, wherein the weight matrix is obtained by the non-splicing weight training method described above.
  • the method may further include the following steps: a weight-spotting step of converting each matrix element of the weight matrix into a first number having a predetermined number of bits; and an error introduction step in Introducing noise with a predetermined standard deviation into the first number to obtain a second number; and training step, training the weight matrix represented by the second number, training until convergence, and obtaining a training result.
  • a neural network chip having a basic module for performing an operation of matrix vector multiplication in hardware by an analog circuit device, wherein each matrix element of the weight matrix is written one by one to represent one A single analog circuit device of matrix elements to represent a single matrix element of a weight matrix by splicing of a single analog circuit device rather than multiple analog circuit devices.
  • the weight matrix may be obtained by the above non-splicing weight training method.
  • a computing device includes a memory and a processor having stored thereon computer executable instructions that, when executed by a processor, perform a non-splicing weight training method according to the above Or according to the above non-splicing weight coding method.
  • a neural network system comprising: the computing device according to the above; and the neural network chip according to the above.
  • an encoding method for a neural network which can greatly reduce resource consumption without affecting effects, thereby saving resource overhead, and arranging a large-scale nerve under conditions of limited resources.
  • the internet The internet.
  • Figure 1 shows a schematic of a chained neural network.
  • Figure 2 shows a schematic diagram of a memristor based crossbar switch structure.
  • Figure 3 shows a weighted statistical distribution map of eight levels of weights on a memristor.
  • Fig. 4 shows a schematic diagram of an application scenario of an encoding technique of a neural network according to the present invention.
  • Figure 5 shows a general flow chart of an encoding method in accordance with the present invention.
  • Fig. 6 shows a comparison of experimental effects using the existing high and low level stitching method and the encoding method according to the present invention.
  • the present application provides a new encoding method (hereinafter referred to as RLevel encoding method), which is essentially different from the existing method in that the new encoding method does not require that the weight values represented by a single device do not overlap, but instead Kinds of errors are introduced into the training.
  • RLevel encoding method By training the weight matrix containing noise and enabling it to train to convergence, the converged values are finally written into a single device, thereby enhancing the noise immunity of the model and reducing the representation of matrix elements.
  • the number of costs reduces resource and resource consumption.
  • Figure 3 shows a weighted statistical distribution map of eight levels of weights on a memristor.
  • circuit devices other than the memristor capable of realizing matrix vector multiplication are also possible, such as a capacitor or a voltage comparator.
  • l and h represent low and high-order devices respectively
  • the weight is expressed as 2 n *h+l
  • the errors of low and high are L ⁇ (l, ⁇ 2 ), H ⁇ (h, ⁇ 2 ), then 2 n *H ⁇ (h, 2 2n * ⁇ 2 ).
  • the weight range is 2 2n -l
  • the standard deviation of the weight error is We use the range of values and the standard deviation as the standard for the final accuracy.
  • the accuracy of the splicing weight method is:
  • a device is used to represent a high-precision weight with an accuracy of (2 n -l) / ⁇ .
  • Fig. 4 shows a schematic diagram of an application scenario of an encoding technique of a neural network according to the present invention.
  • the general inventive concept of the present disclosure is to solve the problem that the network model 1200 employed by the neural network application 1100 is weight-encoded by the encoding method 1300, and the result is written into the memristor device of the neural network chip 1400.
  • the weight based on the memristor neural network indicates the problem of requiring a large number of devices, and finally saves a lot of resources without significant loss of accuracy.
  • FIG. 5 shows a general flow diagram of an encoding method in accordance with the present invention, comprising the following steps:
  • Weight setting process S210 converting each matrix element of the weight matrix into a first number having a predetermined number of bits
  • each weight value is converted into a fixed-point number with a certain precision, and the fixed-point weight is obtained.
  • each weight value is converted into 4 to a specific number of points.
  • step S220 in which noise having a predetermined standard deviation is introduced in the first number to obtain a second number.
  • the first number is set to a fixed point number
  • the second number is equal to the first number plus noise, so the second number is a floating point number.
  • the number of fixed points of 0, 1, 2, and 3 adds noise and becomes four floating point numbers of -0.1, 1.02, 2.03, and 2.88.
  • the first number may be a floating point number. .
  • Training step S230 training the weight matrix represented by the second number, training to convergence, and then writing the training result as a final weight matrix into the circuit device for weight matrix calculation.
  • the fixed point matrix B (Table 2) is decomposed into a high order matrix H (Table 3) and a low order matrix L (Table 4):
  • the fixed-point matrix B is converted into the conductance value of 4*10 -6 to 4*10 -5 according to the RLevel method and the high-low level splicing method respectively, and the Rlevel conductance matrix of Table 5 is obtained.
  • RC, high conductivity matrix HC and low conductivity matrix LC are converted into the conductance value of 4*10 -6 to 4*10 -5 according to the RLevel method and the high-low level splicing method respectively.
  • the training process according to the present invention does not convert the matrix into conductance values, but rather increases the noise of a normal distribution with a standard deviation of ⁇ on the basis of the first number.
  • the introduction of the actual error is caused by noise and disturbance during the reading and writing process of the memristor device or other used circuit device, so the data is based on the conductance value as the analog value below. analysis.
  • the outputs of the Rlevel conductance matrix RC, the high-order conductance matrix HC, and the low-level conductance matrix LC are respectively:
  • the above spliced output is output according to the high bit output *4+ low bit.
  • the RLevel coding method according to the present invention has very close precision to the output of the prior art high and low level splicing method, whether noise is added or not. Therefore, the solution of the present invention is verified from a theoretical point of view. Practicality and feasibility.
  • Fig. 6 shows a comparison of experimental effects using the existing high and low level stitching method and the RLevel encoding method according to the present invention.
  • This experiment used a convolutional neural network to classify the CIFAR10 data set.
  • the data set has 60,000 32*32 pixel color images, each of which belongs to one of 10 categories.
  • the abscissa is the weight precision and the ordinate is the correct rate.
  • the lower line uses the RLevel method to represent the weights of 2, 4, 6, and 8 bits by one device, and the upper line is 2 of 1, 2, 3, and 4 bits, respectively.
  • the devices are spliced to represent 2, 4, 6, and 8 bits.
  • the accuracy of the RLevel method is very close to that of the high-low-level stitching method, but since only one device is used, and no splicing of multiple devices is required, it is non-splicing. Encoding, so you can save 50% of resources.
  • the weight coding method of the present invention it is possible to provide substantially the same accuracy as the existing high and low bit splicing without using high and low bit splicing, and the weight matrix calculation of the neural network by the analog circuit such as a memristor is solved.
  • the need to arrange a large number of circuit devices also reduces costs and saves resources.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Complex Calculations (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

La présente invention concerne un procédé de codage de poids sans assemblage pour un réseau neuronal, comprenant : une étape de conversion de point fixe de poids consistant à convertir chaque élément de matrice d'une matrice de poids en un premier nombre ayant un nombre prédéterminé de bits (S210); une étape d'introduction d'erreur consistant à introduire du bruit ayant une déviation standard prédéterminée dans le premier nombre pour obtenir un second nombre (S220); et une étape d'apprentissage consistant à entraîner une matrice de poids représentée par les seconds nombres jusqu'à ce qu'une convergence se produise, puis à écrire le résultat d'apprentissage sous la forme d'une matrice de poids final dans un dispositif de circuit analogique unique représentant de manière correspondante un élément de matrice (S230), un élément de matrice unique étant représenté par un seul dispositif de circuit analogique plutôt que de multiples dispositifs de circuit analogique assemblés ensemble. Le procédé de codage pour un réseau neuronal peut réduire considérablement la consommation de ressources sans affecter l'effet, économisant ainsi un surdébit de ressources, et ainsi, un réseau neuronal à grande échelle peut être agencé avec des ressources limitées.
PCT/CN2017/119821 2017-12-29 2017-12-29 Procédé de codage de poids pour réseau neuronal, appareil informatique et système matériel Ceased WO2019127363A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2017/119821 WO2019127363A1 (fr) 2017-12-29 2017-12-29 Procédé de codage de poids pour réseau neuronal, appareil informatique et système matériel
CN201780042640.0A CN109791626B (zh) 2017-12-29 2017-12-29 神经网络权重编码方法、计算装置及硬件系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/119821 WO2019127363A1 (fr) 2017-12-29 2017-12-29 Procédé de codage de poids pour réseau neuronal, appareil informatique et système matériel

Publications (1)

Publication Number Publication Date
WO2019127363A1 true WO2019127363A1 (fr) 2019-07-04

Family

ID=66495542

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/119821 Ceased WO2019127363A1 (fr) 2017-12-29 2017-12-29 Procédé de codage de poids pour réseau neuronal, appareil informatique et système matériel

Country Status (2)

Country Link
CN (1) CN109791626B (fr)
WO (1) WO2019127363A1 (fr)

Families Citing this family (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018125928A1 (fr) 2016-12-29 2018-07-05 DeepScale, Inc. Simulation de capteur multicanal pour systèmes de commande autonomes
WO2018176000A1 (fr) 2017-03-23 2018-09-27 DeepScale, Inc. Synthèse de données pour systèmes de commande autonomes
US11409692B2 (en) 2017-07-24 2022-08-09 Tesla, Inc. Vector computational unit
US11157441B2 (en) 2017-07-24 2021-10-26 Tesla, Inc. Computational array microprocessor system using non-consecutive data formatting
US10671349B2 (en) 2017-07-24 2020-06-02 Tesla, Inc. Accelerated mathematical engine
US11893393B2 (en) 2017-07-24 2024-02-06 Tesla, Inc. Computational array microprocessor system with hardware arbiter managing memory requests
US12307350B2 (en) 2018-01-04 2025-05-20 Tesla, Inc. Systems and methods for hardware-based pooling
US11561791B2 (en) 2018-02-01 2023-01-24 Tesla, Inc. Vector computational unit receiving data elements in parallel from a last row of a computational array
US11215999B2 (en) 2018-06-20 2022-01-04 Tesla, Inc. Data pipeline and deep learning system for autonomous driving
US11361457B2 (en) 2018-07-20 2022-06-14 Tesla, Inc. Annotation cross-labeling for autonomous control systems
US11636333B2 (en) 2018-07-26 2023-04-25 Tesla, Inc. Optimizing neural network structures for embedded systems
US11562231B2 (en) 2018-09-03 2023-01-24 Tesla, Inc. Neural networks for embedded devices
ES3063986T3 (en) 2018-10-11 2026-04-21 Tesla Inc Systems and methods for training machine models with augmented data
US11196678B2 (en) 2018-10-25 2021-12-07 Tesla, Inc. QOS manager for system on a chip communications
US11816585B2 (en) 2018-12-03 2023-11-14 Tesla, Inc. Machine learning models operating at different frequencies for autonomous vehicles
US11537811B2 (en) 2018-12-04 2022-12-27 Tesla, Inc. Enhanced object detection for autonomous vehicles based on field view
US11610117B2 (en) 2018-12-27 2023-03-21 Tesla, Inc. System and method for adapting a neural network model on a hardware platform
US11150664B2 (en) 2019-02-01 2021-10-19 Tesla, Inc. Predicting three-dimensional features for autonomous driving
US10997461B2 (en) 2019-02-01 2021-05-04 Tesla, Inc. Generating ground truth for machine learning from time series elements
US11567514B2 (en) 2019-02-11 2023-01-31 Tesla, Inc. Autonomous and user controlled vehicle summon to a target
US10956755B2 (en) 2019-02-19 2021-03-23 Tesla, Inc. Estimating object properties using visual image data
CN114341884B (zh) * 2019-09-09 2025-07-25 高通股份有限公司 用于针对二进制处理应用来修改神经网络的系统和方法
CN110796241B (zh) * 2019-11-01 2022-06-17 清华大学 基于忆阻器的神经网络的训练方法及其训练装置
CN111027619B (zh) * 2019-12-09 2022-03-15 华中科技大学 一种基于忆阻器阵列的K-means分类器及其分类方法
CN113344170B (zh) 2020-02-18 2023-04-25 杭州知存智能科技有限公司 神经网络权重矩阵调整方法、写入控制方法以及相关装置
WO2021163866A1 (fr) * 2020-02-18 2021-08-26 杭州知存智能科技有限公司 Procédé de réglage de matrice de pondération de réseau neuronal, procédé de commande d'écriture et dispositif associé
CN114169489A (zh) * 2020-08-20 2022-03-11 华为技术有限公司 一种神经网络计算芯片的权重调整方法及装置
CN115481562B (zh) * 2021-06-15 2023-05-16 中国科学院微电子研究所 多并行度优化方法、装置、识别方法和电子设备
US12462575B2 (en) 2021-08-19 2025-11-04 Tesla, Inc. Vision-based machine learning model for autonomous driving with adjustable virtual camera
KR20240048533A (ko) 2021-08-19 2024-04-15 테슬라, 인크. 시뮬레이션 콘텐트를 사용한 비전 기반 시스템 훈련
CN114282478B (zh) * 2021-11-18 2023-11-17 南京大学 一种修正可变电阻器件阵列点乘误差的方法
US12623691B2 (en) 2022-05-20 2026-05-12 Tesla, Inc. Fail-safe corrective actions based on vision information for autonomous vehicles

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105224986A (zh) * 2015-09-29 2016-01-06 清华大学 基于忆阻器件的深度神经网络系统
US20170061281A1 (en) * 2015-08-27 2017-03-02 International Business Machines Corporation Deep neural network training with native devices
CN106650922A (zh) * 2016-09-29 2017-05-10 清华大学 硬件神经网络转换方法、计算装置、编译方法和神经网络软硬件协作系统
CN107085628A (zh) * 2017-03-21 2017-08-22 东南大学 一种细胞神经网络可调权值模块仿真方法

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10580401B2 (en) * 2015-01-27 2020-03-03 Google Llc Sub-matrix input for neural network layers
CN106796668B (zh) * 2016-03-16 2019-06-14 香港应用科技研究院有限公司 用于人工神经网络中比特深度减少的方法和系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170061281A1 (en) * 2015-08-27 2017-03-02 International Business Machines Corporation Deep neural network training with native devices
CN105224986A (zh) * 2015-09-29 2016-01-06 清华大学 基于忆阻器件的深度神经网络系统
CN106650922A (zh) * 2016-09-29 2017-05-10 清华大学 硬件神经网络转换方法、计算装置、编译方法和神经网络软硬件协作系统
CN107085628A (zh) * 2017-03-21 2017-08-22 东南大学 一种细胞神经网络可调权值模块仿真方法

Also Published As

Publication number Publication date
CN109791626B (zh) 2022-12-27
CN109791626A (zh) 2019-05-21

Similar Documents

Publication Publication Date Title
CN109791626B (zh) 神经网络权重编码方法、计算装置及硬件系统
US20260004118A1 (en) Analog neuromorphic circuit implemented using resistive memories
CN108009640B (zh) 基于忆阻器的神经网络的训练装置及其训练方法
US11748609B2 (en) On-chip training of memristor crossbar neuromorphic processing systems
CN109791628B (zh) 神经网络模型分块压缩方法、训练方法、计算装置及系统
US20220374688A1 (en) Training method of neural network based on memristor and training device thereof
Kim et al. Input voltage mapping optimized for resistive memory-based deep neural network hardware
US12217164B2 (en) Neural network and its information processing method, information processing system
US20210209450A1 (en) Compressed weight distribution in networks of neural processors
CN107423816A (zh) 一种多计算精度神经网络处理方法和系统
WO2023130725A1 (fr) Procédé et appareil de mise en œuvre matérielle pour modèle informatique de réservoir basé sur un réseau de résistances aléatoires, et dispositif électronique
CN107944545A (zh) 应用于神经网络的计算方法及计算装置
CN106203625A (zh) 一种基于多重预训练的深层神经网络训练方法
US20250200348A1 (en) Model Compression Method and Apparatus, and Related Device
CN108647184B (zh) 一种动态比特位卷积乘法实现方法
CN111539522A (zh) 基于固定尺寸忆阻阵列的大规模ncs容错框架的构建方法
CN118839766A (zh) 一种面向多边缘设备的Transformer模型协同推理方法
CN113705784B (zh) 一种基于矩阵共享的神经网络权重编码方法及硬件系统
CN114997385B (zh) 应用于神经网络的存内计算架构的操作方法、装置和设备
WO2023240578A1 (fr) Procédé, appareil et dispositif de fonctionnement pour une architecture informatique en mémoire destinée à être utilisée dans un réseau neuronal
CN119476396B (zh) 一种适用于忆阻型类脑芯片硬件部署的神经网络训练方法
Lu et al. NVMLearn: a simulation platform for non-volatile-memory-based deep learning hardware
KR102412872B1 (ko) 프로세싱 엘리먼트, 이의 동작 방법, 및 이를 포함하는 가속기
US11977432B2 (en) Data processing circuit and fault-mitigating method
CN117610636A (zh) 一种存内计算存储器人工神经网络的片上训练方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17936307

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17936307

Country of ref document: EP

Kind code of ref document: A1