CN118519613A - Multiply-accumulator operation cluster and data processing method - Google Patents

Abstract

The application discloses a multiply accumulator operation cluster and a data processing method. The multiply-accumulator operation cluster comprises a plurality of multiply-accumulator units, each multiply-accumulator unit is used for completing multiply-accumulator operation of at least one group of data to be processed, and when the calculated dimension of the data to be processed is smaller than the total dimension of the multiply-accumulator units, at least one multiply-accumulator subunit in the multiply-accumulator units is used for completing multiply-accumulator operation of each group of data to be processed, and outputting multiply-accumulate operation results of the data to be processed; and/or when the data to be processed comprises a plurality of data blocks which are sequentially arranged, respectively completing multiply-accumulate operation of one data block through a plurality of multiply-accumulator diversity groups in the multiply-accumulator operation cluster, and outputting the multiply-accumulate operation result of the data block to the next multiply-accumulator diversity group which is sequentially arranged until the multiply-accumulate operation of all the data blocks is completed, and outputting the multiply-accumulate operation result of the data to be processed. The multiply-accumulator operation cluster improves the utilization rate of the multiply-accumulator operation unit.

Description

Multiply-accumulate operation cluster and data processing method

Technical Field

The present application belongs to the field of digital circuit technology, and specifically relates to a multiplier-accumulator operation cluster and a data processing method.

Background Art

The Multiply and Accumulate (MAC) is used to perform multiplication and accumulation operations such as vector multiplication, matrix multiplication, and vector-matrix multiplication. It is an extremely important computing subsystem in processors such as coprocessors, digital signal processors, and central processing units (CPUs), especially the most basic computing unit in artificial intelligence (AI) accelerators.

The MAC in an AI accelerator can generally implement the dot product of X groups of M-dimensional vectors. For example, the multiplication and addition operations of two groups of M-dimensional vectors (vector A and vector B) are completed by first completing the A1 •B1, A2 •B2 ...AX •BX operations, and then adding the products together. The processing dimension of each vector (A or B) is represented by M. Each data may be an integer or floating point number of different bit widths. Taking the input as an 8-bit integer as an example, the computing power of each clock cycle of the MAC is M times X times 2 (2 is understood as two operations of a multiplication and an addition) of 8-bit integer operations. However, in actual applications, the utilization rate of the MAC operation unit is low due to some reasons. For example, when the processing dimension of vector A or vector B is less than the dimension M of the multiplication accumulator (the dimension of the multiplication accumulator represents the number of multipliers contained), some multipliers will be idle when processing data. Only when this processing is completed can the idle multipliers be used for the next time, resulting in low utilization of the multipliers in the multiplication accumulator. For example, if the input data of the multiplier-accumulator is not prepared in time or the output data of the multiplier-accumulator is not output in time, the line will be blocked and the multiplier-accumulator cannot be used next time, which will also lead to low utilization of the multiplier-accumulator operation unit.

Currently, there is no good circuit structure that can solve the problem of low utilization of the multiplication and accumulation unit.

Summary of the invention

In view of this, the present application provides a multiplier-accumulator operation cluster and a data processing method, the main purpose of which is to solve the problem of low utilization rate of the existing multiplier-accumulator operation units.

To solve the above problems, the present application provides a multiplication-accumulation operation cluster, the multiplication-accumulation operation cluster includes a plurality of multiplication-accumulation units, each of the multiplication-accumulation units is used to complete a multiplication-accumulation operation of at least one group of data to be processed, each group of the data to be processed includes two multi-dimensional data, each of the multiplication-accumulation units includes a plurality of multiplication-accumulation sub-units, each of the multiplication-accumulation sub-units is used to complete a multiplication-accumulation operation and output a multiplication-accumulation operation result, wherein,

When the calculation dimension of two of the multi-dimensional data in the data to be processed is smaller than the total dimension of the multiplication-accumulation unit, the multiplication-accumulation operation of each group of the data to be processed is completed by at least one multiplication-accumulation sub-unit in the multiplication-accumulation unit, and the multiplication-accumulation operation result of the data to be processed is output; and/or, when the data to be processed includes a plurality of sequentially arranged data blocks, the multiplication-accumulation operation of one data block is completed respectively by a plurality of multiplication-accumulation sub-clusters in the multiplication-accumulation operation cluster, and the multiplication-accumulation operation result of the data block is output to the next sequentially arranged multiplication-accumulation sub-cluster, until the multiplication-accumulation operation of all the data blocks is completed, and the multiplication-accumulation operation result of the data to be processed is output.

In one embodiment of the present application, optionally, the dimensions of the multiplication accumulator sub-units in the same multiplication accumulator unit are the same, the sum of the dimensions of all the multiplication accumulator sub-units in each of the multiplication accumulator units is the same as the total dimension of the multiplication accumulator unit, and the dimensions of the multiplication accumulator sub-units in multiple different multiplication accumulator units are the same as or different from each other.

In one embodiment of the present application, optionally, when the computational dimension of a group of data to be processed is less than or equal to the dimension of a multiplication accumulator sub-unit in at least one multiplication accumulator unit, a target multiplication accumulator unit for operating the data to be processed is determined according to the computational dimension of the data to be processed and the dimension of the multiplication accumulator sub-unit in the multiplication accumulator unit, and multiplication and accumulation operations are performed on the data to be processed through the multiplication accumulator sub-unit in the target multiplication accumulator unit.

In one embodiment of the present application, optionally, when the calculation dimension of a group of data to be processed is greater than the dimension of the multiplication and accumulation sub-units in all the multiplication and accumulation units and less than the total dimension of each of the multiplication and accumulation units, the multiplication and accumulation operations of the data to be processed are completed by multiple multiplication and accumulation sub-units in the multiplication and accumulation units.

In one embodiment of the present application, optionally, when the computational dimension of the data to be processed is greater than the dimension of the multiplication accumulator subunits in all the multiplication accumulator units and less than the total dimension of each of the multiplication accumulator units, a target multiplication accumulator for operating the data to be processed and a target number of multiplication accumulator subunits in the target multiplication accumulator are determined according to the computational dimension of the data to be processed and the dimension of the multiplication accumulator subunits in each multiplication accumulator unit, and multiplication and accumulation operations are performed on the data to be processed through the target number of multiplication accumulator subunits in the target multiplication accumulator.

In one embodiment of the present application, optionally, at least one multiplier accumulator sub-unit in the multiplier accumulator operation cluster serves as a multiplier accumulator sub-cluster.

In one embodiment of the present application, optionally, when the computational dimensions of all data blocks in the data to be processed are less than or equal to the dimension of the multiplication and accumulation sub-unit in any multiplication and accumulation unit, each multiplication and accumulation sub-unit in the multiplication and accumulation unit is used as a multiplication and accumulation cluster, and the multiplication and accumulation operation of one data block is completed respectively by multiple multiplication and accumulation sub-units in the multiplication and accumulation unit, and the multiplication and accumulation operation result of the data block is output to the next multiplication and accumulation sub-unit arranged in sequence, until the multiplication and accumulation operation of all the data blocks is completed, and the multiplication and accumulation operation result of the data to be processed is output.

In one embodiment of the present application, optionally, the multiplication and accumulation unit includes a plurality of data input modules, a plurality of multiplication operation modules, a plurality of multi-stage addition operation modules and a plurality of data output modules, wherein one multiplication operation module, one multi-stage addition operation module and a plurality of data output modules constitute a multiplication and accumulation sub-unit, each of the multiplication operation modules includes a plurality of multipliers, and each of the multi-stage addition operation modules includes a plurality of adders;

The input end of each of the data input modules is electrically connected to the data providing module, the output end of each of the data input modules is electrically connected to the input end of a multiplication operation module, the output end of each of the multiplication operation modules is electrically connected to the input end of a multi-stage addition operation module, and the output end of each of the multi-stage addition operation modules is electrically connected to multiple data output modules.

The present application also provides a data processing method, which is applied to the multiplication and accumulation operation cluster described above, and the data processing method includes:

Acquire data to be processed, and determine a target multiplication and accumulation unit corresponding to the data to be processed according to the calculation dimension of the data to be processed and the dimension of the multiplication and accumulation subunit in each multiplication and accumulation unit;

The data to be processed are input into a multiplication-accumulation subunit in an idle state in the target multiplication-accumulation unit, and a multiplication-accumulation operation is performed on the data to be processed by the target multiplication-accumulation unit to obtain a multiplication-accumulation operation result of the data to be processed.

In one embodiment of the present application, optionally, the acquiring the data to be processed, and determining the target multiplication and accumulation unit corresponding to the data to be processed according to the calculation dimension of the data to be processed and the sub-dimension of the multiplication and accumulation sub-unit in each multiplication and accumulation unit, includes:

A multiplication accumulator unit in which the dimension of the multiplication accumulator sub-unit is greater than or equal to the calculation dimension of the data to be processed is taken as a candidate multiplication accumulator unit; a candidate multiplication accumulator unit that is in an idle working state and has the largest dimension of multiplication accumulator sub-unit is taken as a target multiplication accumulator unit corresponding to the data to be processed.

Beneficial effects of the present application: The present application provides a multiplication-accumulator operation cluster and a data processing method, wherein the multiplication-accumulator operation cluster includes a plurality of multiplication-accumulator units, each of which includes a plurality of multiplication-accumulator sub-units. When the calculation dimension of two multi-dimensional data in the to-be-processed data is smaller than the total dimension of the multiplication-accumulator unit, the plurality of to-be-processed data are respectively input into a multiplication-accumulator sub-unit. Within the same clock cycle, the plurality of multiplication-accumulator sub-units in each multiplication-accumulator unit can simultaneously perform multiplication-accumulation operations on the to-be-processed data inputted by them, thereby improving the utilization rate of the multiplication-accumulator operation unit and increasing the The output bandwidth of the multiplier-accumulator is increased; when the data to be processed includes multiple sequentially arranged data blocks, one multiplier-accumulator sub-cluster completes a multiplication-accumulation operation of one data block, and outputs the multiplication-accumulation operation result of the data block to the next sequentially arranged multiplier-accumulator sub-cluster, thereby realizing the processing of the data to be processed in a pipeline working mode. In the same clock cycle, multiple multiplier-accumulator sub-clusters process their respective input data at the same time, and the input data of the multiplier-accumulator is prepared in time, and the output end outputs the operation result in time, thereby avoiding the problem of line blocking and improving the utilization rate of the multiplier-accumulator operation unit.

The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

BRIEF DESCRIPTION OF THE DRAWINGS

Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

FIG1 is a schematic diagram of the structure of a traditional multiplier-accumulator unit;

FIG2 is a block diagram of a multiplier-accumulator operation cluster according to an exemplary embodiment of the present application;

FIG3 is a structural block diagram of another multiplier-accumulator operation cluster of an exemplary embodiment of the present application;

FIG4 is a schematic diagram of the operation of a traditional multiply-accumulate unit;

FIG5 is a schematic diagram of the operation of a multiplication and accumulation unit according to an exemplary embodiment of the present application;

FIG6 is a schematic diagram of a pipeline processing method for processing data using a traditional multiplication and accumulation unit;

FIG7 is a schematic diagram of using a multiplication and accumulation unit for processing data in a pipeline processing manner according to an exemplary embodiment of the present application;

FIG8 is a structural block diagram of another multiplier-accumulator operation cluster according to an exemplary embodiment of the present application;

FIG9 is a schematic diagram of the structural connection of a multiplier-accumulator operation cluster according to an exemplary embodiment of the present application;

FIG10 is a schematic diagram of the structural connection of a multiplier-accumulator subunit of an exemplary embodiment of the present application;

FIG11 is a flow chart of a data processing method according to an exemplary embodiment of the present application;

in,

The numbers in the figure are as follows: 1-multiplication accumulator unit; 11-multiplication accumulator sub-unit; 2-multiplication accumulator sub-cluster; 12-data input module; 111-multiplication operation module; 112-multi-stage addition operation module; 113-data output module.

DETAILED DESCRIPTION

The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

In order to further explain the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present application are described in detail below in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form.

In the prior art, the commonly used multiplier-accumulator unit has only one data output terminal, as shown in FIG1 , where D is the data output terminal. If there is an 8-bit integer MAC in an AI processor, where M=32, X=16, and M is the number of multipliers. When performing a normal convolution operation, such as a 3x3 depthwise convolution operation, only 9 multipliers are used when performing a 3x3 convolution operation. The MAC under the usual architecture has only one data output terminal. The multipliers used in the operation participate in the calculation, and the unused multipliers are in an idle waiting state. If this 3x3 convolution is calculated, the number of multipliers used is X'=9, and there are 32 multipliers in the multiplier-accumulator unit. There will be many multipliers in an idle state. Only after the convolution calculation is completed and the calculation result is output, the entire MAC will wait for the next operation. The operation utilization rate of the MAC during the 3x3 convolution operation is only 9/32, which is relatively low.

The present application proposes a multiplication-accumulation operation cluster, as shown in FIG2 , the multiplication-accumulation operation cluster includes a plurality of multiplication-accumulation units 1, each of which is used to complete a multiplication-accumulation operation of at least one group of data to be processed, each group of data to be processed includes two multi-dimensional data, each multiplication-accumulation unit 1 includes a plurality of multiplication-accumulation sub-units 11, each of which is used to complete a multiplication-accumulation operation and output a multiplication-accumulation operation result, wherein when the calculation dimension of the two multi-dimensional data in the data to be processed is smaller than the total dimension of the multiplication-accumulation unit 1, the multiplication-accumulation sub-unit 11 is multiplied by the multiplication-accumulation unit 11. At least one multiplication-accumulation sub-unit 11 in the accumulator unit 1 completes the multiplication-accumulation operation of each group of data to be processed, and outputs the multiplication-accumulation operation result of the data to be processed; and/or, as shown in Figure 3, when the data to be processed includes multiple sequentially arranged data blocks, the multiplication-accumulation operation of one data block is completed respectively by multiple multiplication-accumulation sub-clusters 2 in the multiplication-accumulation operation cluster, and the multiplication-accumulation operation result of the data block is output to the next sequentially arranged multiplication-accumulation sub-cluster 2, until the multiplication-accumulation operation of all data blocks is completed, and the multiplication-accumulation operation result of the data to be processed is output.

Specifically, the multiplication and accumulation operation cluster includes multiple multiplication and accumulation units, each of which includes at least one multiplication and accumulation sub-unit, and each multiplication and accumulation sub-unit can independently process the data to be processed. In the same clock cycle, when multiple groups of data to be processed are respectively sent to multiple multiplication and accumulation sub-units in the same multiplication and accumulation unit, the multiple multiplication and accumulation sub-units respectively perform multiplication and accumulation operations on their respective input data to be processed, and each outputs an operation result, so that in the same clock cycle, the same multiplication and accumulation unit can perform operation processing on multiple data to be processed, thereby improving the operation utilization rate of the multiplication and accumulation unit.

When the computational dimension of the data to be processed is smaller than the dimension of the multiplication and accumulation unit, only a small amount of MAC computing power can be effectively utilized when the data to be processed uses the traditional multiplication and accumulation unit. As shown in FIG4, when there is data to be processed with a 1/4 dimension of the multiplication and accumulation unit input, only 1/4 of the multipliers in the traditional multiplication and accumulation unit are used, and the remaining multipliers are idle, and the computing utilization rate is very low. When the multiplication and accumulation unit is divided into multiple multiplication and accumulation sub-units, for example, as shown in FIG5, when the multiplication and accumulation unit is divided into 4 multiplication and accumulation sub-units, the total input dimension of the multiplication and accumulation unit remains unchanged, and the input dimension of each multiplication and accumulation sub-unit is 1/4 of the total dimension. In one clock cycle, there are 4 groups of 1/4 dimension data to be processed, each of which is input into a multiplication and accumulation sub-unit, and each multiplication and accumulation sub-unit performs data processing separately. Each multiplier in the multiplication and accumulation unit is used, and all computing power is utilized, thereby improving the computing utilization rate.

For example, a multiplication-accumulator unit includes four multiplication-accumulator sub-units, and the number of multipliers included in each multiplication-accumulator sub-unit is 9, that is, the dimension of the multiplication-accumulator sub-unit is 9, and the total dimension of the multiplication-accumulator unit is 36. If there are two groups of data to be input, the calculation dimensions of the two groups of data to be processed are 8 and 9 respectively. In the same clock cycle, the two multiplication-accumulator sub-units in the multiplication-accumulator unit perform multiplication-accumulation operations on the two groups of data to be processed respectively, and each obtains an output result. At this time, the operation utilization rate of the multiplication-accumulator unit is 8+9/36. If the two data to be processed are processed by the traditional multiplication-accumulator unit, two clock cycles are required, and each clock cycle processes a group of data to be processed. The operation utilization rates of the multiplication-accumulator unit in the two clock cycles are 8/36 and 9/36 respectively. However, the multiplication-accumulator operation cluster of the present application can be completed in the same clock cycle, and the operation utilization rate of the multiplication-accumulator unit is high.

When the input data of the multiplier-accumulator operation cluster is not prepared in time or the output data of the multiplier-accumulator operation cluster is not output in time, the line is blocked, resulting in the multiplier-accumulator operation cluster being unable to enter the next use, which also leads to low utilization of the multiplier-accumulator operation unit.

When the data to be processed is data to be processed in an assembly line processing manner, if a traditional multiplier-accumulator is used, after the data to be processed passes through the multiplier-accumulator for the first multiplication-accumulation operation, the operation result is input into the multiplication-accumulator again for the second multiplication-accumulation operation, and the operation result is input into the multiplication-accumulator unit again until the operation of the data to be processed is completed. The operation utilization rate of the multiplication-accumulator is relatively low.

Therefore, for data processed in a pipeline processing manner, the multiplication and accumulation unit operation cluster is divided into a plurality of multiplication and accumulation unit sub-clusters, and the plurality of multiplication and accumulation unit sub-clusters are arranged in sequence. The data to be processed includes a plurality of sequentially arranged data blocks. In each clock cycle, one data block is input to the accumulator sub-clusters in the first arrangement order. In the first clock cycle, the multiplication and accumulation unit sub-clusters in the first arrangement order process the input data, and the processing result is transmitted to the data input end of the multiplication and accumulation unit sub-clusters in the second arrangement order. In the second clock cycle, a new data block enters the multiplication and accumulation unit sub-clusters in the first arrangement order. The multiplication and accumulation unit sub-clusters in the first arrangement order and the multiplication and accumulation unit sub-clusters in the second arrangement order respectively process the input data, and the multiplication and accumulation unit sub-clusters in the second arrangement order are transmitted to the data input end of the multiplication and accumulation unit sub-clusters in the second arrangement order. The processing results of the clusters are transmitted to the data input terminals of the multiplier-accumulator sub-clusters of the third arrangement order, and the processing results of the multiplier-accumulator sub-clusters of the first arrangement order are transmitted to the data input terminals of the multiplier-accumulator sub-clusters of the second arrangement order. In the third clock cycle, a new data block is sent to the input terminals of the multiplier-accumulator sub-clusters of the first arrangement order, and the above process is repeated to input the output data of the multiplier-accumulator sub-clusters of the previous arrangement order to the data input terminals of the adjacent multiplier-accumulator sub-clusters of the next arrangement order, so that in the same clock cycle, multiple multiplier-accumulator sub-clusters process their respective input data at the same time, and the data input terminals of each multiplier-accumulator sub-cluster prepare data in time, and the data output terminals take away data in time, thereby improving the operational utilization of the multiplier-accumulators.

For example, the processing of a neural network model is often defined as the operation process of several layer network layers in sequence. Usually, the calculation process of several layer network layers is that the layer1 network layer calls the entire MAC to perform calculations, and its input and output bandwidths are both W. The MAC calculation result is the input data of the layer2 network layer. After all the calculations of the layer1 network layer are completed, the layer2 network layer continues to call the entire MAC to perform calculations until the entire calculation is completed. As shown in Figure 6, it can be considered as the time-sharing multiplexing of MAC in the AI accelerator system, that is, each layer network layer uses the entire MAC, and the calculation utilization rate of MAC is relatively low. The bandwidth at this time can be considered as the bandwidth of each layer network layer.

The multiplier-accumulator operation cluster proposed in the present application includes multiple multiplier-accumulator sub-clusters, for example, the multiplier-accumulator operation cluster includes four multiplier-accumulator sub-clusters P1_MAC, P2_MAC, P3_MAC, and P4_MAC. A pipeline operation is used between multiple layer network layers, where the layer1 network layer uses the multiplier-accumulator sub-cluster P1_MAC for operation processing, the layer2 network layer uses the multiplier-accumulator sub-cluster P2_MAC for operation processing, the layer3 network layer uses the multiplier-accumulator sub-cluster P3_MAC for operation processing, and the layer4 network layer uses the multiplier-accumulator sub-cluster P4_MAC for operation processing. In the first clock cycle, the data to be processed is input to the input end of the multiplication and accumulation sub-cluster P1_MAC. After the operation of the multiplication and accumulation sub-cluster P1_MAC is completed, the operation result of the multiplication and accumulation sub-cluster P1_MAC is used as the input data of the multiplication and accumulation sub-cluster P2_MAC. In the second clock cycle, new data to be processed is input to the input end of the multiplication and accumulation sub-cluster P1_MAC. The multiplication and accumulation sub-cluster P1_MAC and the multiplication and accumulation sub-cluster P2_MAC perform operation on their respective input data at the same time. After the operation is completed, the operation result of the multiplication and accumulation sub-cluster P1_MAC is used as the input data of the multiplication and accumulation sub-cluster P2_MAC. The result is used as the input data of the multiplier-accumulator sub-cluster P3_MAC. Similarly, the processing result of the multiplier-accumulator sub-cluster of the previous arrangement order is used as the input data of the multiplier-accumulator sub-cluster of the next arrangement order. As shown in Figure 7, P1_MAC has data to be processed input in sequence, and the corresponding P2_MAC~P4_MAC also have data to be processed input in sequence. In the same clock cycle, P1_MAC~P4_MAC each calculates and processes the input data, and the total computing power of the entire multiplier-accumulator sub-cluster is improved. The output bandwidth becomes the sum of the bandwidth of each layer network layer. It can be considered that the output bandwidth becomes 4 times the original, which meets the purpose of increasing the bandwidth, but the output bandwidth of each multiplier-accumulator sub-cluster remains unchanged.

The multiplication and accumulation operation cluster includes a plurality of multiplication and accumulation units, each of which includes at least one multiplication and accumulation sub-unit. Each multiplication and accumulation sub-unit can independently process the data to be processed within the same clock cycle, thereby improving the operation utilization rate of the multiplication and accumulation unit. However, the output bandwidth of the multiplication and accumulation unit increases a lot. The increased output bandwidth may cause the subsequent processing module to fail to process in time. If the data output by the multiplication and accumulation unit cannot be taken away in time, the multiplication and accumulation unit cannot perform the next operation, causing line congestion, and affecting the operation efficiency of the multiplication and accumulation unit. Therefore, the multiplication and accumulation sub-unit is combined with the multiplication and accumulation sub-cluster, and at least one multiplication and accumulation sub-unit constitutes the multiplication and accumulation sub-cluster. As shown in FIG8 , a multiplication and accumulation sub-unit is used as a multiplication and accumulation sub-cluster. The output data of the previous multiplication and accumulation sub-unit is sent to the data input end of the next multiplication and accumulation sub-unit in order of arrangement, so that the output data of the multiplication and accumulation sub-unit is processed in time without causing line congestion, thereby improving the operation utilization rate of the multiplication and accumulation operation cluster.

Compared with the prior art, the present application provides a multiplication-accumulator operation cluster, which includes multiple multiplication-accumulator units, each of which includes multiple multiplication-accumulator sub-units. When the calculation dimension of two multi-dimensional data in the data to be processed is smaller than the total dimension of the multiplication-accumulator unit, within the same clock cycle, the multiple multiplication-accumulator sub-units in each multiplication-accumulator unit can simultaneously perform multiplication-accumulation operations on the data to be processed that are input by each of them, thereby improving the utilization rate of the multiplication-accumulator operation unit and increasing the output bandwidth of the multiplication-accumulator. When the data to be processed includes multiple sequentially arranged data blocks, a multiplication-accumulator sub-cluster completes a multiplication-accumulation operation of a data block, and outputs the multiplication-accumulation operation result of the data block to the next sequentially arranged multiplication-accumulator sub-cluster, thereby realizing the processing of the data to be processed in a pipeline working mode. Within the same clock cycle, multiple multiplication-accumulator sub-clusters simultaneously process the data input by each of them, and the input data of the multiplication-accumulator is prepared in time, and the output end outputs the operation result in time, thereby avoiding the problem of line blockage and improving the utilization rate of the multiplication-accumulator operation unit.

In one embodiment, the dimensions of the multiplication accumulator sub-units in the same multiplication accumulator unit are the same, the sum of the dimensions of all the multiplication accumulator sub-units in each multiplication accumulator unit is the same as the total dimension of the multiplication accumulator unit, and the dimensions of the multiplication accumulator sub-units in multiple different multiplication accumulator units are the same as or different from each other.

Specifically, the number of multipliers included in the multiplication-accumulator unit is the dimension of the multiplication-accumulator unit. The number of multipliers in the multiplication-accumulator sub-units included in the same multiplication-accumulator unit is the same, and the total number of multipliers in all the multiplication-accumulation sub-units in each multiplication-accumulator is equal to the total dimension of each multiplication-accumulator unit. There are multiple multiplication-accumulator units in the multiplication-accumulator operation cluster, and there must be some multiplication-accumulator units with different numbers of multipliers in the multiple multiplication-accumulator units. The number of multiplication-accumulator units with the same number of multipliers can be multiple. For example, the multiplication-accumulator operation cluster includes multiple multiplication-accumulator units, wherein the number of multipliers included in the multiplication-accumulator sub-units in two multiplication-accumulator units is 9, the number of multipliers included in the multiplication-accumulator sub-unit in one multiplication-accumulator unit is 12, the number of multipliers included in the multiplication-accumulator sub-unit in one multiplication-accumulator unit is 16, and so on.

In one embodiment, when the computational dimension of a group of data to be processed is less than or equal to the dimension of a multiplication-accumulator sub-unit in at least one multiplication-accumulator unit, a target multiplication-accumulator unit for operating the data to be processed is determined based on the computational dimension of the data to be processed and the dimension of the multiplication-accumulator sub-unit in the multiplication-accumulator unit, and multiplication-accumulation operations are performed on the data to be processed through the multiplication-accumulator sub-unit in the target multiplication-accumulator unit.

Specifically, when the computational dimension of a group of data to be processed is less than or equal to the dimension of a multiplication accumulator sub-unit in at least one multiplication accumulator unit, the computational dimension of the group of processed data is compared with the dimensions of the multiplication accumulator sub-units in these multiplication accumulator units, and the multiplication accumulator unit with the smallest difference between the computational dimension and the dimension of the multiplication accumulator sub-unit is selected as the target multiplication accumulator unit, and multiplication and addition operations are performed using the multiplication accumulator sub-units in the target multiplication accumulator unit that are in an idle state. For example, when the dimensions of the multiplication and accumulation sub-units in multiple multiplication and accumulation units are 6, 9, 12 and 16, and the number of multipliers required for processing data is 8, the multiplication and accumulation units whose dimensions are greater than 8 are taken as candidate multiplication and accumulation units, that is, the multiplication and accumulation units whose dimensions are 9, 12 and 16 are taken as candidate multiplication and accumulation units, and the difference between the calculation dimension and the dimension of the multiplication and accumulation sub-unit in each candidate multiplication and accumulation unit is obtained. The candidate multiplication and accumulation unit with the smallest difference is the candidate multiplication and accumulation unit with the dimension of the multiplication and accumulation sub-unit being 9, and this candidate multiplication and accumulation unit is taken as the target multiplication and accumulation unit.

In one embodiment, when the computational dimension of a set of data to be processed is greater than the dimension of the multiplication and accumulation sub-units in all the multiplication and accumulation units and less than the total dimension of each multiplication and accumulation unit, the multiplication and accumulation operation of the data to be processed is completed by multiple multiplication and accumulation sub-units in the multiplication and accumulation unit.

Specifically, when the computational dimension of a group of data to be processed is larger than the dimension of the multiplication accumulator sub-units in all the multiplication accumulator units, it means that one multiplication accumulator sub-unit in the multiplication accumulator unit cannot be used for calculation and processing, and it is necessary to use multiple multiplication accumulator sub-units in the multiplication accumulator unit to combine and process the data to be processed, that is, the data to be processed is jointly processed by multiple multiplication accumulator sub-units in the multiplication accumulator unit.

In one embodiment, when the computational dimension of the data to be processed is greater than the dimension of the multiplication accumulator sub-units in all the multiplication accumulator units and less than the total dimension of each multiplication accumulator unit, a target multiplication accumulator for operating the data to be processed and a target number of multiplication accumulator sub-units in the target multiplication accumulator are determined according to the computational dimension of the data to be processed and the dimension of the multiplication accumulator sub-units in each multiplication accumulator unit, and multiplication and accumulation operations are performed on the data to be processed through the target number of multiplication accumulator sub-units in the target multiplication accumulator.

Specifically, when the computational dimension of a group of data to be processed is greater than the dimension of the multiplication accumulator subunits in all the multiplication accumulator units, it is necessary to use multiple multiplication accumulator subunits in the multiplication accumulator unit to communicate and process the data to be processed. The quotient of the computational dimension of the data to be processed and the dimension of the multiplication accumulator subunit in any multiplication accumulator unit plus one is used as the target number, and the data to be processed is processed by the target number of multiplication accumulator subunits in this multiplication accumulator unit.

In one embodiment, at least one MACC sub-unit in a MACC operation cluster serves as a MACC sub-cluster.

Specifically, when multiple MACC subunits in the same MACC unit perform calculation processing independently, the calculation utilization rate of the MACC unit is improved, but at the same time, the output bandwidth of the MACC unit becomes multiple times of the original, and the increase in output bandwidth will cause processing line blockage. Therefore, at least one MACC subunit is used as a MACC sub-cluster, and the MACC sub-cluster processes the data to be processed in a pipeline operation manner, that is, the data output by the MACC sub-cluster of the previous arrangement order is input to the data input end of the MACC sub-cluster of the next arrangement order. In this way, although the output bandwidth of the MACC unit is increased, the output data of the MACC subunit is processed in time, and no blockage occurs, thereby improving the calculation utilization rate of the MACC.

In one embodiment, when the computational dimensions of all data blocks in the data to be processed are less than or equal to the dimensions of the multiplication-accumulator sub-unit in any multiplication-accumulator unit, each multiplication-accumulator sub-unit in the multiplication-accumulator unit is used as a multiplication-accumulator sub-cluster, and the multiplication-accumulation operation of one data block is completed respectively by multiple multiplication-accumulator sub-units in the multiplication-accumulator unit, and the multiplication-accumulation operation result of the data block is output to the next multiplication-accumulation sub-unit arranged in sequence, until the multiplication-accumulation operation of all data blocks is completed, and the multiplication-accumulation operation result of the data to be processed is output.

Specifically, when the computational dimensions of all data blocks in the data to be processed are less than or equal to the dimensions of the multiplication and accumulation sub-units in any multiplication and accumulation unit, and the first operation results of these data blocks need to be subjected to another multiplication and accumulation operation, that is, when processed in an assembly line processing manner, each multiplication and accumulation sub-unit in the multiplication and accumulation unit is used as a multiplication and accumulation sub-cluster, and the data to be processed is processed in an assembly line manner through the multiplication and accumulation sub-clusters, that is, the data output by the multiplication and accumulation sub-unit of the previous arrangement order is input into the multiplication and accumulation sub-unit of the next arrangement order. In this way, although the output bandwidth of the multiplication and accumulation unit is increased, the output bandwidth of the multiplication and accumulation sub-unit remains unchanged, and no congestion will occur, thereby improving the computational utilization rate of the multiplication and accumulation unit.

In one embodiment, as shown in FIG9 , the multiplier-accumulator unit includes a plurality of data input modules 12, a plurality of multiplication operation modules 111, a plurality of multi-stage addition operation modules 112 and a plurality of data output modules 113, wherein one multiplication operation module 111, one multi-stage addition operation module 112 and a plurality of data output modules 113 constitute a multiplication-accumulator subunit 11, each multiplication operation module includes a plurality of multipliers, and each multi-stage addition operation module includes a plurality of adders; the input end of each data input module 12 is respectively electrically connected to the data providing module, the output end of each data input module 12 is respectively electrically connected to the input end of a multiplication operation module 111, the output end of each multiplication operation module 111 is respectively electrically connected to the input end of a multi-stage addition operation module 112, and the output end of each multi-stage addition operation module 112 is respectively electrically connected to the plurality of data output modules 113.

Specifically, the multiplication and accumulation unit includes a plurality of data input modules, a plurality of multiplication operation modules, a plurality of multi-stage addition operation modules and a plurality of data output modules, and one multiplication operation module, one multi-stage addition operation module and a plurality of data output modules constitute a multiplication and accumulation sub-unit. For each multiplication and accumulation sub-unit, the data providing module inputs the data to be processed into the input end of the data input module, the output end of the data input module transmits the data to the input end of the multiplication operation module, the multiplication operation module performs multiplication operation on the input data, and then outputs the multiplication operation result to the input end of the multi-stage addition operation module, the multi-stage addition operation module performs accumulation operation on the input data to obtain a plurality of accumulation operation results, and outputs each accumulation operation result to a data output module.

As shown in Fig. 10, a plurality of multipliers form a multiplication operation module, a plurality of adders form a multi-stage addition operation module, a plurality of data output terminals C1, C2, C3, C4 and D, the input data is A1 ... A16, and the input data is B1 ... A16. The multiplication operation module, the multi-stage addition operation module and the plurality of data output terminals serve as a multiplication accumulator subunit.

The present application also provides a data processing method, which is applied to the above-mentioned multiplication and accumulation operation cluster. The data processing method, as shown in FIG11, includes:

112: Obtain the data to be processed, and determine the target multiplication and accumulation unit corresponding to the data to be processed according to the calculation dimension of the data to be processed and the dimension of the multiplication and accumulation subunit in each multiplication and accumulation unit;

114: Input the data to be processed into the multiplication and accumulation subunit in the target multiplication and accumulation unit that is in an idle state, and perform multiplication and accumulation operations on the data to be processed through the target multiplication and accumulation unit to obtain the multiplication and accumulation operation results of the data to be processed.

Specifically, according to the calculation dimension of the data to be processed and the dimension of the multiplication and accumulation unit sub-unit in each multiplication and accumulation unit, the target multiplication and accumulation unit corresponding to the data to be processed is determined, and the multiplication and accumulation operation is performed on the data to be processed by the multiplication and accumulation sub-unit in the target multiplication and accumulation unit that is in an idle state to obtain the multiplication and accumulation operation result of the data to be processed.

For example, when the calculation dimension of the data to be processed is 8, and the dimensions of the multiplication and accumulation sub-units in multiple multiplication and accumulation units are 6, 9, 12, and 16 respectively, a multiplication and accumulation unit in which the dimension of each multiplication and accumulation sub-unit in the multiplication and accumulation unit is 9 is selected as the target multiplication and accumulation unit, and multiplication and accumulation operations are performed on the data to be processed through the multiplication and accumulation sub-units in the target multiplication and accumulation unit that are in an idle state.

When there are multiple groups of data to be processed, the target multiplier-accumulator unit corresponding to each group of data to be processed is determined respectively, and the group of data to be processed is input into its corresponding target multiplier-accumulator unit, so that multiple target multiplier-accumulator units can simultaneously process their respective input data, thereby improving the data processing efficiency of the multiplier-accumulator operation cluster.

In one embodiment, obtaining data to be processed, and determining a target multiplication and accumulation unit corresponding to the data to be processed according to a calculation dimension of the data to be processed and a sub-dimension of a multiplication and accumulation unit in each multiplication and accumulation unit, comprises:

A multiplication accumulator unit in which the dimension of the multiplication accumulator sub-unit is greater than or equal to the calculation dimension of the data to be processed is taken as a candidate multiplication accumulator unit; a candidate multiplication accumulator unit that is in an idle working state and has the largest dimension of the multiplication accumulator sub-unit is taken as a target multiplication accumulator unit corresponding to the data to be processed.

Specifically, the computational dimension of the data to be processed and the dimension of the multiplication accumulator sub-unit in each multiplication accumulator unit are compared, and the multiplication accumulator unit whose dimension of the multiplication accumulator sub-unit is greater than or equal to the computational dimension of the data to be processed is taken as a candidate multiplication accumulator unit. There are multiple candidate accumulator units, and a candidate multiplication accumulator unit that is in an idle state and has the largest dimension of its multiplication accumulator sub-unit is selected from the multiple candidate multiplication accumulator units as the target multiplication accumulator unit, and the target multiplication accumulator unit is used to perform multiplication and addition operations on the data to be processed.

Compared with the prior art, the present application provides a data processing method, which determines the target multiplication accumulator unit corresponding to the data to be processed according to the calculation dimension of the data to be processed and the dimension included in the multiplication accumulator sub-unit in each multiplication accumulator unit, and inputs the data to be processed into the data input end of the multiplication accumulator sub-unit in the target multiplication accumulator unit. Since the multiple multiplication accumulator sub-units in the target multiplication accumulator unit can work independently, the multiple multiplication accumulator sub-units can process data simultaneously, thereby improving the computational utilization and total output bandwidth of the multiplication accumulator module.

It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of the embodiments. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.

The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and/or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims (10)

1. A multiply-accumulator operation cluster, characterized in that it comprises a plurality of multiply-accumulator units, each of which is adapted to perform a multiply-accumulator operation of at least one set of data to be processed, each set of data to be processed comprising two multi-dimensional data, each of which comprises a plurality of multiply-accumulator subunits, each of which is adapted to perform a multiply-accumulator operation and to output a multiply-accumulate operation result, wherein,

When the calculated dimension of two pieces of multi-dimensional data in the data to be processed is smaller than the total dimension of the multiply-accumulate units, completing multiply-accumulate operation of each group of data to be processed through at least one multiply-accumulate subunit in the multiply-accumulate units, and outputting multiply-accumulate operation results of the data to be processed; and/or the number of the groups of groups,

When the data to be processed comprises a plurality of data blocks which are sequentially arranged, respectively completing multiply-accumulate operation of one data block through a plurality of multiply-accumulate diversity groups in the multiply-accumulate operation cluster, and outputting the multiply-accumulate operation result of the data block to the next multiply-accumulate diversity group which is sequentially arranged until the multiply-accumulate operation of all the data blocks is completed, and outputting the multiply-accumulate operation result of the data to be processed.

2. The multiply-accumulator operation cluster of claim 1, wherein the dimensions of the multiply-accumulator subunits in a same multiply-accumulator unit are identical, the sum of the dimensions of all multiply-accumulator subunits in each of the multiply-accumulator units is identical to the total dimension of the multiply-accumulator units, and the dimensions of the multiply-accumulator subunits in a plurality of different multiply-accumulator units are identical or different from each other.

3. The multiply-accumulator operation cluster of claim 2, wherein when a calculated dimension of a set of data to be processed is less than or equal to a dimension of a multiply-accumulator subunit in at least one multiply-accumulator unit, a target multiply-accumulator unit to operate on the data to be processed is determined from the calculated dimension of the data to be processed and the dimension of the multiply-accumulator subunit in the multiply-accumulator unit, and the multiply-accumulate operation is performed on the data to be processed by the multiply-accumulator subunit in the target multiply-accumulator unit.

4. The multiply-accumulator operation cluster of claim 2, wherein a multiply-accumulate operation of a set of data to be processed is completed by a plurality of multiply-accumulator subunits in the multiply-accumulator unit when a calculated dimension of the data to be processed is greater than a dimension of a multiply-accumulator subunit in all of the multiply-accumulator units and less than a total dimension of each of the multiply-accumulator units.

5. The multiply-accumulator operation cluster of claim 4, wherein when the calculated dimension of the data to be processed is greater than the dimension of the multiply-accumulator sub-units in all the multiply-accumulator units and less than the total dimension of each of the multiply-accumulator units, determining a target multiply-accumulator for operating the data to be processed and a target number of multiply-accumulator sub-units in the target multiply-accumulator based on the calculated dimension of the data to be processed and the dimension of the multiply-accumulator sub-units in each of the multiply-accumulator units, and performing a multiply-accumulate operation on the data to be processed by the target number of multiply-accumulator sub-units in the target multiply-accumulator.

6. The multiply-accumulator operational cluster of claim 1, wherein at least one multiply-accumulator subunit in the multiply-accumulator operational cluster is a multiply-accumulator sub-cluster.

7. The multiply-accumulator operation cluster according to claim 6, wherein when the calculation dimension of all the data blocks in the data to be processed is smaller than or equal to the dimension of the multiply-accumulator sub-units in any multiply-accumulator unit, each multiply-accumulator sub-unit in the multiply-accumulator unit is used as a multiply-accumulator sub-cluster, the multiply-accumulate operation of one data block is respectively completed through a plurality of multiply-accumulator sub-units in the multiply-accumulator unit, and the multiply-accumulate operation result of the data block is output to the next multiply-accumulator sub-unit in sequence until the multiply-accumulate operation of all the data blocks is completed, and the multiply-accumulate operation result of the data to be processed is output.

8. The multiply-accumulator operational cluster of any one of claims 1-7, wherein said multiply-accumulator unit comprises a plurality of data input modules, a plurality of multiply-operation modules, a plurality of multi-stage summation modules, and a plurality of data output modules, wherein one multiply-operation module, one multi-stage summation module, and a plurality of data output modules comprise a multiply-accumulator subunit, each of said multiply-operation modules comprising a plurality of multipliers, each of said multi-stage summation modules comprising a plurality of adders;

The input end of each data input module is electrically connected with the data providing module respectively, the output end of each data input module is electrically connected with the input end of one multiplication operation module respectively, the output end of each multiplication operation module is electrically connected with the input end of one multi-stage addition operation module, and the output end of each multi-stage addition operation module is electrically connected with a plurality of data output modules respectively.

9. A data processing method applied to the multiply-accumulator operation cluster according to any one of the preceding claims 1-8, characterized in that the data processing method comprises:

acquiring data to be processed, and determining a target multiply-accumulator unit corresponding to the data to be processed according to the calculated dimension of the data to be processed and the dimension of a multiply-accumulator subunit in each multiply-accumulator unit;

And inputting the data to be processed into a multiply-accumulate subunit in an idle state in the target multiply-accumulate device, and performing multiply-accumulate operation on the data to be processed through the target multiply-accumulate device to obtain a multiply-accumulate operation result of the data to be processed.

10. The method of claim 9, wherein the obtaining the data to be processed, determining the target multiply-accumulator unit corresponding to the data to be processed according to the calculated dimension of the data to be processed and the sub-dimension of the multiply-accumulator sub-unit in each multiply-accumulator unit, comprises:

taking a multiply accumulator unit, of which the dimension of a multiply accumulator subunit is greater than or equal to the calculated dimension of the data to be processed, as a candidate multiply accumulator unit;

And taking the candidate multiply accumulator unit which is in the idle working state and has the largest dimension of the multiply accumulator subunit as a target multiply accumulator unit corresponding to the data to be processed.