WO2019141160A1 - 一种数据处理的方法及装置 - Google Patents

一种数据处理的方法及装置 Download PDF

Info

Publication number
WO2019141160A1
WO2019141160A1 PCT/CN2019/071737 CN2019071737W WO2019141160A1 WO 2019141160 A1 WO2019141160 A1 WO 2019141160A1 CN 2019071737 W CN2019071737 W CN 2019071737W WO 2019141160 A1 WO2019141160 A1 WO 2019141160A1
Authority
WO
WIPO (PCT)
Prior art keywords
candidate
candidate region
group
suppression
instruction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/071737
Other languages
English (en)
French (fr)
Inventor
崔鲁平
屠嘉晋
刘虎
袁宏辉
廖恒
林灏勋
李冰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to EP19741008.7A priority Critical patent/EP3736765A4/en
Publication of WO2019141160A1 publication Critical patent/WO2019141160A1/zh
Priority to US16/932,768 priority patent/US11823303B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Definitions

  • the present application relates to the field of data processing technologies, and in particular, to a data processing method and apparatus.
  • Non-maximum suppression is an indispensable component for object detection and is responsible for obtaining unique detection of the same object.
  • the basic principle of the NMS is to compare the ratio of the overlapping areas of the two candidate regions. If the ratio is greater than a certain threshold, the candidate regions with low scores are suppressed.
  • the candidate area is a rectangular box represented by the upper left corner coordinates (y1, x1) and the lower right corner coordinates (y2, x2).
  • the score of the candidate region reflects the size of the possibility that the candidate region contains a complete object. The higher the score, the higher the probability that the candidate region contains a complete object.
  • the input of the NMS is N candidate regions that have been ranked in order from highest to lowest, and output M candidate regions with the highest score and not suppressed.
  • the central processing unit In the calculation of NMS, the central processing unit (CPU) is usually used for NMS calculation. However, due to the limited parallel computing power of the CPU, multiple cycles are required to obtain a suppression vector calculation, which causes a long calculation time.
  • the use of a non-uniform architecture results in additional communication overhead between the CPU and the GPU.
  • the calculation of the NMS is completed. Although the calculation time is much faster than the CPU calculation alone, it still takes a long time and adds extra power.
  • the present application provides a data processing method and apparatus, which can shorten the calculation time of the NMS.
  • a method of data processing comprising:
  • each candidate region sequence includes a plurality of candidate regions, R is a positive integer, and a vector region proposal area calculation (VRPAC) instruction is used to calculate an area of each candidate region in each candidate region sequence.
  • the VRPAC instruction is to calculate the area of the K candidate regions in a single time, K is associated with the data type, and then call the vector overlap over union for the jth candidate region sequence of the R group candidate region sequence.
  • VIOU VIOU
  • VAADD vector area add
  • the GPU or the CPU calculates the NMS, the call instruction is reduced, and the instruction execution step is reduced. Reduced the computation time of the NMS.
  • the calling VIOU instruction and the VAADD instruction determining the j suppression matrices of the jth group candidate region sequence include: calling the VIOU instruction to calculate the first group candidate region sequence to the jth group candidate region The overlapping area between the candidate regions in the two adjacent candidate region sequences in the sequence and the overlapping area between the j-th candidate region sequence and the candidate region in the j-th candidate region sequence, the VAADD is called The instruction calculates an area between the candidate group of the first group of candidate region sequences to the candidate group of the j-th group candidate region sequence, and a candidate region between the j-th candidate region sequence and the j-th group candidate region sequence The area sum between the two sets, the area of the first set of candidate area sequences to the candidate area of the adjacent two sets of candidate area sequences in the jth group candidate area sequence, and the jth group candidate area The area and factorization between the sequence and the candidate region in the j-th candidate region sequence, and the candidate regions in the adjacent two candidate region sequences in the
  • the call instruction By calling the VIOU instruction and the VAADD instruction to obtain the suppression matrix of each candidate region sequence, compared with the prior art calculation of the NMS by the GPU or the CPU, the call instruction is reduced, the steps of the instruction execution are reduced, and the data acquisition is reduced. The number of times, thus shortening the calculation time of the NMS.
  • the determining, by the j suppression matrices, a suppression vector of the j-th candidate region sequence includes: calling a non-diagonal matrix condition of a candidate region-based network or (region proposal network condition) Or for a non-diagonal suppression matrix, RPN_COR) instruction for calculating the first j-1 suppression matrices in the j suppression matrices, obtaining an intermediate result of the suppression vector; calling a diagonal matrix condition of the network based on the candidate region or (region The proposal network diagonal condition or for non-diagonal suppression matrix, RPN_COR_DIAG) is calculated by calculating a j-th suppression matrix in the j suppression matrices, and intermediate the result of the calculation with the suppression vector to obtain the j-th group The suppression vector of the candidate region sequence.
  • the suppression vector is obtained, which can reduce the number of calculations and further shorten the calculation time of the NMS.
  • the RPN_COR_DIAG instruction is a calculation of a diagonal matrix block of the suppression matrix to obtain a suppression vector; the RPN_COR instruction is a non-diagonal matrix block calculation of the suppression matrix, and an intermediate result of the suppression vector is obtained.
  • a second aspect provides an apparatus for data processing, including: a communication unit and a processing unit;
  • the communication unit is configured to acquire an R group candidate region sequence, where each group of candidate region sequences includes multiple candidate regions, and R is a positive integer;
  • the processing unit is configured to invoke a VRPAC instruction to calculate an area of each candidate region in each candidate region sequence; the VRPAC instruction is to calculate an area of K candidate regions in a single time, K is associated with a data type; and for the R group a sequence of the j-th candidate region of the candidate region sequence, calling the VIOU instruction, the VAADD instruction to determine j suppression matrices of the j-th candidate region sequence, and determining the sequence of the j-th candidate region sequence according to the j suppression matrices a suppression vector; wherein, the VIOU instruction is a single calculation of an overlap area between two pairs of candidate regions in two sets of candidate region sequences; the VAADD instruction is a single calculation between candidate regions of two sets of candidate region sequences The area sum; j is an integer greater than or equal to 1 and less than or equal to R; and a candidate region that is not suppressed is determined according to the suppression vector of each of the sets of candidate region sequences.
  • the processing unit determines the j suppression matrices of the j-th candidate region sequence by calling the VIOU instruction and the VAADD instruction, the processing unit is specifically configured to:
  • the area sum between two pairs of candidate regions in the adjacent two sets of candidate region sequences in the group j candidate region sequence that is factorized, and the candidate region of the jth group candidate region sequence and the jth group candidate region sequence The area sum between the two groups, and the overlap area between the candidate groups of the first group of candidate region sequences to the j-th group candidate region sequence and the j-th candidate region sequence Comparing the overlapping areas between the candidate regions in the j-th candidate region sequence, the j suppression matrices of the j-th candidate region sequence are obtained.
  • the processing unit determines the suppression vector of the jth group candidate region sequence according to the j suppression matrices, the processing unit is specifically configured to:
  • the RPN_COR_DIAG instruction is called to calculate the j-th suppression matrix in the j suppression matrices, and the calculation result is intermediate with the suppression vector or the suppression vector of the j-th group candidate region sequence is obtained.
  • the RPN_COR_DIAG instruction is a calculation of a diagonal matrix block of the suppression matrix to obtain a suppression vector; the RPN_COR instruction is a non-diagonal matrix block calculation of the suppression matrix, and an intermediate result of the suppression vector is obtained.
  • a third aspect provides an apparatus for data reading, comprising: a processor and a memory; the memory stores computer execution instructions, the processor is connected to the memory through the bus, and when the device is running, the processor executes the The computer stored in the memory executes instructions to cause the apparatus to perform the method of any of the preceding aspects.
  • a chip comprising: a processing unit and a communication unit, the processing unit being, for example, a processor, and the communication unit can be, for example, an input/output interface, a pin or a circuit, or the like.
  • the processing unit may execute a computer-executed instruction stored by the storage unit to cause the method of data processing of any of the above-described first aspects to be performed.
  • the storage unit is a storage unit in the chip, such as a register, a cache, etc., and the storage unit may also be a storage unit located outside the chip in the terminal device, such as a read-only memory, other types that can store static information and instructions. Static storage devices, random access memories, and the like.
  • a fifth aspect a computer readable storage medium comprising computer readable instructions which, when read and executed by a computer, cause a computer to perform the method of any of the first aspects.
  • a computer program product comprising computer readable instructions which, when read and executed by a computer, cause the computer to perform the method of any of the first aspects.
  • FIG. 1 is a schematic diagram of a system architecture provided by the present application.
  • FIG. 2 is a schematic flowchart of a data processing method provided by the present application.
  • FIG. 3 is a schematic diagram of execution of a VRPAC instruction provided by the present application.
  • FIG. 4 is a schematic diagram of execution of a VIOU instruction provided by the present application.
  • FIG. 5 is a schematic diagram of execution of a VIOU instruction provided by the present application.
  • FIG. 6 is a schematic diagram of execution of a VAADD instruction provided by the present application.
  • FIG. 7 is a schematic diagram of execution of an RPN_COR_DIAG instruction provided by the present application.
  • FIG. 8 is a schematic diagram of an RPN_COR instruction execution according to the present application.
  • FIG. 9 is a schematic diagram of an RPN_COR instruction and an RPN_COR_DIAG instruction performed in conjunction with the present application;
  • FIG. 10 is a schematic diagram of a hardware structure provided by the present application.
  • FIG. 11 is a schematic structural diagram of an apparatus for data processing according to the present application.
  • FIG. 12 is a schematic structural diagram of an apparatus for data processing according to the present application.
  • the present application provides a method of data processing, which can be applied to the system architecture shown in FIG. 1.
  • FIG. 1 a possible system architecture provided by an embodiment of the present application is shown.
  • the system structure includes a fractal matrix multiply processor 101, a second buffer (L2buffer) 102, and a main memory (mian memory) 103.
  • the fractal matrix multiplication processor 101 includes an Instruction Fetch Unit (IFU) 1001, a Dispatch Unit 1002, and a Level 1 source operand buffer (L1src Buffer (Static Random Access Memory). Memory, SRAM))) (built with SRAM) 1003, zero-level source operand buffer (L0src Buffer (DFF)) (built with registers) 1004, accumulator (Accumulator) 1005, zero-order destination operand buffer ( L0Dest Buffer (DFF) 1006, L1Dest Buffer (SRAM) 1007, Direct Memory Access Unit (DMA Unit) 1008, Vector Unit 1009, scalar unit ( Scalar Unit 1010, Unified Buffer (UB) 1011 and Bus Interface Unit (BIU) 1012.
  • IFU Instruction Fetch Unit
  • Dispatch Unit 1002 a Level 1 source operand buffer
  • L1src Buffer Static Random Access Memory
  • Memory, SRAM level 1 source operand buffer
  • DFF zero-level source operand buffer
  • DMA Unit
  • L1src Buffer (SRAM) 1003, L0src Buffer (DFF) 1004, Accumulator 1005, L0Dest Buffer (DFF) 1006, L1Dest Buffer (SRAM) 1007 can constitute a fractal matrix multiplication unit.
  • the IFU 1001 can acquire instructions from the main memory 103 through the BIU 1012, and decode and control the execution flow.
  • the instruction distribution unit 1002 can submit corresponding type instructions to the four pipeline units, and the instruction distribution unit 1002 can organically control the execution order between the four pipelines. .
  • the pipeline unit has two types of asynchronous execution and synchronous execution. All types are guaranteed-release.
  • the difference is that the asynchronous execution unit execution instruction ends asynchronously, and the synchronous execution unit ends synchronously.
  • the scalar unit 1010 is synchronously executed, and the fractal matrix multiplication processing unit, the DMA unit 1008, and the vector unit 1009 are asynchronously executed. All execution units can read and write interactive data through the unified cache 1011.
  • the fractal matrix multiplication unit has two levels of dedicated caches of L1 and L0, and the L1 cache and unified cache 1011 exchange data with the external storage space through the DMA unit 1008.
  • the external storage space consists of multiple levels of storage units. For example, the capacity of the unified cache 1011 is 256 KB, which is implemented in 64 banks, and each bank is 32 B wide.
  • the candidate regions in the order can be stored in the unified cache 1011 in the manner shown in Table 1.
  • Bit index 30 28 26 twenty four twenty two 20 18 16 14 12 10 8 6 4 2 0
  • FIG. 2 exemplarily shows a flow of a data processing method provided by the present application, and the method of the data amount can be applied to the field of image processing, which can be executed by the above-mentioned fractal matrix multiplication processor. .
  • the process specifically includes:
  • Step 201 Acquire a sequence of R group candidate regions.
  • the R group candidate area is data to be input, and is a candidate area that has been sorted.
  • Each set of candidate region sequences may include a plurality of candidate regions, the number of each candidate region being related to the number of bits of the input data type, wherein the data type may include an 8-bit unsigned integer (8-bit unsigned integer, u8), 8 bits. 8-bit signed integer (s8), 16-bit floating point (fp16), and 32-bit floating point (fp32).
  • the data type is fp16 and N candidate regions are input
  • R is the number of groups of candidate regions.
  • This R group candidate area can be stored in the UB according to the storage address as shown in Table 1.
  • the types of data in this application are not limited to the above types, but are merely exemplary.
  • Step 202 The VRPAC instruction is invoked to calculate an area of each candidate region in each group of candidate region sequences.
  • non-maximum suppression it is first necessary to call the VRPAC instruction to calculate the area of each candidate region in each group of candidate region sequences.
  • the VRPAC instruction calculates the area of K candidate regions in a single pass, where K is associated with the number of bits of the data type. That is to say, the value of K can be the same as the number of bits of the data type.
  • the VRPAC instruction can calculate the area of 16 candidate regions in a single time. K can take values of 8, 16, or 32, and so on.
  • the VRPAC instruction can support the repeat mode, that is, the calculation of the area of the plurality of candidate regions is automatically completed by setting the number of repetitions.
  • the VRPAC instruction can reuse existing M adder resources and (M/4) multiplier resources.
  • the VRPAC command can be divided into 1 beat, 2 beat, 3 beat, etc. according to the target frequency to complete the calculation task.
  • the VRPAC instruction can determine whether the VRPAC instruction is a full pipeline or a non-complete pipeline based on the number of resources of the vector unit addition and multiplication.
  • K has a value of 16, a data type of fp16, 64 fp16 adders of a multiplexed vector unit, and 16 fp16 multipliers, which can be completely pipeline executed.
  • P0 to P15 are candidate regions, and are calculated by an adder and a multiplier, and finally output the area of 16 candidate regions.
  • a VRPAC instruction can be used to obtain K* at one time.
  • R results repeated R times, but only one instruction is called.
  • the first instruction is to calculate the abscissa x2-x1 of the candidate region to obtain an intermediate result res_tmp0; the second instruction is calculated.
  • the sum of res_tmp0 and 1 can get the intermediate result res_tmp1; the third instruction calculates the ordinate y2-y1 to get the intermediate result res_tmp2; the fourth instruction calculates the sum of res_tmp2 and 1, and can get the intermediate result res_tmp3; the fifth instruction calculates res_tmp1 The product of res_tmp3, thereby obtaining the area value of a candidate region.
  • res_tmp1 and res_tmp3 do not have any data dependencies, but must be obtained sequentially, thereby reducing performance; in addition, res_tmp1 depends on res_tmp0 (same reason, res_tmp3 depends on res_tmp2), res_tmp1 must wait until res_tmp0 Write back to the register to start execution.
  • Modern processors usually bypass the performance bubble caused by data correlation, but usually the floating point addition multiplication is done in multiple shots, which will result in the bubble not being completely eliminated. problem. From the power consumption storage, 5 instructions need to occupy 5 instructions storage space, and each instruction must complete the operation of fetching, decoding, executing, and writing back to the register, and the energy efficiency ratio is low.
  • the GPU When calculating the area of a candidate region through the GPU, the GPU usually performs execution by assigning calculations of multiple candidate regions to different threads to improve parallelism.
  • the calculation process of a single thread is similar to that of the CPU, and each thread also faces Performance, power, and storage issues on the CPU.
  • the GPU can hide the computed bubbles by switching between different threads, but at the expense of additional register resources.
  • the VRPAC instruction provided by the present application uses only one instruction, and is divided into three basic steps in the actual implementation.
  • the first step is to obtain res_tmp0 and res_tmp2 at the same time
  • the second step is to obtain res_tmp1 and res_tmp3 at the same time.
  • the third step is to get the final result.
  • the basic steps are two steps less than the above two methods, and the steps and steps are completely executed by the hardware pipeline, and there is no potential performance bubble problem, which shortens the calculation time.
  • From the perspective of power consumption storage there is only one instruction, and one instruction supporting the repeated mode can complete the calculation of candidate regions of R*K (R is the number of repetitions, and K is the degree of parallelism of one repetition).
  • R*K is the number of repetitions
  • K is the degree of parallelism of one repetition
  • Step 203 Determine, according to the j-th group candidate region sequence of the R group candidate region sequence, a VIOU instruction, a VAADD instruction, and determine j suppression matrices of the j-th group candidate region sequence, and determine, according to the j suppression matrix The suppression vector of the jth group candidate region sequence.
  • the VIOU instruction may calculate the overlapping area between two pairs of candidate regions in the two candidate region sequences in a single calculation. For example, if there are K candidate regions in each group, the K*K overlapping regions are calculated. .
  • the VIOU command can also support the repeat mode. By setting the number of repetitions, the overlap area of multiple candidate regions can be automatically calculated, and the existing M comparators, M adders, and (M/4) multiplications can be reused. According to the target frequency, it can be divided into 1 beat, 2 beats, 3 beats, etc. to complete the calculation. It is possible to determine whether the VIOU instruction is a full pipeline execution or a non-complete pipeline execution based on the number of resources of the vector unit's comparator, adder, and multiplier.
  • the calculation of a single overlap area can be as shown in FIG. 4, and the two candidate regions can be completed by one comparison, two additions, two additions and comparisons, and one multiplication.
  • K 16
  • M 128, it can be implemented using hardware as shown in FIG.
  • the basic steps performed by the VIOU instruction are 7 steps less than the existing way of using the CPU or GPU, and the steps and steps are completely executed by the hardware pipeline, and there is no potential performance bubble problem. Since the calculation steps are reduced, the calculation time is also shortened. From the perspective of power consumption storage, there is only one instruction, and one instruction supporting repeated mode can complete the calculation of the overlap area of R*K (R is the number of repetitions, K is the degree of parallelism of one repetition, and several calculations at a time). .
  • the VIOU instruction only involves one value at a time, one decoding, and only the final result is written back to the UB. This minimizes additional power consumption. The energy efficiency ratio will be higher than the GPU and CPU.
  • the above VAADD instruction may calculate the area sum between two pairs of candidate regions in the two candidate region sequences in a single calculation. For example, if there are K candidate regions per group, then K*K area sums are calculated.
  • the VAADD instruction supports the repeat mode, and the calculation of the area sum of the plurality of candidate regions is automatically completed by setting the number of repetitions.
  • the existing M adders can be multiplexed, and can be divided into 1 beat, 2 beats, 3 beats, etc. according to the target frequency, and the number of resources of the vector unit adder is determined whether the command is a full pipeline execution or a non-complete pipeline. carried out.
  • the VIOU instruction when determining the j suppression matrices of the j-th candidate region sequence, the VIOU instruction may be called to calculate the first two candidate candidates in the first group candidate region sequence to the j-th group candidate region sequence.
  • the overlapping area between the candidate regions in the region sequence and the overlapping area between the j-th candidate region sequence and the candidate region in the j-th candidate region sequence, and the VAADD instruction is called to calculate the first group candidate region sequence to The area between the candidate regions in the adjacent two sets of candidate region sequences in the j-th candidate region sequence and the area between the j-th candidate region sequence and the candidate region in the j-th candidate region sequence, and then The first group candidate region sequence to the area between the candidate regions of the two sets of candidate region sequences in the j-th group candidate region sequence and the candidate region of the j-th candidate region sequence and the j-th candidate region sequence The area and factorization between the two.
  • the area between the two candidate regions in the adjacent two candidate region sequences in the factored candidate region sequence and the candidate region of the jth candidate region sequence and the candidate region in the jth candidate region sequence The area sum between the first group candidate region sequence and the candidate group region of the adjacent two groups of candidate region sequences in the jth group candidate region sequence, and the jth group candidate region sequence and the jth group candidate.
  • the overlapping areas between the candidate regions in the region sequence are compared to obtain j suppression matrices of the j-th candidate region sequence.
  • j may be an integer greater than or equal to 1 and less than or equal to R.
  • the VIOU command is first called, and the number of repetitions is set to 3.
  • the overlapping area of the first group candidate region and the second group candidate region, and the intersection of the second group candidate region and the third group candidate region are calculated.
  • Calling the VAADD command, setting the number of repetitions to 3, calculating the area sum of the first group candidate region and the second group candidate region, the area sum of the second group candidate region and the third group candidate region, the third group candidate region and the third group The area and area of the candidate area.
  • the calculated area and factorization are then used, that is, the resulting area sum is multiplied by a parameter factor a, which can be set empirically.
  • the factorized area can be compared with the calculated overlap area, so that the three suppression matrices of the third group candidate region sequence can be obtained.
  • the resulting suppression matrix is an N*N matrix (N is the total number of candidate regions), and the diagonal of the matrix and the element values above it are both zero.
  • R*K times of data need to be taken in one calculation, where R is the number of candidate region sequence groups, and K is each group. The number of candidate regions included in the candidate region sequence.
  • the secondary data that is, R 2 *2*K times.
  • the secondary data that is, (R*K) 2 times.
  • the technical solution of the present application can save a large number of data acquisition times, thereby saving computation time.
  • the CPU or GPU calculates more steps than the technical solution of the present application, further increasing the difference in the number of times the data is fetched. Therefore, compared with the existing NMS computing solution, the technical solution provided by the present application can save a lot in the number of times of taking data, correspondingly reducing the number of accesses to the main memory, thereby shortening the calculation time and reducing the extra time. Power consumption.
  • the suppression vectors of the j-th candidate region sequence may be determined according to the j suppression matrices.
  • the pre-j-1 suppression matrix in the j suppression matrices may be calculated by using a region proposal network condition or for non-diagonal suppression matrix (RPN_COR) instruction, Obtaining the vector result of the suppression vector, and then calling the region proposal network diagonal condition or for non-diagonal suppression matrix (RPN_COR_DIAG) instruction to calculate the jth suppression matrix in the j suppression matrices And the result of the calculation is intermediate with the suppression vector or the suppression vector of the jth candidate region sequence is obtained.
  • RPN_COR region proposal network condition or for non-diagonal suppression matrix
  • the RPN_COR_DIAG instruction can be calculated for the diagonal matrix block of the suppression matrix to obtain a suppression vector. That is, the calculation from the diagonal matrix block (KxK) of the suppression matrix to the suppression vector (Kx1) can be divided into 1 beat, 2 beat or multi shot according to the target frequency, and can be completely pipeline executed. There are two inputs to the RPN_COR_DIAG instruction, one is a diagonal matrix block, and the other is a special register that stores intermediate results. In the suppression vector calculation process, the result of low score depends on the result of high score, that is, the calculation of the i-th component depends on the calculation result of the 0-i-1 component.
  • the candidate region 2 is not suppressed by the candidate regions 0, 1, so the candidate region 2 is not eventually suppressed.
  • the candidate region 2 suppresses the candidate region 5, and therefore, the candidate region 5 is finally suppressed.
  • the candidate region 12 is suppressed by the candidate region 5, but the candidate region 5 has been suppressed, so the suppression of the candidate region 12 by the candidate region 5 is canceled, and the final candidate region 12 is not suppressed.
  • the obtained intermediate result is bitwise ORed with the special register RPN_COR_IR (resulting result before storage), and the final suppression vector of the 16 candidate regions is obtained.
  • the RPN_COR instruction can be calculated for the non-diagonal matrix block of the suppression matrix, resulting in an inhibitor vector intermediate result. That is, the calculation of the intermediate result of the matrix non-diagonal matrix block (KxK) to the suppression vector is suppressed, and the special register RPN_COR_IR is updated.
  • the repeat mode can be supported, and the calculation of multiple non-diagonal matrix blocks can be realized by setting the number of repetitions. According to the frequency target, it can be completed by 1 beat, 2 beats, 3 beats, etc., and can be completely pipeline executed.
  • the RPN_COR instruction There are three inputs to the RPN_COR instruction, one is the non-diagonal matrix box of the suppression matrix (16x16), one is the value of the 16-suppression vector with a higher score, and the other is the special register RPN_COR_IR that stores the intermediate result.
  • the input suppression vector has 1, the suppression of the candidate region with a candidate region with a low subsequent score is canceled.
  • the suppression vector result obtained this time is ORed on the RPN_COR_IR and written to the special register RPN_COR_IR to continuously accumulate the calculation result.
  • the final suppression vector of the 16 candidate regions is obtained by the RPN_COR_DIAG instruction.
  • the candidate region 0 has suppression of the candidate region 16, and the candidate region 0 is not suppressed, so the suppression of the candidate region 16 by the candidate region 0 is effective.
  • the candidate region 15 also suppresses the candidate region 16, but the candidate region 15 is suppressed. Therefore, the suppression of the candidate region 16 by the candidate region 15 is canceled, and the result of the current candidate calculation of the final candidate region 16 is suppressed.
  • the candidate region 5 and the candidate region 7 both suppress the candidate region 20, but both the candidate region 5 and the candidate region 7 are suppressed, so the suppression of the candidate region 20 by the candidate region 5 and the candidate region 7 is canceled, and finally the candidate region 20 is calculated in the current round. Not suppressed.
  • the 16 intermediate results obtained in this round are either bitwise or the value of the special register and the special register is updated again.
  • RPN_COR_IR is initialized to 0. Then, the RPN_COR instruction is used to calculate the matrix frame of the first two suppression matrices, and the suppression vector intermediate result is obtained, and the suppression vector intermediate result is updated with RPN_COR_IR, and then the RPN_COR_DIAG pair is called. The matrix box of the third suppression matrix is calculated, and the result of the calculation is compared with the intermediate result of the suppression vector in RPN_COR_IR by bitwise OR to obtain the final suppression vector.
  • the RPN_COR_IR register When determining the suppression vector of the j-th candidate region sequence, the RPN_COR_IR register is initialized to 0, the RPN_COR instruction is called, the repetition number is set to j-1 times, the source operand is the matrix box of the non-diagonal suppression matrix, and the result is updated RPN_COR_IR The register is then called RPN_COR_DIAG to obtain the suppression vector of the jth group candidate region.
  • Step 204 Determine a candidate region that is not suppressed according to the suppression vector of each group of candidate region sequences.
  • the first M non-zero candidate regions may be taken as the final result, that is, the first M unsuppressed candidate regions.
  • the process of data processing will be described below in a specific implementation scenario.
  • the above VRPAC instruction, VIOU instruction and VAADD instruction can be multiplexed with a hardware structure, as shown in FIG. 10, one calculation can generate a VRPAC result, one VIOU result and four VAADD results.
  • the calculation process of the NMS is described in combination with the VRPAC instruction, the VIOU instruction, the VAADD instruction, the RPN_COR instruction, and the RPN_COR_DIAG instruction.
  • the data type is fp16, wherein the VRPAC instruction generates 16 calculation results at a time, and the VIOU instruction calculates the overlapping area between the two sets of candidate regions (16 in each group), and generates 256 calculation results, and the VAADD instruction calculates The area sum between two groups of candidate regions (16 in each group) yields 256 calculation results.
  • the RPN_COR instruction obtains an intermediate result of 16 suppression vectors at a time, and the RPN_COR_DIAG instruction obtains 16 suppression vectors at a time.
  • Step 1 The VRPAC instruction is called, and the number of repetitions is set to R. After the instruction is executed, the area values of the N candidate regions can be obtained in the UB.
  • Step 2 Calculate the result of the 0th group candidate region suppression vector. This step can be divided into the following substeps:
  • Step 2.1 Call the VIOU instruction to calculate the overlap area of the 0th group candidate area (16 candidate areas) and the 0th group candidate area.
  • Step 2.2 Call the VAADD instruction to calculate the area sum of the 0th group candidate area and the 0th group candidate area.
  • Step 2.3 Call the normal vector instruction (scalar multiplication vector) and multiply the result obtained in step 2.2 by a parameter factor a.
  • Step 2.4 Invoke the normal vector comparison instruction, and compare the overlap area obtained in step 2.1 with the obtained factorized area sum in step 2.3, thereby obtaining one suppression matrix of the 0th group candidate region.
  • Step 2.5 Initialize the RPN_COR_IR special register to 0, call the RPN_COR_DIAG instruction, and the source operand is the result of step 2.4, and obtain the suppression vector of the 0th candidate region.
  • Step 3 Calculate the result of the first set of candidate region suppression vectors. This step can be divided into the following substeps:
  • Step 3.1 The VIOU command is called, and the number of repetitions is set to 2 times, and the overlapping area values of the 0th group candidate area and the 1st group candidate area, and the 1st group candidate area and the 1st group candidate area are obtained.
  • Step 3.2 The VAADD instruction is called, and the number of repetitions is set to 2 times, and the area sum of the 0th group candidate area and the 1st group candidate area, and the 1st group candidate area and the 1st group candidate area are obtained.
  • Step 3.3 Call the normal vector instruction (scalar multiplication vector) and multiply the result obtained in step 3.2 by a parameter factor a.
  • Step 3.4 Invoke the normal vector comparison instruction, compare the result of step 3.1 with the result of step 3.3 to obtain the two suppression matrices of the candidate group of the first group.
  • Step 3.5 Initialize the RPN_COR_IR special register to 0, call the RPN_COR instruction, set the number of repetitions to 1 time, the source operand is the matrix box of the non-diagonal suppression matrix, and update the RPN_COR_IR register, then call RPN_COR_DIAG to get the first group.
  • the suppression vector of the candidate region is the RPN_COR_IR special register to 0
  • step N+1 calculating the result of the (R-1)th group, that is, the last set of suppression vectors, the step can be divided into the following substeps:
  • Step N+1.1 The VIOU command is called, and the number of repetitions is set to R times, and the 0th group candidate area and the R-1th group candidate area, the 1st group candidate area and the 2nd group candidate area are calculated. . . The overlapping area of the candidate region of the R-1 group and the candidate region of the R-1 group.
  • Step N+1.2 The VAADD instruction is called, and the number of repetitions is set to R times, and the 0th group candidate area and the R-1th group candidate area, the 1st group candidate area and the 2nd group candidate area are calculated. . . , the area sum of the candidate region of the R-1 group and the candidate region of the R-1 group.
  • Step N+1.3 Call the normal vector instruction and multiply the result obtained in step N+1.2 by a parameter factor a.
  • Step N+1.4 Invoke the normal vector comparison instruction, compare the result of step N+1.1 with the result of step N+1.3, and obtain R suppression matrices of the (R-1)th group.
  • Step N+1.5 Initialize the RPN_COR_IR register to 0, call the RPN_COR instruction, set the number of repetitions to (R-1) times, and the source operand is the matrix box of the non-diagonal suppression matrix. The result is to update the RPN_COR_IR register, and then call RPN_COR_DIAG. Thereby, the suppression vector of the (R-1)th group candidate region is obtained.
  • Step N+2 According to the suppression vector, the first M non-zero candidate regions are taken as the final result.
  • the time required for NMS calculation can be significantly shortened, the computational energy efficiency is greatly improved, and the competitiveness is enhanced in an application field involving image detection and image recognition.
  • FIG. 11 exemplarily shows that the present application provides a structure of an apparatus for data processing, which can perform a flow of data processing.
  • the apparatus may be a chip that performs the method of data processing in any of the above embodiments.
  • the apparatus 1100 includes at least one processor 1101, a communication interface 1102, and optionally a memory 1103.
  • the processor 1101, the communication interface 1102, and the memory 1103 are connected to each other.
  • the processor 1101 can be a general purpose central processing unit, a microprocessor, an image processor, an application specific integrated circuit, or one or more integrated circuits for controlling program execution of embodiments of the present application.
  • the communication interface 1102 is configured to input or output data.
  • the memory 1103 can be a read only memory or other type of static storage device random access memory that can store static information and instructions or other types of dynamic storage devices that can store information and instructions, or can be an electrically erasable programmable read only memory. , read-only disc or other disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), disk storage media or other magnetic storage devices, or capable of carrying or storing instructions or data
  • the desired program code in the form of a structure and any other medium that can be accessed by a computer, but is not limited thereto.
  • the memory 1103 may be independent and connected to the processor 1101.
  • the memory 1103 can also be integrated with the processor.
  • the memory 1103 is configured to store application code that executes an embodiment of the present application and is controlled to be executed by the processor 1101.
  • the processor 1101 is configured to execute application code stored in the memory 1103.
  • the processor 1101 may include one or more CPUs, such as CPU0 and CPU1 in FIG.
  • apparatus 1100 can include multiple processors, such as processor 1101 and processor 1108 in FIG. Each of these processors may be a single-CPU processor or a multi-core processor, where the processor may refer to one or more devices, circuits, and/or A processing core for processing data, such as computer program instructions.
  • processors such as processor 1101 and processor 1108 in FIG.
  • processors may be a single-CPU processor or a multi-core processor, where the processor may refer to one or more devices, circuits, and/or A processing core for processing data, such as computer program instructions.
  • the application may divide the function module into the terminal device according to the above method example.
  • each function module may be divided according to each function, or two or more functions may be integrated into one processing module.
  • the above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the present application is schematic, and is only a logical function division, and may be further divided in actual implementation.
  • FIG. 12 shows a schematic diagram of a device, which may be the terminal device involved in the above embodiment, and the device includes a processing unit 1201 and a communication unit 1202.
  • the communication unit 1202 is configured to acquire an R group candidate region sequence, where each group of candidate region sequences includes multiple candidate regions, and R is a positive integer;
  • the processing unit 1201 is configured to invoke a VRPAC instruction to calculate an area of each candidate region in each candidate region sequence; the VRPAC instruction is to calculate an area of K candidate regions in a single time, K is associated with a data type; and for the R
  • the j-th candidate region sequence of the group candidate region sequence is determined by calling the VIOU instruction and the VAADD instruction to determine j suppression matrices of the j-th group candidate region sequence, and determining the j-th group candidate region sequence according to the j suppression matrices
  • the suppression vector wherein the VIOU instruction is a single calculation of the overlap area between the two candidate regions in the two candidate region sequences; the VAADD instruction is a single calculation of the candidate regions in the two candidate region sequences.
  • the area sum between; j is an integer greater than or equal to 1 and less than or equal to R; and a candidate region that is not suppressed is determined according to the suppression vector of the sequence of each candidate region.
  • the processing unit 1201 determines the j suppression matrix of the j-th candidate region sequence by calling the VIOU instruction and the VAADD instruction, the processing unit 1201 is specifically configured to:
  • the area sum between two pairs of candidate regions in the adjacent two sets of candidate region sequences in the group j candidate region sequence that is factorized, and the candidate region of the jth group candidate region sequence and the jth group candidate region sequence The area sum between the two groups, and the overlap area between the candidate groups of the first group of candidate region sequences to the j-th group candidate region sequence and the j-th candidate region sequence Comparing the overlapping areas between the candidate regions in the j-th candidate region sequence, the j suppression matrices of the j-th candidate region sequence are obtained.
  • the processing unit 1201 determines the suppression vector of the j-th group candidate region sequence according to the j suppression matrices, the processing unit 1201 is specifically configured to:
  • the RPN_COR_DIAG instruction is called to calculate the suppression vector intermediate result, and the suppression vector of the j-th group candidate region sequence is obtained.
  • the RPN_COR_DIAG instruction is performed on a diagonal matrix block of the suppression matrix to obtain a suppression vector; and the RPN_COR instruction is a non-diagonal matrix block calculation on the suppression matrix, and the suppression vector intermediate result is obtained.
  • Embodiments of the present application also provide a computer storage medium for storing computer software instructions for use in the above data processing, including program code designed to execute the above method embodiments.
  • the application can be provided as a method, system, or computer program product.
  • the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment in combination of software and hardware.
  • the application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
  • the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Pure & Applied Mathematics (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Analysis (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Medical Informatics (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Algebra (AREA)
  • Advance Control (AREA)
  • Complex Calculations (AREA)

Abstract

一种数据处理的方法及装置，该方法包括：获取R组候选区域序列（201），每组候选区域序列包括多个候选区域；调用VRPAC指令计算各组候选区域序列中各候选区域的面积（202）；针对R组候选区域序列的第j组候选区域序列，调用VIOU指令、VAADD指令确定所述第j组候选区域序列的j个抑制矩阵，并根据j个抑制矩阵确定第j组候选区域序列的抑制矢量（203）；根据各组候选区域序列的抑制矢量，确定出未被抑制的候选区域（204）。所述方法减少了调用指令，减少了指令执行的步骤，从而缩短了NMS的计算时间。

Description

一种数据处理的方法及装置
本申请要求在2018年01月19日提交中华人民共和国知识产权局、申请号为201810054405.1、发明名称为“一种数据处理的方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及数据处理技术领域，尤其涉及一种数据处理的方法及装置。
背景技术
物体检测和图像识别是卷积神经网络的重要应用方向，非最大值抑制(non maximum suppression，NMS)是物体检测不可或缺的组件，负责获取同一物体唯一的检测。NMS的基本原理是比较2个候选区域交叠面积的比例，若该比例大于某一阈值，则会抑制得分低的候选区域。候选区域是一个矩形框，通过左上角坐标(y1,x1)和右下角坐标(y2,x2)进行表示。候选区域的得分反映该候选区域内部包含一个完整物体可能性的大小，得分越高则该候选区域包含一个完整物体的可能性越高。
NMS的输入是N个已经按照得分从高到低排好序的候选区域，输出M个得分最高且未被抑制的候选区域。
在计算NMS时，通常是使用中央处理器(central processing unit，CPU)进行NMS的计算，但是由于CPU并行计算能力有限，需要多次循环才能得到一个抑制矢量的计算，造成计算时间长的问题。另有通过CPU和图形处理器(graphics processing unit，GPU)GPU的组合进行NMS的计算的方案，其中，GPU可以执行NMS中并行计算时的计算工作，CPU可以执行除了并行计算之外的计算工作。但是使用非统一架构导致CPU与GPU之间需要额外的通信开销已完成NMS的计算，计算时间虽然比单纯使用CPU计算要快的多，但是仍然用时较长，同时还会增加额外的功耗。
发明内容
本申请提供一种数据处理的方法及装置，可以缩短NMS的计算时间。
第一方面，提供一种数据处理的方法，包括：
获取R组候选区域序列，每组候选区域序列包括多个候选区域，R为正整数，调用矢量候选区域面积计算(vector region proposal area calculation，VRPAC)指令计算各候选区域序列中各候选区域的面积；所述VRPAC指令为单次计算K个候选区域的面积，K与数据类型相关联，然后针对所述R组候选区域序列的第j组候选区域序列，调用矢量交叠面积(vector intersection over union，VIOU)指令、矢量面积和(vector area add，VAADD)指令确定所述第j组候选区域序列的j个抑制矩阵，并根据所述j个抑制矩阵确定所述第j组候选区域序列的抑制矢量，其中，所述VIOU指令为单次计算两组候选区域序列中候选区域两两之间的交叠面积，所述VAADD指令为单次计算两组候选区域序列中候选区域两两之间的面积和；j为大于等于1小于等于R的整数，最后根据所述各组候选区域序列的抑制矢量，确定出未被抑制的候选区域。
由于通过调用VRPAC指令各候选区域的面积，调用VIOU指令、VAADD指令来得到 抑制矩阵，与现有技术中通过GPU或者CPU来计算NMS相比，减少了调用指令，减少了指令执行的步骤，从而缩短了NMS的计算时间。
一种可能的设计中，所述调用VIOU指令、VAADD指令确定所述第j组候选区域序列的j个抑制矩阵，包括：调用所述VIOU指令计算第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的交叠面积以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的交叠面积，调用所述VAADD指令计算第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的面积和以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的面积和，将所述第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的面积和以及所述第j组候选区域序列与第j组候选区域序列中候选区域两两之间的面积和因子化，将因子化后的所述第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的面积和以及所述第j组候选区域序列与第j组候选区域序列中候选区域两两之间的面积和，与所述第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的交叠面积以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的交叠面积进行比较，得到所述第j组候选区域序列的j个抑制矩阵。
通过调用VIOU指令、VAADD指令来得到每组候选区域序列的抑制矩阵，与现有技术中通过GPU或者CPU来计算NMS相比，减少了调用指令，减少了指令执行的步骤，减少了取数据的次数，从而缩短了NMS的计算时间。
一种可能的设计中，所述根据所述j个抑制矩阵确定所述第j组候选区域序列的抑制矢量，包括：调用基于候选区域的网络的非对角线矩阵条件或(region proposal network condition or for non-diagonal suppression matrix，RPN_COR)指令对所述j个抑制矩阵中的前j-1个抑制矩阵计算，得到抑制矢量中间结果；调用基于候选区域的网络的对角线矩阵条件或(region proposal network diagonal condition or for non-diagonal suppression matrix，RPN_COR_DIAG)指令对所述j个抑制矩阵中的第j个抑制矩阵计算，并将计算结果与所述抑制矢量中间结果或，得到所述第j组候选区域序列的抑制矢量。
通过RPN_COR指令和RPN_COR_DIAG指令结合对抑制矩阵进行计算，得到抑制矢量，可以减少计算次数，进一步的缩短NMS的计算时间。
一种可能的设计中，所述RPN_COR_DIAG指令为对抑制矩阵的对角线矩阵块计算，得到抑制矢量；所述RPN_COR指令为对抑制矩阵的非对角线矩阵块计算，得到抑制矢量中间结果。
第二方面，提供一种数据处理的装置，包括：通信单元和处理单元；
所述通信单元，用于获取R组候选区域序列，每组候选区域序列包括多个候选区域，R为正整数；
所述处理单元，用于调用VRPAC指令计算各候选区域序列中各候选区域的面积；所述VRPAC指令为单次计算K个候选区域的面积，K与数据类型相关联；以及针对所述R组候选区域序列的第j组候选区域序列，调用VIOU指令、VAADD指令确定所述第j组候选区域序列的j个抑制矩阵，并根据所述j个抑制矩阵确定所述第j组候选区域序列的抑制矢量；其中，所述VIOU指令为单次计算两组候选区域序列中候选区域两两之间的交叠面积；所述VAADD指令为单次计算两组候选区域序列中候选区域两两之间的面积和；j为大于等于1小于等于R的整数；并根据所述各组候选区域序列的抑制矢量，确定出未被抑 制的候选区域。
一种可能的设计中，所述处理单元在调用VIOU指令、VAADD指令确定所述第j组候选区域序列的j个抑制矩阵时，具体用于：
调用所述VIOU指令计算第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的交叠面积以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的交叠面积；
调用所述VAADD指令计算第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的面积和以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的面积和；
将所述第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的面积和以及所述第j组候选区域序列与第j组候选区域序列中候选区域两两之间的面积和因子化；
将因子化后的所述第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的面积和以及所述第j组候选区域序列与第j组候选区域序列中候选区域两两之间的面积和，与所述第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的交叠面积以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的交叠面积进行比较，得到所述第j组候选区域序列的j个抑制矩阵。
一种可能的设计中，所述处理单元在根据所述j个抑制矩阵确定所述第j组候选区域序列的抑制矢量时，具体用于：
调用RPN_COR指令对所述j个抑制矩阵中的前j-1个抑制矩阵计算，得到抑制矢量中间结果；
调用RPN_COR_DIAG指令对所述j个抑制矩阵中的第j个抑制矩阵计算，并将计算结果与所述抑制矢量中间结果或，得到所述第j组候选区域序列的抑制矢量。
一种可能的设计中，所述RPN_COR_DIAG指令为对抑制矩阵的对角线矩阵块计算，得到抑制矢量；所述RPN_COR指令为对抑制矩阵的非对角线矩阵块计算，得到抑制矢量中间结果。
第三方面，提供一种数据读取的装置，包括：处理器和存储器；该存储器存储有计算机执行指令，该处理器与该存储器通过该总线连接，当该设备运行时，该处理器执行该存储器存储的该计算机执行指令，以使该设备执行如上述第一方面任意一项所述的方法。
第四方面，提供一种芯片，包括：处理单元和通信单元，处理单元例如可以是处理器，通信单元例如可以是输入/输出接口、管脚或电路等。该处理单元可执行存储单元存储的计算机执行指令，以使上述第一方面任意一项的数据处理的方法被执行。可选地，存储单元为芯片内的存储单元，如寄存器、缓存等，存储单元还可以是终端设备内的位于芯片外部的存储单元，如只读存储器、可存储静态信息和指令的其他类型的静态存储设备、随机存取存储器等。
第五方面，提供一种计算机可读存储介质，包括计算机可读指令，当计算机读取并执行所述计算机可读指令时，使得计算机执行如第一方面任意一项所述的方法。
第六方面，提供一种计算机程序产品，包括计算机可读指令，当计算机读取并执行所述计算机可读指令，使得计算机执行如第一方面任意一项所述的方法。
附图说明
图1为本申请提供的一种系统架构的示意图；
图2为本申请提供的一种数据处理的方法的流程示意图；
图3为本申请提供的一种VRPAC指令执行的示意图；
图4为本申请提供的一种VIOU指令执行的示意图；
图5为本申请提供的一种VIOU指令执行的示意图；
图6为本申请提供的一种VAADD指令执行的示意图；
图7为本申请提供的一种RPN_COR_DIAG指令执行的示意图；
图8为本申请提供的一种RPN_COR指令执行的示意图；
图9为本申请提供的一种RPN_COR指令与RPN_COR_DIAG指令结合执行的示意图；
图10为本申请提供的一种硬件结构的示意图；
图11为本申请提供的一种数据处理的装置的结构示意图；
图12为本申请提供的一种数据处理的装置的结构示意图。
具体实施方式
本申请提供的一种数据处理的方法，该方法可以应用于如图1所示的系统架构中。参考图1所示，为本申请的实施例提供的一种可能的系统架构。如图1所示，该系统结构包括分形矩阵乘法处理器(fractal matrix multiply processor)101、二级缓存器(L2buffer)102和主存储器(mian memory)103。
其中，分形矩阵乘法处理器101包括取指单元(Instruction Fetch Unit，IFU)1001、指令分发单元(Dispatch Unit)1002、一级源操作数缓存器(L1src Buffer(静态随机存取存储器(Static Random Access Memory，SRAM)))(用SRAM搭建的)1003、零级源操作数缓存器(L0src Buffer(DFF))(用寄存器搭建)1004、累加器(Accumulator)1005、零级目的操作数缓存器(L0Dest Buffer(DFF))1006、一级目的操作数缓存器(L1Dest Buffer(SRAM))1007、直接存储器访问单元(Direct Memory Access Unit，DMA Unit)1008、矢量单元(Vector Unit)1009、标量单元(Scalar Unit)1010、统一缓存(Unified Buffer，UB)1011和总线接口单元(Bus Interface Unit，BIU)1012。其中，L1src Buffer(SRAM)1003、L0src Buffer(DFF)1004、Accumulator1005、L0Dest Buffer(DFF)1006、L1Dest Buffer(SRAM)1007可以组成分形矩阵乘法单元。IFU1001可以通过BIU1012从主存103中获取指令，并解码和控制执行流程，指令分发单元1002可以提交对应类型指令到4个流水线单元，指令分发单元1002可以有机控制4个流水线之间的执行保序。流水线单元有异步执行和同步执行两种类型，所有类型都为保序发射，区别是异步执行单元执行指令异步结束，同步执行单元为同步结束。标量单元1010为同步执行，分形矩阵乘法处理单元、DMA单元1008和矢量单元1009为异步执行。所有执行单元可以通过统一缓存1011读写交互数据。分形矩阵乘法单元内部有L1和L0两级专用缓存，L1缓存和统一缓存1011通过DMA单元1008和外部存储空间交互数据。外部存储空间由多级存储单元组成。例如，统一缓存1011的容量为256KB，分64个存储体实现，每个存储体为32B宽度，排好序的候选区域在统一缓存1011中可以按照如表1所示的方式进行存储。
表1
比特索引 30 28 26 24 22 20 18 16 14 12 10 8 6 4 2 0
UB地址off 0         y2[0] x2[0] y1[0] x1[0]         y2[0] x2[0] y1[0] x1[0]
UB地址off 1         y2[1] x2[1] y1[1] x1[1]         y2[1] x2[1] y1[1] x1[1]
UB地址off 2         y2[2] x2[2] y1[2] x1[2]         y2[2] x2[2] y1[2] x1[2]
UB地址off 3         y2[3] x2[3] y1[3] x1[3]         y2[3] x2[3] y1[3] x1[3]
UB地址off 4         y2[4] x2[4] y1[4] x1[4]         y2[4] x2[4] y1[4] x1[4]
UB地址off 5         y2[5] x2[5] y1[5] x1[5]         y2[5] x2[5] y1[5] x1[5]
UB地址off 6         y2[6] x2[6] y1[6] x1[6]         y2[6] x2[6] y1[6] x1[6]
UB地址off 7         y2[7] x2[7] y1[7] x1[7]         y2[7] x2[7] y1[7] x1[7]
基于上述描述，图2示例性的示出了本申请提供的一种数据处理的方法的流程，该数据数量的方法可以应用于图像处理的领域，该流程可以由上述分形矩阵乘法处理器来执行。
如图2所示，该流程具体包括：
步骤201，获取R组候选区域序列。
该R组候选区域为待输入的数据，为已经排好序的候选区域。每组候选区域序列可以包括多个候选区域，每组候选区域的数量与输入的数据类型的位数有关，其中数据类型可以包括8位无符号整数(8-bit unsigned integer，u8)、8位有符号整数(8-bit signed integer，s8)、16位浮点数(16-bit floating point，fp16)和32位浮点数(32-bit floating point，fp32)。例如，假设数据类型为fp16，输入N个候选区域，则N为16的倍数，N＝16*R，R为候选区域的组数。这R组候选区域可以按照如表1所示存储地址存储在UB中。在本申请中数据类型不限于上述几种类型，仅是示例作用。
步骤202，调用VRPAC指令计算各组候选区域序列中各候选区域的面积。
在计算非最大值抑制(non maximum suppression，NMS)过程中，首先需要调用VRPAC指令来计算各组候选区域序列中每个候选区域的面积。该VRPAC指令为单次计算K个候选区域的面积，其中，K与数据类型的位数相关联。也就是说，K的取值可以与数据类型的位数相同，例如，数据类型为fp16时，该VRPAC指令单次可以计算16个候选区域的面积。K可以取值为8、16或32等。该VRPAC指令可以支持重复模式，即通过设置重复次数来自动完成多组候选区域的面积的计算。该VRPAC指令可以复用现有的M个加法器资源和(M/4)个乘法器资源。该VRPAC指令可以根据目标频率的不同分为1拍、2拍、3拍等完成计算任务。该VRPAC指令可以根据矢量单元加法及乘法的资源数量来确定该VRPAC指令是完全管线(pipeline)执行还是非完全pipeline执行。
举例来说，如图3所示的结构图，K取值为16，数据类型为fp16，复用矢量单元的64个fp16加法器和16个fp16乘法器，可完全pipeline执行。P0到P15为候选区域，经过加法器和乘法器的计算，最后输出16个候选区域的面积。
从图3中可以看出，要计算出一个候选区域的面积，需要进行2次加法操作、2次加法操作和1次乘法操作就可以完成，也就是调用一次VRPAC指令可以一次性的得到K*R个结果，重复R次，但是只调用一次指令。而目前通过CPU计算得到多个候选区域的面积时，每次循环需要顺序执行5条指令，其中，第1条指令是计算候选区域的横坐标x2-x1得到中间结果res_tmp0；第2条指令计算res_tmp0与1的和，可以得到中间结果res_tmp1；第3条指令计算纵坐标y2-y1得到中间结果res_tmp2；第4条指令计算res_tmp2与1的和，可以得到中间结果res_tmp3；第五条指令计算res_tmp1与res_tmp3的乘积，从而得到一个候选区域的面积值。通过这种方式执行，在性能上，res_tmp1与res_tmp3没有任何数据依赖，但是必须顺序得到，从而降低性能；另外res_tmp1依赖于res_tmp0的得到(同理， res_tmp3依赖于res_tmp2的得到)，res_tmp1须等到res_tmp0写回到寄存器才可以开始执行，现代处理器通常会通过旁路的方式来减少因数据相关引起的性能气泡，但是通常浮点加法乘法都要分多拍完成，会导致气泡无法完全消除的性能问题。从功耗存储上讲，5条指令需要占用5条指令的存储空间，而且每条指令都要完整的进行取指、译码、执行、回写寄存器的操作，能效比低。
在通过GPU来计算候选区域的面积时，GPU通常通过将多个候选区域的计算分派到不同的线程上去执行，以提高并行度，单个线程的计算过程跟CPU类似，每个线程也会面临在CPU上面临的性能、功耗、存储问题。GPU可以通过不同线程的切换来隐藏计算的气泡，但这是以额外的寄存器资源为代价的。
而本申请提供的VRPAC指令，只使用一条指令，且在实际实现的时候分成3个基本步骤，第一步同时得到res_tmp0和res_tmp2，第二步，同时得到res_tmp1和res_tmp3。第三步，得到最终的结果。从性能上，基本步骤比上述2种方式少2步，且步骤跟步骤之间完全由硬件流水执行，且不会存在潜在的性能气泡问题，缩短了计算时间。从功耗存储上来讲，只有一条指令，且支持重复模式一条指令可完成R*K(R为重复次数，K为一次重复的并行度)个候选区域的计算。指令在执行的时候只涉及一次取值，一次译码，只有最终的结果会回写到UB中。从而最大程度减少了额外功耗。能效比会比GPU和CPU都要高，节省了大量计算时间。
步骤203，针对所述R组候选区域序列的第j组候选区域序列，调用VIOU指令、VAADD指令确定所述第j组候选区域序列的j个抑制矩阵，并根据所述j个抑制矩阵确定所述第j组候选区域序列的抑制矢量。
在本申请中，该VIOU指令可以为单次计算两组候选区域序列中候选区域两两之间的交叠面积，例如，每组有K个候选区域，则是计算K*K个交叠面积。该VIOU指令也可以支持重复模式，通过设置重复次数，自动完成多组候选区域的交叠面积的计算，可以复用现用的M个比较器、M个加法器和(M/4)个乘法器，根据目标频率的不同可以分成1拍、2拍、3拍等完成计算。可以根据矢量单元的比较器、加法器及乘法器的资源数量来确定该VIOU指令是完全pipeline执行还是非完全pipeline执行。
例如，单个交叠面积的计算可以如图4所示，两个候选区域将坐标经过一次比较、2次加法、2次加法和比较以及1次乘法即可以完成。当K为16，M为128时，可以使用如图5所示的硬件来实现。
在现有的方案中，使用CPU或者GPU来实现计算候选区域的交叠面积时，如果使用一个线程来做，需要11条指令来完成，也就是相同于通过11个步骤来实现。而采用VIOU指令，则只需要4个基本步骤，第一步，同时得到xx1,yy1,xx2,yy2的获得。第二步，同时得到res_tmp0和res_tmp2，第三步，同时得到res_tmp1和res_tmp3，且会完成res_tmp4和res_tmp5的得到，因为跟0做比较只需耗费很少逻辑。第四步，得到最终结果，也就是需要计算的交叠面积。从性能上看，采用VIOU指令执行的基本步骤比现有的使用CPU或GPU的方式少7个步骤，且步骤跟步骤之间完全由硬件流水执行，且不会存在潜在的性能气泡问题，而由于减少了计算步骤，同样是缩短了计算时间。从功耗存储上来讲，只有一条指令，且支持重复模式一条指令可完成R*K(R为重复次数，K为一次重复的并行度，一次计算几个)个候选区域的交叠面积的计算。VIOU指令在执行的时候只涉及一次取值，一次译码，只有最终的结果会回写到UB中。从而最大程度减少了额外功耗。能效比会比 GPU和CPU都要高。
上述VAADD指令可以为单次计算两组候选区域序列中候选区域两两之间的面积和。例如，每组有K个候选区域，则是计算K*K个面积和。该VAADD指令支持重复模式，通过设置重复次数自动完成多组候选区域的面积和的计算。可以复用现有的M个加法器，根据目标频率的不同，可以分为1拍、2拍、3拍等执行，根据矢量单元加法器的资源数量确定该指令是完全pipeline执行还是非完全pipeline执行。
如图6所示，在K取值为16，M取值为128时，可以使用如图6所示的硬件结构来实现，没2拍可以产生256个结果，非完全pipeline执行。VAADD指令从计算过程上看，仅仅只有加法步骤，由于要计算2组候选区域两两之间的面积和，采用VAADD指令可以减少访问UB的顺序。而CPU和GPU对同一个候选区域要取回多次才可以完成同样的效果，因此同样是缩短了计算时间。
基于上述VIOU指令和VAADD指令，在确定第j组候选区域序列的j个抑制矩阵时，具体可以为：调用VIOU指令计算第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的交叠面积以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的交叠面积，调用VAADD指令计算第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的面积和以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的面积和，然后将第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的面积和以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的面积和因子化。最后将因子化后的第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的面积和以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的面积和，与第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的交叠面积以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的交叠面积进行比较，得到第j组候选区域序列的j个抑制矩阵。其中，j可以为大于等于1小于等于R的整数。
例如，j取值为3时，首先调用VIOU指令，设置重复次数为3，计算第1组候选区域与第2组候选区域的交叠面积、第2组候选区域与第3组候选区域的交叠面积、第3组候选区域与第3组候选区域的交叠面积。调用VAADD指令，设置重复次数为3，计算第1组候选区域与第2组候选区域的面积和、第2组候选区域与第3组候选区域的面积和、第3组候选区域与第3组候选区域的面积和。然后将计算得到的面积和因子化，也就是将得到的面积和乘以一个参数因子a，该参数因子可以依据经验设置。将面积和因子化后，可以将因子化的面积和与计算得到的交叠面积进行比较，从而可以得到第3组候选区域序列的3个抑制矩阵。最终得到的抑制矩阵为一个N*N的矩阵(N为总的候选区域的数量)，该矩阵的对角线及其以上的元素值都为0。
需要说明的是，为了得到抑制矩阵，在使用VRPAC指令在主存中取需要计算的数据时，一次计算需要取R*K次数据，其中，R为候选区域序列组的数量，K为每组候选区域序列中包括的候选区域的数量。在使用VIOU指令或VAADD指令去需要计算的数据时，一次计算需要取
Figure PCTCN2019071737-appb-000001
次数据，也就是R 2*2*K次。而现有的使用CPU或者GPU的方案中，在为了得到抑制矩阵，调用每个指令一次计算需要取
Figure PCTCN2019071737-appb-000002
次数据，也就 是(R*K) 2次。将CPU或GPU计算时取数据的次数与本申请中使用VIOU指令或VAADD指令取数据的次数相比，可以得到两者一次计算取数据的次数相差为
Figure PCTCN2019071737-appb-000003
倍。例如，只取一组候选区域，且一组候选区域包括16个候选区域，取数据的次数两者相差8倍。又例如，当取64组候选区域，每组候选区域包括16个候选区域时，本申请中使用VIOU指令或VAADD指令取数据的次数为64 2*2*16＝131072次，而现有的技术方案在使用指令取数据的次数为(64*16) 2＝1048576次，两者相比差917504次。当取的数据越多时，两者的差距越明显，显然，本申请的技术方案可以节省大量的取数据的次数，从而节省了计算时间。而且，CPU或GPU计算的步骤要多于本申请的技术方案，进一步的增加了取数据的次数的差距。因此，相比于现有的NMS的计算方案，本申请提供的技术方案在取数据的次数上看可以节省很多，相应的降低了访问主存的次数，也就缩短了计算时间，减少了额外的功耗。
在得到第j组候选区域序列的j个抑制矩阵后，可以根据该j个抑制矩阵确定出该第j组候选区域序列的抑制矢量。具体的，可以调用基于候选区域的网络的非对角线矩阵条件或(region proposal network condition or for non-diagonal suppression matrix，RPN_COR)指令对j个抑制矩阵中的前j-1个抑制矩阵计算，得到抑制矢量中间结果，然后调用基于候选区域的网络的对角线矩阵条件或(region proposal network diagonal condition or for non-diagonal suppression matrix，RPN_COR_DIAG)指令对j个抑制矩阵中的第j个抑制矩阵计算，并将计算结果与所述抑制矢量中间结果或，得到第j组候选区域序列的抑制矢量。
RPN_COR_DIAG指令可以为对抑制矩阵的对角线矩阵块计算，得到抑制矢量。也就是从抑制矩阵的对角线矩阵块(KxK)到抑制矢量(Kx1)的计算，其可以根据目标频率不同可分为1拍，2拍或多拍实现，可完全pipeline执行。该RPN_COR_DIAG指令的输入有2个，一个是对角线矩阵块，一个是存放中间结果的特殊寄存器。抑制矢量计算过程中，得分低的结果依赖于得分高的结果，即第i个分量的计算，依赖于0～i-1分量的计算结果。若某个抑制矢量的分量已经为1，则该候选区域对后续得分低的候选区域的抑制关系要取消。之前计算得到的中间结果放到一个特殊寄存器RPN_COR_IR，与本次得到的结果按位或，可得到该16个候选区域的最终抑制结果。
举例来说，如图7所示，得分最高的候选区域不会被抑制，因此抑制矢量中间结果[0]＝0。候选区域2没有被候选区域0、1抑制，因此候选区域2最终不会被抑制。但是候选区域2将候选区域5抑制，因此，候选区域5最终被抑制。候选区域12被候选区域5抑制，但是候选区域5已经被抑制，因此候选区域5对候选区域12的抑制取消，最终候选区域12未被抑制。得到的中间结果与特殊寄存器RPN_COR_IR(存放之前的抑制结果)进行按位或，得到这16个候选区域的最终抑制矢量。
该RPN_COR指令可以为对抑制矩阵的非对角线矩阵块计算，得到抑制矢量中间结果。也就是抑制矩阵非对角线矩阵块(KxK)到抑制矢量中间结果的计算，并更新特殊寄存器RPN_COR_IR。可以支持重复模式，通过设置重复次数，可实现对多个非对角线矩阵块的计算。根据频率目标可1拍，2拍，3拍等完成，可完全pipeline执行。该RPN_COR指令的输入有3个，一个是抑制矩阵的非对角线矩阵框(16x16)，一个是得分更高的16抑制矢量的值，另一个是存放中间结果的特殊寄存器RPN_COR_IR。在计算过程中，若输入的抑制矢量有为1的，则该候选区域对后续得分低的候选区域的抑制要取消。将本次得到的抑 制矢量结果OR上RPN_COR_IR并写入到特殊寄存器RPN_COR_IR，用以不断累积计算结果。最终由RPN_COR_DIAG指令得到该16个候选区域的最终抑制矢量。
举例来说，如图8所示，候选区域0对候选区域16有抑制，且候选区域0未被抑制，因此候选区域0对候选区域16的抑制有效。候选区域15对候选区域16也有抑制，但是候选区域15被抑制掉，因此候选区域15对候选区域16的抑制取消，最终候选区域16本轮计算结果被抑制掉。候选区域5和候选区域7对候选区域20都有抑制，但是候选区域5和候选区域7都被抑制，因此候选区域5和候选区域7对候选区域20的抑制取消，最终本轮计算候选区域20未被抑制。最后，本轮计算得到的16个中间结果或按位或上特殊寄存器的值并再次更新该特殊寄存器。
又例如，结合RPN_COR指令和RPN_COR_DIAG指令联合使用，如图9所示，要得到抑制矢量[15:0]的值，先初始化RPN_COR_IR为0，然后对矩阵框0调用RPN_COR_DIAG一次。要得到抑制矢量[31:16]的值，先初始化RPN_COR_IR为0，然后对矩阵框1调用RPN_COR一次，然后对矩阵框2调用RPN_COR_DIAG一次。要得到抑制矢量[47:32]的值，先初始化RPN_COR_IR为0，然后对矩阵框3、4分别调用RPN_COR，最后对矩阵框5调用RON_COR_DIAG一次。还例如，得到3个抑制矩阵，先初始化RPN_COR_IR为0，然后调用RPN_COR指令对前两个抑制矩阵的矩阵框进行计算，得到抑制矢量中间结果，将抑制矢量中间结果更新RPN_COR_IR，然后在调用RPN_COR_DIAG对第3个抑制矩阵的矩阵框进行计算，并将计算结果与RPN_COR_IR中的抑制矢量中间结果按位或，得到最终的抑制矢量。
在确定第j组候选区域序列的抑制矢量时，将RPN_COR_IR寄存器初始化成0，调用RPN_COR指令，重复次数设置为j-1次，源操作数为非对角线抑制矩阵的矩阵框，结果更新RPN_COR_IR寄存器，之后调用RPN_COR_DIAG，从而得到第j组候选区域的抑制矢量。
步骤204，根据所述各组候选区域序列的抑制矢量，确定出未被抑制的候选区域。
在得到所有的候选区域序列的抑制矢量后，可以取前M个非0的候选区域为最终结果，也就是前M个未被抑制的候选区域。
为了更好的解释本申请提供的数据处理方法，下面将在具体的实施场景下来描述数据处理的过程。如图10所示的硬件结构，上述VRPAC指令、VIOU指令和VAADD指令可以复用一个硬件结构，如图10所示的结构，一次计算可以产生一个VRPAC结果，一个VIOU结果和4个VAADD结果。若要一次产生更多个VRPAC、VIOU或VAADD结果，则需要复制上述电路结构若干份即可实现。
基于图10所示的硬件结构，结合VRPAC指令、VIOU指令、VAADD指令、RPN_COR指令和RPN_COR_DIAG指令来描述NMS的计算过程。
具体的，数据类型为fp16，其中，VRPAC指令一次产生16个计算结果，VIOU指令计算2组候选区域(每组16个)两两之间的交叠面积，产生256个计算结果，VAADD指令计算2组候选区域(每组16个)两两之间的面积和，产生256个计算结果，RPN_COR指令一次得到16个抑制矢量的中间结果，RPN_COR_DIAG指令一次得到16个抑制矢量。
假设输入为N(假定N为16的倍数，N＝16*R，R为总的候选区域组数)个已经排好序的候选区域。
步骤1：调用VRPAC指令，重复次数设置为R，指令执行完之后可在UB中得到N 个候选区域的面积值。
步骤2：计算第0组候选区域抑制矢量的结果，该步骤可分为如下子步骤：
步骤2.1：调用VIOU指令，计算第0组候选区域(16个候选区域)和第0组候选区域的交叠面积。
步骤2.2：调用VAADD指令，计算第0组候选区域和第0组候选区域的面积和。
步骤2.3：调用普通的矢量指令(标量乘向量)，将步骤2.2得到的结果乘一个参数因子a。
步骤2.4：调用普通的矢量比较指令，比较步骤2.1得到的交叠面积与步骤2.3的得到的因子化的面积和，从而得到第0组候选区域的1个抑制矩阵。
步骤2.5：对RPN_COR_IR特殊寄存器初始化成0，调用RPN_COR_DIAG指令，源操作数为步骤2.4的结果，可得到第0组候选区域的抑制矢量。
步骤3：计算第1组候选区域抑制矢量的结果，该步骤可分为如下子步骤：
步骤3.1：调用VIOU指令，重复次数设置为2次，可得到第0组候选区域跟第1组候选区域，与第1组候选区域跟第1组候选区域的交叠面积值。
步骤3.2：调用VAADD指令，重复次数设置为2次，可得到第0组候选区域跟第1组候选区域，与第1组候选区域跟第1组候选区域的面积和。
步骤3.3：调用普通的矢量指令(标量乘向量)，将步骤3.2得到的结果乘一个参数因子a。
步骤3.4：调用普通的矢量比较指令，比较步骤3.1的结果跟步骤3.3的结果得到第1组候选区域的2个抑制矩阵。
步骤3.5：将RPN_COR_IR特殊寄存器初始化成0，调用RPN_COR指令，重复次数设置为1次，源操作数为非对角线抑制矩阵的矩阵框，结果更新RPN_COR_IR寄存器，之后调用RPN_COR_DIAG，从而得到第1组候选区域的抑制矢量。
依次类推，步骤N+1：计算第(R-1)组，即最后一组抑制矢量的结果，该步骤可分为如下子步骤：
步骤N+1.1：调用VIOU指令，重复次数设置为R次，计算第0组候选区域跟第R-1组候选区域，第1组候选区域跟第2组候选区域，。。。，第R-1组候选区域跟第R-1组候选区域的交叠面积。
步骤N+1.2：调用VAADD指令，重复次数设置为R次，计算第0组候选区域跟第R-1组候选区域，第1组候选区域跟第2组候选区域，。。。，第R-1组候选区域跟第R-1组候选区域的面积和。
步骤N+1.3：调用普通的矢量指令，将步骤N+1.2得到的结果乘一个参数因子a。
步骤N+1.4：调用普通的矢量比较指令，比较步骤N+1.1的结果跟步骤N+1.3的结果，得到第(R-1)组的R个抑制矩阵。
步骤N+1.5：将RPN_COR_IR寄存器初始化成0，调用RPN_COR指令，重复次数设置为(R-1)次，源操作数为非对角线抑制矩阵的矩阵框，结果更新RPN_COR_IR寄存器，之后调用RPN_COR_DIAG，从而得到第(R-1)组候选区域的抑制矢量。
步骤N+2：根据抑制矢量，取前M个非0的候选区域为最终的结果。
通过上述实施例，可以显著缩短NMS计算所需时间，大幅提高计算能效，在涉及图像检测和图像识别的应用领域提升竞争力。
基于相同的技术构思，图11示例性的示出了本申请提供了一种数据处理的装置的结构，该装置可以执行数据处理的流程。该装置可以是芯片，可执行上述任一实施例中数据处理的方法。
该装置1100包括至少一个处理器1101，通信接口1102，可选地，还包括存储器1103。所述处理器1101、通信接口1102、存储器1103互相连接。
处理器1101可以是一个通用中央处理器，微处理器，图像处理器，特定应用集成电路，或一个或多个用于控制本申请的实施例的程序执行的集成电路。
所述通信接口1102，用于输入或输出数据。
存储器1103可以是只读存储器或可存储静态信息和指令的其他类型的静态存储设备随机存取存储器或者可存储信息和指令的其他类型的动态存储设备，也可以是电可擦可编程只读存储器、只读光盘或其他光盘存储、光碟存储(包括压缩光碟、激光碟、光碟、数字通用光碟、蓝光光碟等)、磁盘存储介质或者其他磁存储设备、或者能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质，但不限于此。存储器1103可以是独立存在，与处理器1101相连接。存储器1103也可以和处理器集成在一起。其中，所述存储器1103用于存储执行本申请的实施例的应用程序代码，并由处理器1101来控制执行。所述处理器1101用于执行所述存储器1103中存储的应用程序代码。
在具体实现中，作为一种实施例，处理器1101可以包括一个或多个CPU，例如图11中的CPU0和CPU1。
在具体实现中，作为一种实施例，装置1100可以包括多个处理器，例如图11中的处理器1101和处理器1108。这些处理器中的每一个可以是一个单核(single-CPU)处理器，也可以是一个多核(multi-CPU)处理器，这里的处理器可以指一个或多个设备、电路、和/或用于处理数据(例如计算机程序指令)的处理核。
本申请可以根据上述方法示例对终端设备进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。比如，在采用对应各个功能划分各个功能模块的情况下，图12示出了一种装置示意图，该装置可以是上述实施例中所涉及的终端设备，该装置包括处理单元1201和通信单元1202。
所述通信单元1202，用于获取R组候选区域序列，每组候选区域序列包括多个候选区域，R为正整数；
所述处理单元1201，用于调用VRPAC指令计算各候选区域序列中各候选区域的面积；所述VRPAC指令为单次计算K个候选区域的面积，K与数据类型相关联；以及针对所述R组候选区域序列的第j组候选区域序列，调用VIOU指令、VAADD指令确定所述第j组候选区域序列的j个抑制矩阵，并根据所述j个抑制矩阵确定所述第j组候选区域序列的抑制矢量；其中，所述VIOU指令为单次计算两组候选区域序列中候选区域两两之间的交叠面积；所述VAADD指令为单次计算两组候选区域序列中候选区域两两之间的面积和；j为大于等于1小于等于R的整数；并根据所述各组候选区域序列的抑制矢量，确定出未被抑制的候选区域。
可选的，所述处理单元1201在调用VIOU指令、VAADD指令确定所述第j组候选区域序列的j个抑制矩阵时，具体用于：
调用所述VIOU指令计算第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的交叠面积以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的交叠面积；
调用所述VAADD指令计算第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的面积和以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的面积和；
将所述第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的面积和以及所述第j组候选区域序列与第j组候选区域序列中候选区域两两之间的面积和因子化；
将因子化后的所述第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的面积和以及所述第j组候选区域序列与第j组候选区域序列中候选区域两两之间的面积和，与所述第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的交叠面积以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的交叠面积进行比较，得到所述第j组候选区域序列的j个抑制矩阵。
可选的，所述处理单元1201在根据所述j个抑制矩阵确定所述第j组候选区域序列的抑制矢量时，具体用于：
调用RPN_COR指令对所述j个抑制矩阵计算，得到抑制矢量中间结果；
调用RPN_COR_DIAG指令对所述抑制矢量中间结果计算，得到所述第j组候选区域序列的抑制矢量。
可选的，所述RPN_COR_DIAG指令为对抑制矩阵的对角线矩阵块计算，得到抑制矢量；所述RPN_COR指令为对抑制矩阵的非对角线矩阵块计算，得到抑制矢量中间结果。
本申请的实施例还提供了一种计算机存储介质，用于储存为上述数据处理所用的计算机软件指令，其包含用于执行上述方法实施例所设计的程序代码。
本领域内的技术人员应明白，本申请可提供为方法、系统、或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本申请是参照根据本申请的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机 或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
显然，本领域的技术人员可以对本申请进行各种改动和变型而不脱离本申请的精神和范围。这样，倘若本申请的这些修改和变型属于本申请权利要求及其等同技术的范围之内，则本申请也意图包含这些改动和变型在内。

Claims (11)

  1. 一种数据处理的方法，其特征在于，包括：
    获取R组候选区域序列，每组候选区域序列包括多个候选区域，R为正整数；
    调用矢量候选区域面积计算VRPAC指令计算各组候选区域序列中各候选区域的面积；所述VRPAC指令为单次计算K个候选区域的面积，K与数据类型相关联；
    针对所述R组候选区域序列的第j组候选区域序列，调用矢量交叠面积VIOU指令、矢量面积和VAADD指令确定所述第j组候选区域序列的j个抑制矩阵，并根据所述j个抑制矩阵确定所述第j组候选区域序列的抑制矢量；其中，所述VIOU指令为单次计算两组候选区域序列中候选区域两两之间的交叠面积；所述VAADD指令为单次计算两组候选区域序列中候选区域两两之间的面积和；j为大于等于1小于等于R的整数；
    根据所述各组候选区域序列的抑制矢量，确定出未被抑制的候选区域。
  2. 如权利要求1所述的方法，其特征在于，所述调用VIOU指令、VAADD指令确定所述第j组候选区域序列的j个抑制矩阵，包括：
    调用所述VIOU指令计算第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的交叠面积以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的交叠面积；
    调用所述VAADD指令计算第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的面积和以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的面积和；
    将所述第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的面积和以及所述第j组候选区域序列与第j组候选区域序列中候选区域两两之间的面积和因子化；
    将因子化后的所述第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的面积和以及所述第j组候选区域序列与第j组候选区域序列中候选区域两两之间的面积和，与所述第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的交叠面积以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的交叠面积进行比较，得到所述第j组候选区域序列的j个抑制矩阵。
  3. 如权利要求1所述的方法，其特征在于，所述根据所述j个抑制矩阵确定所述第j组候选区域序列的抑制矢量，包括：
    调用基于候选区域的网络的非对角线矩阵条件或RPN_COR指令对所述j个抑制矩阵中的前j-1个抑制矩阵计算，得到抑制矢量中间结果；
    调用基于候选区域的网络的对角线矩阵条件或RPN_COR_DIAG指令对所述j个抑制矩阵中的第j个抑制矩阵计算，并将计算结果与所述抑制矢量中间结果或，得到所述第j组候选区域序列的抑制矢量。
  4. 如权利要求3所述的方法，其特征在于，所述RPN_COR_DIAG指令为对抑制矩阵的对角线矩阵块计算，得到抑制矢量；所述RPN_COR指令为对抑制矩阵的非对角线矩阵块计算，得到抑制矢量中间结果。
  5. 一种数据处理的装置，其特征在于，包括：通信单元和处理单元；
    所述通信单元，用于获取R组候选区域序列，每组候选区域序列包括多个候选区域， R为正整数；
    所述处理单元，用于调用矢量候选区域面积计算VRPAC指令计算各候选区域序列中各候选区域的面积；所述VRPAC指令为单次计算K个候选区域的面积，K与数据类型相关联；以及针对所述R组候选区域序列的第j组候选区域序列，调用矢量交叠面积VIOU指令、矢量面积和VAADD指令确定所述第j组候选区域序列的j个抑制矩阵，并根据所述j个抑制矩阵确定所述第j组候选区域序列的抑制矢量；其中，所述VIOU指令为单次计算两组候选区域序列中候选区域两两之间的交叠面积；所述VAADD指令为单次计算两组候选区域序列中候选区域两两之间的面积和；j为大于等于1小于等于R的整数；并根据所述各组候选区域序列的抑制矢量，确定出未被抑制的候选区域。
  6. 如权利要求5所述的装置，其特征在于，所述处理单元在调用VIOU指令、VAADD指令确定所述第j组候选区域序列的j个抑制矩阵时，具体用于：
    调用所述VIOU指令计算第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的交叠面积以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的交叠面积；
    调用所述VAADD指令计算第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的面积和以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的面积和；
    将所述第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的面积和以及所述第j组候选区域序列与第j组候选区域序列中候选区域两两之间的面积和因子化；
    将因子化后的所述第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的面积和以及所述第j组候选区域序列与第j组候选区域序列中候选区域两两之间的面积和，与所述第1组候选区域序列到第j组候选区域序列中相邻两组候选区域序列中候选区域两两之间的交叠面积以及第j组候选区域序列与第j组候选区域序列中候选区域两两之间的交叠面积进行比较，得到所述第j组候选区域序列的j个抑制矩阵。
  7. 如权利要求5所述的装置，其特征在于，所述处理单元在根据所述j个抑制矩阵确定所述第j组候选区域序列的抑制矢量时，具体用于：
    调用基于候选区域的网络的非对角线矩阵条件或RPN_COR指令对所述j个抑制矩阵中的前j-1个抑制矩阵计算，得到抑制矢量中间结果；
    调用基于候选区域的网络的对角线矩阵条件或RPN_COR_DIAG指令对所述j个抑制矩阵中的第j个抑制矩阵计算，并将计算结果与所述抑制矢量中间结果或，得到所述第j组候选区域序列的抑制矢量。
  8. 如权利要求7所述的装置，其特征在于，所述RPN_COR_DIAG指令为对抑制矩阵的对角线矩阵块计算，得到抑制矢量；所述RPN_COR指令为对抑制矩阵的非对角线矩阵块计算，得到抑制矢量中间结果。
  9. 一种芯片，其特征在于，所述芯片与存储器相连，用于读取并执行所述存储器中存储的软件程序，以实现根据权利要求1至4任一项所述的方法。
  10. 一种计算机存储介质，其特征在于，包括计算机可读指令，当计算机读取并执行所述计算机可读指令时，使得计算机执行如权利要求1至4任一项所述的方法。
  11. 一种计算机程序产品，其特征在于，包括计算机可读指令，当计算机读取并执行 所述计算机可读指令，使得计算机执行如权利要求1-4任意一项所述的方法。
PCT/CN2019/071737 2018-01-19 2019-01-15 一种数据处理的方法及装置 Ceased WO2019141160A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP19741008.7A EP3736765A4 (en) 2018-01-19 2019-01-15 DATA PROCESSING PROCESS AND APPARATUS
US16/932,768 US11823303B2 (en) 2018-01-19 2020-07-19 Data processing method and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810054405.1 2018-01-19
CN201810054405.1A CN110060195B (zh) 2018-01-19 2018-01-19 一种数据处理的方法及装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/932,768 Continuation US11823303B2 (en) 2018-01-19 2020-07-19 Data processing method and apparatus

Publications (1)

Publication Number Publication Date
WO2019141160A1 true WO2019141160A1 (zh) 2019-07-25

Family

ID=67301259

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/071737 Ceased WO2019141160A1 (zh) 2018-01-19 2019-01-15 一种数据处理的方法及装置

Country Status (4)

Country Link
US (1) US11823303B2 (zh)
EP (1) EP3736765A4 (zh)
CN (1) CN110060195B (zh)
WO (1) WO2019141160A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7089179B2 (ja) * 2018-08-30 2022-06-22 富士通株式会社 画像認識装置、画像認識方法および画像認識プログラム
WO2023089745A1 (ja) * 2021-11-18 2023-05-25 オリンパス株式会社 演算処理装置
US12204897B2 (en) 2022-11-21 2025-01-21 Nvidia Corporation Application programming interface to wait on matrix multiply-accumulate

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105654031A (zh) * 2014-10-22 2016-06-08 通用汽车环球科技运作有限责任公司 用于物体检测的系统和方法
CN106056101A (zh) * 2016-06-29 2016-10-26 哈尔滨理工大学 用于人脸检测的非极大值抑制方法
CN107463912A (zh) * 2017-08-10 2017-12-12 武汉大学深圳研究院 基于运动显著性的视频人体行为识别方法

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1526480A1 (en) * 2000-10-17 2005-04-27 Fuji Photo Film Co., Ltd Apparatus for suppressing noise by adapting filter characteristics to input image signal based on characteristics of input image signal
US7706586B2 (en) * 2005-06-22 2010-04-27 General Electric Company Real-time structure suppression in ultrasonically scanned volumes
WO2010137324A1 (ja) * 2009-05-29 2010-12-02 三菱電機株式会社 画像符号化装置、画像復号装置、画像符号化方法、および画像復号方法
CN102750707A (zh) * 2011-08-29 2012-10-24 新奥特（北京）视频技术有限公司 一种基于感兴趣区域的图像处理方法和装置
US10169661B2 (en) * 2014-03-28 2019-01-01 International Business Machines Corporation Filtering methods for visual object detection
CN104239854B (zh) * 2014-08-30 2017-07-11 电子科技大学 一种基于区域稀疏积分通道的行人特征提取和表示方法
US10643101B2 (en) * 2015-07-09 2020-05-05 Texas Instruments Incorporated Window grouping and tracking for fast object detection
US9984305B2 (en) * 2016-04-19 2018-05-29 Texas Instruments Incorporated Efficient SIMD implementation of 3x3 non maxima suppression of sparse 2D image feature points
CN106452769B (zh) * 2016-09-13 2019-11-29 杭州华为数字技术有限公司 一种数据处理方法、数据处理装置以及处理器
CN111859273B (zh) 2017-12-29 2025-01-21 华为技术有限公司 矩阵乘法器

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105654031A (zh) * 2014-10-22 2016-06-08 通用汽车环球科技运作有限责任公司 用于物体检测的系统和方法
CN106056101A (zh) * 2016-06-29 2016-10-26 哈尔滨理工大学 用于人脸检测的非极大值抑制方法
CN107463912A (zh) * 2017-08-10 2017-12-12 武汉大学深圳研究院 基于运动显著性的视频人体行为识别方法

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
SANNIANYIMENG, NON- MAXIMUM SUPPRESSION (NMS), 14 December 2017 (2017-12-14), pages 1 - 3, XP055720605, Retrieved from the Internet <URL:www.cnblogs.com/king-lps/p/8040458.html> *
See also references of EP3736765A4

Also Published As

Publication number Publication date
US11823303B2 (en) 2023-11-21
EP3736765A1 (en) 2020-11-11
US20200364289A1 (en) 2020-11-19
CN110060195A (zh) 2019-07-26
EP3736765A4 (en) 2021-03-10
CN110060195B (zh) 2021-05-04

Similar Documents

Publication Publication Date Title
Lu et al. Optimizing depthwise separable convolution operations on gpus
US12406526B2 (en) Indirectly accessing sample data to perform multi-convolution operations in a parallel processing system
US10140251B2 (en) Processor and method for executing matrix multiplication operation on processor
CN107315574B (zh) 一种用于执行矩阵乘运算的装置和方法
US9830156B2 (en) Temporal SIMT execution optimization through elimination of redundant operations
CN105453045B (zh) 使用动态宽度计算的壁垒同步
CN102640132B (zh) 用于访问与线程组相关联的断言信息的计算机实现的方法
CN110415157B (zh) 一种矩阵乘法的计算方法及装置
US7725518B1 (en) Work-efficient parallel prefix sum algorithm for graphics processing units
JP5883462B2 (ja) 範囲検出を行うための命令及びロジック
CN106846235B (zh) 一种利用NVIDIA Kepler GPU汇编指令加速的卷积优化方法及系统
US8615541B2 (en) Extended-precision integer arithmetic and logical instructions
US12579003B2 (en) Techniques for balancing workloads when parallelizing multiply-accumulate computations
US20220350863A1 (en) Technology to minimize the negative impact of cache conflicts caused by incompatible leading dimensions in matrix multiplication and convolution kernels without dimension padding
US11416261B2 (en) Group load register of a graph streaming processor
US10152310B2 (en) Fusing a sequence of operations through subdividing
US20120084539A1 (en) Method and sytem for predicate-controlled multi-function instructions
US11823303B2 (en) Data processing method and apparatus
US20140380317A1 (en) Single-pass parallel prefix scan with dynamic look back
CN112540793A (zh) 支持多访存模式的可重构处理单元阵列及控制方法、装置
WO2023206074A1 (zh) 一种对称矩阵与向量相乘的并行计算方法及其系统
US20230195651A1 (en) Host device performing near data processing function and accelerator system including the same
US6928605B2 (en) Add-compare-select accelerator using pre-compare-select-add operation
KR20230095795A (ko) Ndp 기능을 포함하는 호스트 장치 및 이를 포함하는 가속기 시스템
KR102644951B1 (ko) 산술 논리 장치 레지스터 시퀀싱

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19741008

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019741008

Country of ref document: EP

Effective date: 20200804