WO2016141803A1 - 图像识别加速器、终端设备及图像识别方法 - Google Patents

图像识别加速器、终端设备及图像识别方法 Download PDF

Info

Publication number
WO2016141803A1
WO2016141803A1 PCT/CN2016/074240 CN2016074240W WO2016141803A1 WO 2016141803 A1 WO2016141803 A1 WO 2016141803A1 CN 2016074240 W CN2016074240 W CN 2016074240W WO 2016141803 A1 WO2016141803 A1 WO 2016141803A1
Authority
WO
WIPO (PCT)
Prior art keywords
image recognition
parameter
adjusted
image data
success rate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/074240
Other languages
English (en)
French (fr)
Inventor
余浩
王雨豪
倪磊滨
杨伟
赵俊峰
肖世海
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Nanyang Technological University
Original Assignee
Huawei Technologies Co Ltd
Nanyang Technological University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd, Nanyang Technological University filed Critical Huawei Technologies Co Ltd
Priority to KR1020177024724A priority Critical patent/KR102057471B1/ko
Priority to JP2017546832A priority patent/JP6399534B2/ja
Priority to SG11201706525RA priority patent/SG11201706525RA/en
Priority to BR112017018752-3A priority patent/BR112017018752B1/pt
Priority to EP16761030.2A priority patent/EP3244345B1/en
Priority to CN201680012483.4A priority patent/CN107851175B/zh
Publication of WO2016141803A1 publication Critical patent/WO2016141803A1/zh
Priority to US15/695,681 priority patent/US10346701B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/94—Hardware or software architectures specially adapted for image or video understanding
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining

Definitions

  • the present invention relates to the field of computer technologies, and in particular, to an image recognition accelerator, a terminal device, and an image recognition method.
  • Image recognition technology is an important area of artificial intelligence.
  • Image recognition refers to the technique of processing and analyzing images using a computer to identify various targets and objects.
  • the demand for real-time image data analysis in mobile devices has gradually increased.
  • image data analysis consumes more system resources, the limited battery life of mobile devices limits the application of image data analysis on mobile devices.
  • an image data processing method in the prior art is to reduce the write current of image data into a static random access memory (SRAM). Reduce system power consumption.
  • SRAM static random access memory
  • the error rate of the data stored in the SRAM also increases.
  • the system power consumption is reduced when data is written, the computational complexity of the CPU in the image recovery process is high, and system resources are wasted.
  • the SRAM needs to remain powered. Therefore, the SRAM still has static power consumption, and the above image data processing method cannot completely eliminate the static power consumption required when the SRAM holds data. Therefore, in general, when the image data processing method is used to process image data, the system power consumption is still large.
  • An image recognition accelerator, a terminal device and an image recognition method provided in the embodiments of the present invention can ensure the accuracy of image recognition on the basis of reducing the system power consumption of the terminal device.
  • an embodiment of the present invention provides an image recognition accelerator for identifying an image in a terminal device, including:
  • a dimensionality reduction processing module configured to reduce a dimension of the first image data according to the set dimensionality reduction parameter ⁇ , wherein the reduced dimensional first image data includes a plurality of values;
  • a non-volatile memory NVM configured to store the low ⁇ bit of each value of the reduced dimensional first image data in the first storage area of the NVM according to the set first current I, and the dimension after the dimension reduction
  • the high (N- ⁇ ) bit of each value of an image data is stored in the second storage area of the NVM according to the set second current I s , where N is the bit occupied by each value, and ⁇ is the setting.
  • Width parameter, the first current I is smaller than the second current I s , the dimensionality reduction parameter ⁇ , the width parameter ⁇ , and the first current I are identified according to the system power consumption and the first image of the terminal device Success rate is obtained;
  • an image matching module configured to determine whether image data stored in the image library stored in the NVM includes image data that matches the reduced dimensional first image data.
  • the image recognition accelerator further includes: a parameter adjustment module, configured to: if the statistical image recognition success rate and the set second image recognition success rate Adjusting the success rate of the difference between the second image recognition success rate and the system power consumption of the terminal device according to the second image recognition success rate: the dimension reduction parameter ⁇ , the width parameter ⁇ , and the first current I, wherein the second image recognition success rate is different from the first image recognition success rate;
  • the dimension reduction processing module is further configured to reduce a dimension of the second image data according to the adjusted dimensionality reduction parameter ⁇ ';
  • the non-volatile memory NVM is further configured to store the low ⁇ ' bit of each value of the reduced dimensional second image data in the first storage area of the NVM according to the adjusted first current I′, The high (N- ⁇ ') bit of each value of the second image data after the dimension is stored in the second storage area of the NVM according to the second current I s , where ⁇ ' is the adjusted width parameter, The I' is smaller than the I s ;
  • the image matching module is further configured to determine whether image data stored in the image library stored in the NVM includes image data that matches the reduced dimensional second image data.
  • the dimension reduction processing module is specifically configured to:
  • the parameter adjustment module is specifically configured to:
  • the difference between the statistical image recognition success rate and the second image recognition success rate is greater than a preset threshold, respectively adjusting the dimension reduction parameter ⁇ , the width parameter ⁇ , or the value of the first current I to reduce the system
  • the power consumption E is obtained, and the adjusted image recognition success rate is respectively obtained, wherein the value of the E is proportional to the value of ⁇ ((N- ⁇ )*I s 2 + ⁇ *I);
  • an embodiment of the present invention provides a terminal device, where the terminal device includes a CPU and an image recognition accelerator, where the CPU is configured to send, to the image recognition accelerator, first image data to be identified;
  • the image recognition accelerator is configured to reduce the dimension of the first image data according to the set dimensionality reduction parameter ⁇ , wherein the dimensionally reduced first image data includes a plurality of values;
  • the high (N- ⁇ ) bit is stored in the second storage area of the NVM according to the set second current I s , where N is the bit occupied by each value, and ⁇ is the set width parameter, and the I Less than the I s , the dimensionality reduction parameter ⁇ , the width parameter ⁇ , and the first current I are obtained according to the system power consumption of the terminal device and the set first image recognition success rate;
  • the image recognition accelerator is further configured to: if a difference between a statistical image recognition success rate and a set second image recognition success rate The absolute value is greater than the preset threshold, and the at least one parameter is adjusted according to the second image recognition success rate and the system power consumption of the terminal device: a dimension reduction parameter ⁇ , a width parameter ⁇ , and a first current I, where The second image recognition success rate is different from the first image recognition success rate;
  • the CPU is further configured to send second image data to the image recognition accelerator
  • the image recognition accelerator is further configured to:
  • the low ⁇ ' bit of each value of the reduced dimensional second image data is stored in the first storage area of the NVM according to the adjusted first current I′, and each value of the second image data after the dimensionality reduction is The high (N- ⁇ ') bit is stored in the second storage area of the NVM according to the second current I s , wherein ⁇ ′ is an adjusted width parameter, and the I′ is smaller than the I s;
  • the CPU is further used for Calculating a matching result of the image recognition accelerator output during a preset statistical period, obtaining a statistical image recognition success rate; determining a difference between the statistical image recognition success rate and the set second image recognition success rate The absolute value is greater than the preset threshold;
  • the image recognition accelerator is further configured to adjust at least one parameter according to the second image recognition success rate and the system power consumption of the terminal device: a dimension reduction parameter ⁇ , a width parameter ⁇ , and a first current I, wherein The second image recognition success rate is different from the first image recognition success rate;
  • the CPU is further configured to send second image data to the image recognition accelerator
  • the image recognition accelerator is further configured to reduce a dimension of the second image data according to the adjusted dimensionality reduction parameter ⁇ ';
  • the low ⁇ ' bit of each value of the reduced dimensional second image data is stored in the first storage area of the NVM according to the adjusted first current I′, and each value of the second image data after the dimensionality reduction is The high (N- ⁇ ') bit is stored in the second storage area of the NVM according to the second current I s , wherein the ⁇ ′ is an adjusted width parameter, and the I′ is smaller than the I s ;
  • the CPU is further configured to:
  • the second image recognition success rate Determining at least one parameter: a dimensionality reduction parameter ⁇ , a width parameter ⁇ , and a first current I, wherein the second image recognition success rate is different from the first image recognition success rate;
  • the image recognition accelerator is further configured to:
  • the low ⁇ ' bit of each value of the reduced dimensional second image data is stored in the first storage area of the NVM according to the adjusted first current I′, and each value of the second image data after the dimensionality reduction is The high (N- ⁇ ') bit is stored in the second storage area of the NVM according to the second current I s , wherein ⁇ ′ is an adjusted width parameter, and the I′ is smaller than the I s;
  • the image recognition accelerator is specifically configured to:
  • the image recognition accelerator is specifically configured to:
  • the CPU is specifically configured to:
  • an embodiment of the present invention provides an image recognition method applied to a terminal device, where the method is performed by an image recognition accelerator in the terminal device, and the method includes:
  • the low ⁇ bit of each value of the dimensionally reduced first image data according to the set first current I and the dimension reduction
  • the high (N- ⁇ ) bit of each value of an image data is stored in the second storage area of the NVM according to the set second current I s , where N is the bit occupied by each value, and ⁇ is the setting.
  • Width parameter, the I is smaller than the I s , the dimensionality reduction parameter ⁇ , the width parameter ⁇ , and the first current I are obtained according to the system power consumption of the terminal device and the set first image recognition success rate;
  • the method further includes:
  • Adjusting at least one parameter according to the second image recognition success rate and the system power consumption of the terminal device a dimensionality reduction parameter ⁇ , a width parameter ⁇ , and a first current I, wherein the second image recognition success rate is The first image recognition success rate is different;
  • the low ⁇ ' bit of each value of the reduced dimensional second image data is stored in the first storage area of the NVM according to the adjusted first current I′, and each value of the second image data after the dimensionality reduction is The high (N- ⁇ ') bit is stored in the second storage area of the NVM according to the second current I s , wherein ⁇ ′ is an adjusted width parameter, and the I′ is smaller than the I s ;
  • the reducing the dimension of the first image data according to the set dimensionality reduction parameter ⁇ includes:
  • the determining success rate according to the second image And adjusting, by the system power consumption of the terminal device, at least one parameter: a dimension reduction parameter ⁇ , a width parameter ⁇ , and a first current I, including:
  • an embodiment of the present invention provides a computer program product, comprising: a computer readable storage medium storing program code, the program code comprising instructions for performing the method described in the foregoing third aspect.
  • the present application provides yet another image recognition accelerator for identifying an image in a terminal device.
  • the image recognition accelerator includes a dimensionality reduction processing module, a non-volatile memory NVM, and an image matching module.
  • the dimension reduction processing module is configured to receive the dimension reduction parameter ⁇ , and reduce the dimension of the first image data according to the received dimension reduction parameter ⁇ , wherein the dimension reduction first image data includes a plurality of values, and the dimension reduction parameter ⁇ It is obtained according to the system power consumption of the terminal device and the set first image recognition success rate.
  • the non-volatile memory NVM is configured to receive the width parameter ⁇ and the first current I, and obtain the storage bit number S according to the received width parameter ⁇ , and then lower the value of each of the dimensionally reduced first image data.
  • the S bit is stored in the first storage area of the NVM according to the set first current I
  • the high (NS) bit of each value of the dimensionally reduced first image data is stored in the second current I s according to the set.
  • a second storage area of the NVM wherein N is a bit occupied by each value, the first current I is smaller than the second current I s , and the width parameter ⁇ and the first current I are based on The system power consumption of the terminal device and the set first image recognition success rate are obtained.
  • the image matching module is configured to determine whether image data stored in the image library stored in the NVM includes image data that matches the reduced dimensional first image data.
  • the image recognition accelerator further includes a parameter adjustment module.
  • the parameter adjustment module is configured to adjust at least one parameter according to the set first image recognition success rate and the system power consumption of the terminal device: a dimension reduction parameter, a width parameter, and a value of the first current to obtain an adjustment
  • the subsequent dimensionality reduction parameter ⁇ , the width parameter ⁇ , and the first current I, and the adjusted dimensionality reduction parameter ⁇ is sent to the dimensionality reduction processing module, and the adjusted width parameter ⁇ and the adjusted The first current I is sent to the NVM.
  • the parameter adjustment module is specifically configured to: separately adjust the dimension reduction parameter, the width parameter, or the value of the first current And respectively obtaining a plurality of adjusted image recognition success rates and a plurality of adjusted system power consumptions, each adjusted image recognition success rate corresponding to each adjusted system power consumption; determining each image after adjustment Determining a difference between the success rate and the first image recognition success rate, and selecting at least one adjusted corresponding to the at least one adjusted image recognition success rate that the absolute value of the difference is not greater than the preset threshold Minimum system power consumption in system power consumption; selecting a dimension reduction parameter, a width parameter, and a value of the first current that obtain a maximum image recognition success rate when the minimum system power consumption is satisfied as the adjusted a dimension reduction parameter ⁇ , the width parameter ⁇ , and the first current I, and transmitting the adjusted dimensionality reduction parameter ⁇ to the dimensionality reduction processing module, and the adjusted width parameter ⁇ and the A first current I is sent to the N
  • An image recognition accelerator applied to a terminal device for performing image recognition according to an embodiment of the present invention includes a dimensionality reduction processing module, an NVM, and an image matching module.
  • the dimension of the first image data is first reduced by the dimensionality reduction processing module according to the set dimensionality reduction parameter ⁇ .
  • the NVM may write the low ⁇ bits of each value in the reduced dimensional first image data to the first storage area in the NVM according to the set first current I, and the respective values in the dimensionally reduced first image data.
  • the high N- ⁇ bit is written to the second memory region in the NVM in accordance with the set second current I s . Wherein the first current is less than the second current.
  • the matching module may determine whether image data matching the reduced dimensional first image data is included in the image library stored in the NVM to obtain an image recognition result of the first image data. Since the set dimension reduction parameter ⁇ , the width parameter ⁇ , and the first current I are both obtained according to the system power consumption of the terminal device and the set first image recognition success rate, it can be guaranteed to be stored in the value of the first storage area. The error of the lower part of the stored part in the storage process has less influence on the recognition success rate of the first image data.
  • the image recognition accelerator provided by the embodiment of the invention can ensure the accuracy of image recognition on the basis of reducing the system power consumption of the terminal device, and can improve the recognition speed of the image data.
  • FIG. 1 is a schematic structural diagram of a terminal device according to an embodiment of the present disclosure
  • FIG. 2 is a schematic structural diagram of another terminal device according to an embodiment of the present invention.
  • FIG. 3 is a schematic structural diagram of an image recognition accelerator according to an embodiment of the present invention.
  • FIG. 4 is a flowchart of an image recognition method according to an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of a dimension reduction processing module according to an embodiment of the present disclosure.
  • FIG. 6 is a schematic structural diagram of hardware of an NVM according to an embodiment of the present disclosure.
  • FIG. 7 is a schematic structural diagram of still another image recognition accelerator according to an embodiment of the present disclosure.
  • FIG. 8 is a flowchart of still another image recognition method according to an embodiment of the present invention.
  • FIG. 9 is a flowchart of a parameter adjustment method according to an embodiment of the present invention.
  • 10(a) and 10(b) are schematic diagrams showing recording parameters in a parameter adjustment process according to an embodiment of the present invention.
  • FIG. 11 is a schematic structural diagram of still another terminal device according to an embodiment of the present disclosure.
  • FIG. 12 is a schematic structural diagram of still another terminal device according to an embodiment of the present disclosure.
  • FIG. 13 is a signaling diagram of an image recognition method according to an embodiment of the present invention.
  • FIG. 1 is a schematic structural diagram of a terminal device according to an embodiment of the present invention.
  • a central processing unit (CPU) 10 and an image recognition accelerator 20 directly exchange data via a bus 15.
  • the bus 15 may be a system bus such as a PCI, a PCIE bus or an image acceleration interface AGP bus, and the type of the bus 15 is not limited in the embodiment of the present invention.
  • the terminal device 100 shown in FIG. 1 may be a terminal device such as a computer, a mobile phone, or a mobile terminal, and is not limited thereto, as long as it is a terminal device that needs to implement image recognition.
  • the CPU 10 is an operation core (Core) and a control unit (Control Unit) of the terminal device 100.
  • the CPU 10 can be a very large scale integrated circuit.
  • An operating system and other software programs are installed in the CPU 10, so that the CPU 10 can implement access to a storage space such as a memory or a cache.
  • the CPU 10 is only one example of the processor.
  • the processor may be an Application Specific Integrated Circuit (ASIC) or one or more integrated circuits configured to implement embodiments of the present invention.
  • ASIC Application Specific Integrated Circuit
  • the image recognition accelerator 20 is one of hardware accelerators.
  • the image recognition accelerator 20 is a hardware accelerator based on Non-Volatile Memory (NVM).
  • NVM Non-Volatile Memory
  • Hardware acceleration technology uses hardware modules instead of software algorithms to take advantage of the fast features inherent in hardware to increase the processing speed of computer systems.
  • memory is only used to store images. Data, all image data processing and analysis are done by the CPU. Therefore, the processing speed of the CPU and the transmission bandwidth of the memory become the bottleneck of the development of image recognition technology.
  • image data processing work is realized by a dedicated image recognition accelerator by adding a simple logic processing circuit in the memory. In the terminal device 100 shown in FIG.
  • the CPU 10 only needs to transmit the image data to be recognized to the image recognition accelerator 20 and accept the recognition result obtained by the image recognition accelerator 20, thereby reducing the burden on the CPU 10 and improving the terminal device identification.
  • the speed of the image since the system configuration shown in FIG. 1 reduces the amount of data transmitted between the CPU 10 and the image recognition accelerator 20, it is possible to solve the problem of limiting the speed of image recognition due to the transmission bandwidth of the memory.
  • FIG. 2 is a schematic structural diagram of another terminal device according to an embodiment of the present invention.
  • the terminal device 100 shown in FIG. 2 may include a CPU 10, an image recognition accelerator 20, and an image data collector 30.
  • the CPU 10 and the image data collector 30 are connected to the image recognition accelerator 20, respectively.
  • the image data collector 30 is configured to collect image data information, and send the collected image data information to the image recognition accelerator 20 for image recognition.
  • the image data collector 30 can collect images of people and objects, and the image information is not specifically limited herein. After the image data collector 30 acquires the image information, the acquired image information can be converted into image data.
  • the image data collector may include devices that implement functions such as photography or videography.
  • the image data collector can be a camera on a mobile phone.
  • the image recognition accelerator 20 is configured to identify the image data information transmitted from the image data collector 30 and the stored image data information, and transmit the recognition result to the CPU 10. It is to be understood that the description of the function and implementation of the CPU 10 and the image recognition accelerator 20 shown in FIG. 2 can be referred to the description of FIG. 1 above, and details are not described herein again.
  • the above description is only two schematic structures of the terminal device 100 in the embodiment of the present invention, and two application scenarios of the image recognition accelerator 20 are illustrated.
  • the image recognition accelerator 20 may further receive image data information transmitted by the CPU 10 for image recognition, and then transmit the image recognition result to other devices or devices.
  • image recognition accelerator 20 may also receive image data information transmitted by other devices (e.g., image data collector 30 in Fig. 2) and feed back image recognition results to the device.
  • the device in communication with the image recognition accelerator 20 is not limited herein. The specific structure and operation process of the image recognition accelerator 20 provided by the embodiment of the present invention will be described in detail below.
  • FIG. 3 is a schematic structural diagram of an image recognition accelerator 20 according to an embodiment of the present invention.
  • the structure of the image recognition accelerator 20 is illustrated in more detail.
  • the image recognition accelerator 20 may include a dimensionality reduction processing module 205, a nonvolatile memory NVM 210, and an image matching module 215.
  • both the dimensionality reduction processing module 205 and the image matching module 215 may be in the form of logic circuits. Exist, it can also exist in the form of an integrated circuit.
  • the image recognition accelerator 20 may be an application specific integrated circuit (ASIC) or a single board. In the embodiment of the present invention, the specific existence form of the image recognition accelerator 20 is not limited.
  • ASIC application specific integrated circuit
  • the image recognition accelerator 20 processes the first image data as an example for description.
  • the dimension reduction processing module 205 is configured to reduce the dimension of the image data according to the set dimension reduction parameter ⁇ . Specifically, as shown in FIG. 4, in step 400, the dimensionality reduction processing module 205 can reduce the dimension of the first image data according to the set dimensionality reduction parameter ⁇ .
  • the image data is a collection of gray values for each pixel represented by a numerical value.
  • image data is a discrete array obtained by sequentially extracting information for each pixel of an image, which may represent a contiguous image.
  • the first image data may be represented as a matrix of k rows * m columns, wherein each value in the matrix is used to represent a gray value of one pixel in the first image data.
  • the first image data is a set of gray values for each pixel of the first image represented by a numerical value.
  • the dimensionality reduction processing module 205 may reduce the dimension of the first image data by using a random mapping manner based on the sparse representation.
  • the compression algorithm implemented by the dimensionality reduction processing module 205 is not limited in the embodiment of the present invention, as long as it can implement a random mapping algorithm by sparse representation.
  • the dimensionality reduction processing module 205 can be implemented by using a matrix multiplier. Specifically, the dimensionality reduction processing module 205 may multiply the first image data by the set low-dimensional binary matrix by using a matrix multiplier, thereby achieving the purpose of reducing the dimension of the first image data.
  • the binary matrix means that all the values in the matrix are represented by 0 or 1.
  • the purpose of implementing dimensionality reduction using a binary matrix is to reduce the computational complexity in the dimension reduction process.
  • the set low-dimensional binary matrix may be a Bernoulli matrix, but the embodiment of the present invention does not limit the specific binary matrix form, as long as the binary matrix capable of achieving the dimension reduction purpose can be realized by means of sparse representation. can.
  • the first image data is a matrix X of k rows * m columns
  • the set binary matrix is a Bernoulli matrix Z of m rows * n columns, where k, m, and n are both positive integers, and m is greater than n.
  • the first image data can be multiplied by the set Bernoulli matrix Z by a matrix multiplier, so that a k-row*n-column matrix Y can be obtained, and the matrix Y is the dimensionally reduced first image data.
  • reducing the dimension of the X matrix is actually to reduce the number of columns of the X matrix.
  • the dimensionality reduction parameter ⁇ can also be referred to as a dimensionality reduction rate.
  • multiplier is one used to perform two mutually uncorrelated simulations.
  • An electronic device that acts by multiplying a signal or digital signal.
  • the multiplier can multiply two binary numbers.
  • a matrix multiplier is a device composed of a plurality of multipliers and adders for realizing the function of multiplying a matrix by a matrix. Since the calculations of the multipliers and adders of different columns in the matrix multiplier are independent of each other, parallel operations can be realized. Therefore, the dimension of the matrix can be adjusted by increasing or decreasing the number of columns of the multiplier and the adder in the matrix multiplier.
  • a circuit composed of a multiplier and an adder for realizing matrix multiplication is simply referred to as a multiplier.
  • the dimensionality reduction processing module 205 can achieve the purpose of reducing the dimension of the image data by reducing the multiplier of the partial columns in the matrix multiplier. Specifically, the multipliers of the partial columns can be reduced by turning off the power of the multipliers of the partial columns in the dimensionality reduction processing module 205.
  • FIG. 5 is a schematic structural diagram of a dimension reduction processing module 205 according to an embodiment of the present invention. As shown in FIG. 5, the dimension reduction module 205 includes m column multipliers, and the operations between the column multipliers are independent of each other. Each column multiplier controls whether the column multiplier operates by an independent switch.
  • switch S1 is used to control the first column multiplier
  • switch S2 is used to control the second column multiplier
  • switch Sm is used to control the mth column multiplier.
  • the switch can be implemented by a field effect transistor or a switching circuit.
  • the switch can be a junction field effect transistor (JFET) and a metal oxide semiconductor field effect transistor (metal-oxide semiconductor). FET, MOS-FET), here is not limited to the implementation of the switch.
  • the dimensionality reduction processing module 205 can receive the first image data X sent by the CPU 10 or the image data collector 30, where X is a matrix of k rows*m columns. It is assumed that the set Bernoulli matrix Z is a matrix of m rows * n columns.
  • An m-column multiplier can be provided in the dimensionality reduction processing module 205. In one cycle, a value in the first image data can be separately transferred to the m column multiplier of the matrix multiplier. Each column multiplier in the matrix multiplier can separately multiply the received value and one of the values in a row of Bernoulli matrix Z stored in the dimensionality reduction processing module 205, and output the calculation result.
  • each column multiplier can calculate a value in the X matrix and a value in the Z matrix in one cycle, and in a period, the m column multiplier can obtain the X matrix. This value is calculated as a result of a row of values in the Z matrix. It can be understood that, according to this manner, after m*k cycles, the operation result of the K line value and the Bernoulli matrix Z in the X matrix can be obtained.
  • the dimensionality reduction processing module 205 may obtain n according to the set dimensionality reduction parameter ⁇ and the value of m in the first image data.
  • the value, and according to the value of n obtained, turns off the switch in the matrix multiplier that controls the mn column multiplier.
  • the switch controlling the n+1th column to the mth column multiplier in the matrix multiplier can be turned off, so that the n+1th column to the mth column multiplier in the matrix multiplier are operated. No operations are performed during the process.
  • the dimensionality reduction processing module 205 can implement the multiplication operation of the X matrix and the Z matrix to obtain the first image data after the dimensionality reduction, wherein the first image data after the dimensionality reduction is represented by a Y matrix of k rows*n columns. Said.
  • a non-volatile memory (NVM) 210 is configured to store image data to be recognized and image data in a preset image library. Specifically, as shown in FIG. 4, in step 410, the NVM 210 may store the low ⁇ bits of each value of the dimensionally reduced first image data in the image recognition accelerator 20 according to the set first current I.
  • the first storage area 2104 of the NVM 210 stores the high (N- ⁇ ) bits of each value of the reduced dimensional first image data in the second storage area of the NVM 210 according to the set second current I s . 2106.
  • the dimension reduction parameter ⁇ , the width parameter ⁇ , and the first current I are obtained according to the system power consumption of the terminal device and the set first image recognition success rate.
  • the NVM 210 is a new generation of non-volatile memory.
  • the access speed of the NVM 210 is comparable to that of conventional volatile memory (eg, dynamic random access memory (DRAM) or static random access memory (SRAM).
  • DRAM dynamic random access memory
  • SRAM static random access memory
  • the NVM 210 has the reliability of a semiconductor product and has a long service life, enabling byte-by-byte addressing to write data in a storage medium in units of bits. Therefore, the NVM 210 can be hung on the memory bus as a memory directly accessed by the CPU 10. It should be noted that, unlike the conventional volatile memory, the NVM 210 is non-Volatile. When the terminal device 100 is powered off, the information in the NVM 210 still exists.
  • the NVM 210 may include a phase change memory (PCM), a resistive random access memory (RRAM), a magnetic random access memory (MRAM), and a ferroelectric randomization.
  • PCM phase change memory
  • RRAM resistive random access memory
  • MRAM magnetic random access memory
  • FRAM Feroelectric Random Access Memory
  • NVM next-generation non-volatile memory represented by a memory (Ferroelectric Random Access Memory, FRAM) or the like.
  • FRAM Ferroelectric Random Access Memory
  • the NVM 210 may be an STT-MRAM.
  • the NVM 210 may include an NVM controller 2102, a first storage area 2104, and a second storage area 2106.
  • the NVM controller 2102 is configured to access the first storage area 2104 and the second storage area 2106.
  • the NVM controller 2102 can write data to or read data from the first storage area 2104 and the second storage area 2106.
  • the NVM controller 2102 may include a processor, an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.
  • a cache, a communication interface, and the like may also be included in the NVM controller 2102, and the specific structure of the NVM controller 2012 is not limited herein.
  • the first storage area 2104 and the second storage area 2106 may be storage areas composed of a plurality of storage units.
  • the storage unit refers to a minimum storage medium unit for storing data, and each storage unit is for storing 1 bit of data.
  • the memory unit may include a nonvolatile memory unit such as a phase change memory unit, a magnetic memory unit, a resistive memory unit, or the like.
  • the first The storage area 2104 and the second storage area 2106 may be a storage array composed of a plurality of magnetic storage units. Those skilled in the art will appreciate that each magnetic memory cell includes two magnetic layers and one tunnel layer.
  • the electromagnetic direction of one magnetic layer is fixed, and the electromagnetic direction of the other magnetic layer can be changed by an external electromagnetic field.
  • the resistance of the magnetic memory cell is low for representing data "0"; when the directions of the two magnetic layers are opposite, the resistance of the magnetic memory cell is high for representing data " 1".
  • those skilled in the art will be able to change the magnetic layer of the electromagnetic direction by an external electromagnetic field as a free layer.
  • the direction of the magnetic field of the free layer can be altered by passing a spin polarization current through the magnetic memory cell.
  • the first storage area 2104 and the second storage area 2106 are not necessarily consecutive address spaces.
  • a storage space (not shown) for storing other data may be included, which is not limited herein.
  • the non-volatile memory basically has no static power consumption, but the energy cost caused by the read operation and the write operation to the non-volatile memory (also called dynamic work) The consumption is larger.
  • static power consumption refers to the energy overhead caused by not performing read and write operations on non-volatile memory.
  • the dynamic power consumption of the NVM can be controlled by controlling the magnitude of the write current during the write operation.
  • the NVM 210 uses image storage data in combination with different write currents.
  • the NVM controller 2102 can write the lower part and the upper part of each value in the first image data after the dimensionality reduction processing by the dimensionality reduction processing module 205 into the first storage area 2104 and the first by controlling the write current.
  • Two storage areas 2106. Specifically, in the embodiment of the present invention, the write current I of the first storage area 2104 is lower than the write current I s of the second storage area 2106.
  • the write current of the first storage region 2104 may be the first current I
  • the write current I s of the second storage region 2106 may be 2I.
  • the NVM controller 2102 can control the magnitude of the write current by controlling the write voltage.
  • FIG. 6 is a schematic structural diagram of hardware of an NVM 210 according to an embodiment of the present invention.
  • the first storage area 2104 and the second storage area 2106 include a storage array of a plurality of magnetic storage units 610.
  • NVM controller 2102 may control the first current I by controlling the first voltage V
  • NVM controller 2102 may control a second current I s by controlling the second voltage Vs.
  • the magnetic storage unit 610 of the same column can be connected to a multiplexer (MUX) 605.
  • MUX multiplexer
  • the NVM controller 2102 can control whether the multiplexer 605 outputs the first voltage V or the second voltage Vs through the control signal to achieve selection of the low ⁇ of each value in the first image data that is to be reduced by the first current I.
  • the bit is written to the first storage area 2104 or the high (N- ⁇ ) bits of each value are written to the second storage area 2106 by the second current I s .
  • N is the bit occupied by each value
  • ⁇ is the set width parameter.
  • the value of the image data to be identified is 64 bits
  • the lower 16 bits of the value may be written into the first storage area 2104 according to the first current I
  • the high 48 bits of the value may be written into the second according to the second current I s .
  • ⁇ is referred to as a width parameter.
  • the value of ⁇ and the value of the first current I need to be determined according to the system power consumption of the terminal device 100 and the set image recognition success rate. It can be understood that the type of the image data to be identified is different, and the requirements for the image recognition success rate are also different.
  • the set width parameter ⁇ and the value of the first current I are different, wherein the value of ⁇ is a positive integer.
  • FIG. 6 is merely a schematic illustration of a partial structure in the NVM 210 for illustrating how the NVM 210 in the image recognition accelerator 20 partitions image data.
  • the multiplexer MUX 605 may not directly connect to the magnetic memory unit 610, but instead write data to the magnetic memory unit 610 through a write device (not shown in FIG. 6) in the STT-MRAM.
  • one MUX 605 may be provided for the multi-column magnetic storage unit 610, or one MUX 605 may be provided for one or more rows of magnetic storage units 610.
  • the number of MUXs 605 and the connection relationship between the MUX 605 and the magnetic storage unit 610 are not limited as long as different portions of the numerical values in the image data can be written to the different magnetic storage units 610 in accordance with different currents.
  • the image matching module 215 is configured to determine whether the image database stored in the NVM includes image data that matches the reduced-dimensional first image data, and output a matching result. Specifically, in conjunction with FIG. 4, in step 410, the image matching module 215 may determine whether image data stored in the image library stored in the NVM 210 includes image data that matches the dimensionally reduced first image data to obtain A result of matching the reduced dimensional first image data with image data in an image library stored in the NVM 210. For example, the image matching module 215 can read the reduced dimensional first image data from the first storage area 2104 and the second storage area 2106, respectively, and directly compare the reduced dimensional first image data with the image library stored in the NVM 210. The image data is matched to determine whether the first image data can be successfully identified.
  • the image matching module 215 may be a logic circuit or an ASIC chip.
  • the image matching module 215 may calculate the reduced-dimensional first image data and the image data in the image library according to a Matching Pursuits (MP) algorithm through a logic circuit or an ASIC chip, thereby determining the storage in the NVM. Whether image data matching the dimensionally reduced first image data is included in the image library to obtain a matching result.
  • the image data in the image library may also be a map stored in the NVM 210 through the same processing method as the first image data. Like data.
  • the embodiment of the present invention does not limit the specific implementation manner of the image matching module 215, as long as the matching process of the image data can be implemented.
  • the specific matching algorithm is not limited in the embodiment of the present invention.
  • an Orthogonal Matching Pursuit (OMP) algorithm may be used, or other matching algorithms may be used, and the image matching module 215 is not used here.
  • OMP Orthogonal Matching Pursuit
  • the matching algorithm is limited.
  • the matching result may be returned to the CPU or the matching result may be generated to other data processing modules, which is not limited herein.
  • the NVM 210 writes different portions of the respective values in the reduced dimensional first image data into the first storage area 2104 and the second storage area 2106 according to different currents, and the first current I Less than the second current Is, therefore, storing data in the first storage area 2104 according to the first current I is more system power consuming than storing data in the second storage area 2106 according to the second current Is.
  • the lower the write current the greater the chance that the stored data will be erroneous.
  • the recognition success rate of the image data will decrease.
  • the stored image data is usually restored by a recovery method such as deconvolution optimization, and then image recognition is performed.
  • the set width parameter ⁇ and the first current I are obtained according to the system power consumption of the terminal device 100 and the set first image recognition success rate, the values stored in the first storage area 2104 are caused.
  • the errors that occur in the lower part of the storage process during the storage process have less influence on the recognition success rate. Therefore, in the process of implementing image data matching, the image matching module 215 does not need to restore the image data for matching, and can directly directly reduce the first image data stored in the NVM 210 and the image in the image library. The data is matched.
  • the image recognition method provided by the embodiment of the invention can satisfy the set image recognition success rate while saving system power consumption, and ensure the accuracy of the stored image data.
  • the image recognition accelerator 20 provided in the embodiment of the present invention can also be set.
  • FIG. 7 is a schematic structural diagram of still another image recognition accelerator 20 according to an embodiment of the present invention.
  • the parameter adjustment module 220 is connected to the dimensionality reduction processing module 205 and the NVM 210, respectively.
  • the statistics module 225 is connected to the matching module 215 and the parameter adjustment module 220, respectively.
  • the statistics module 225 is configured to count the matching results output by the image matching module 215 during the preset statistical period to obtain a statistical image recognition success rate.
  • the parameter adjustment module 220 can recognize the image according to the statistical module 225.
  • the success rate and the set second image recognition success rate determine whether the image recognition parameter needs to be adjusted.
  • the statistics module 225 can count the matching result output by the image matching module 215 during the preset statistical period, and obtain the statistical image recognition success rate. It can be understood that the image recognition success rate obtained by the statistics module 225 is obtained according to the recognition result of the plurality of image data.
  • the statistic module 225 can be a device such as a counter, and the specific implementation form of the statistic module 225 is not limited herein.
  • FIG. 7 is only a schematic diagram of a structure of the statistic module 225.
  • the statistic module 225 may be separately disposed in the terminal device 100, or the statistic module 225 may be disposed in the CPU 10.
  • the statistic module 225 is disposed in another device that is connected to the matching module 215.
  • the specific location set by the statistic module 225 is not limited in the embodiment of the present invention.
  • the parameter adjustment module 220 is configured to: if the absolute value of the difference between the statistical image recognition success rate and the set second image recognition success rate is greater than a preset threshold, identify the success rate according to the second image, and the terminal
  • the system power consumption of the device is adjusted by at least one of the following parameters: a dimensionality reduction parameter ⁇ , a width parameter ⁇ , and a first current I.
  • the dimension reduction parameter ⁇ , the width parameter ⁇ , and the first current I may be collectively referred to as an image recognition parameter.
  • the parameter adjustment module 220 may determine whether the value of the image recognition parameter needs to be adjusted according to the absolute value of the difference between the image recognition success rate and the set second image recognition success rate calculated by the statistics module 225.
  • the second image recognition success rate is a reset image recognition success rate, and the second image recognition success rate is different from the foregoing first image recognition success rate. It can be understood that the second image recognition success rate can be obtained from the CPU 10 in advance. As shown in FIG.
  • the parameter adjustment module 220 may adjust at least one parameter according to the second image recognition success rate and the system power consumption of the terminal device: a dimension reduction parameter ⁇ , a width parameter ⁇ , and a first current I.
  • FIG. 9 is a flowchart of a parameter adjustment method according to an embodiment of the present invention.
  • the image recognition success rate is adjusted from the first image recognition success rate to the second image recognition success rate.
  • the parameter adjustment method may include the following steps.
  • the parameter adjustment module 220 gradually adjusts the values of the dimensionality reduction parameter ⁇ , the width parameter ⁇ , or the first current I to reduce the system power consumption E of the terminal device, and obtains adjustments through the statistics module 225, respectively.
  • the subsequent image recognition success rate As can be understood from the foregoing description, the smaller the value of the first current I, the smaller the dynamic power consumption when the terminal device 100 stores image data, and the system power consumption of the terminal device 100 is smaller. As the value of the width parameter ⁇ increases, the more data stored in the first storage area 2104 in accordance with the first current I, the smaller the system power consumption of the terminal device 100.
  • the value of the system power consumption E of the terminal device is proportional to the value of ⁇ ((N- ⁇ )*I s 2 + ⁇ *I), where I s is the set standard write current, or may be referred to as The safe current ensures the accuracy of the image data when data is written to the NVM 210 in accordance with I s .
  • the value of I s can be obtained according to the physical parameters of the NVM 210, and the I s of the NVM 210 does not change during the process of using the NVM 210.
  • NVM 210 manufactured by different processes may have different physical parameters, so I s may also be different.
  • the second image data is a set of gray values of respective pixels of the second image represented by numerical values, and the second image data may include a plurality of numerical values.
  • the parameter adjustment module 220 may reduce the terminal device 100 by gradually increasing the value of the width parameter ⁇ or gradually decreasing the dimension reduction parameter ⁇ or gradually decreasing the value of the first current I.
  • the form of the record may be in the form of a table as shown in FIG. 10(a), or in the form of a figure as shown in FIG.
  • the image recognition success rate may also be referred to as a quality of service QoS. It can be understood that the image recognition success rate can be obtained by experiments in which a plurality of image data are identified by the determined image recognition parameters.
  • the system power consumption can be calculated according to the formula of ⁇ ((N- ⁇ )*I s 2 + ⁇ *I). It can be understood that the value of the system power consumption can be an estimated value.
  • the image recognition accelerator shown in FIG. 3 can identify multiple experimental data, Obtaining a recognition success rate of the plurality of experimental data according to the adjusted dimensionality reduction parameter ⁇ , the width parameter ⁇ or the value of the first current I, and calculating according to ⁇ ((N ⁇ )*I s 2 + ⁇ *I)
  • the system power consumption after each parameter adjustment is obtained to obtain the values of the plurality of sets of parameter values shown in FIG. 10(a) and the corresponding system power consumption and image recognition success rate.
  • the experimental data is also image data.
  • the parameter values adjusted during the adjustment process are exemplified by ⁇ 3, ⁇ 3, and I3 in Fig. 10(a).
  • the parameter adjustment module 220 may be based on ⁇ ((N- ⁇ )* The formula calculation of I s 2 + ⁇ *I) obtains the system power consumption E5 corresponding to the first set of parameter values.
  • the parameter adjustment module 220 can transmit the adjusted parameter value ⁇ 3 to the dimensionality reduction processing module 205, and send ⁇ 3 and I3 to the NVM 210.
  • the dimensionality reduction processing module 205, the NVM 210, and the image recognition module respectively identify the experimental data according to the adjusted parameter values ⁇ 3, ⁇ 3, and I3 according to the method shown in FIG. 4 to obtain corresponding system power consumption and image recognition success rate. Specifically, the dimensionality reduction processing module 205 performs dimensionality reduction processing on the experimental data according to the received dimensionality reduction parameter value ⁇ 3.
  • the NVM 210 stores the low ⁇ 3 bits in the dimensionally reduced experimental data in the first storage area 2104 according to I3, and stores the high (N- ⁇ 3) bits in the dimensionally reduced experimental data in the second storage area according to Is. 2106.
  • the image matching module 215 can read the dimensionally reduced experimental data from the first storage area 2104 and the second storage area 2106, respectively, and directly match the reduced-dimensional experimental data with the image data in the image library stored in the NVM 210. To determine whether the experimental data can be successfully identified. In this way, after the plurality of experimental data are identified in accordance with ⁇ 3, ⁇ 3, and I3, the image recognition success rate Qos5 corresponding to the set of parameter values can be obtained. If the Qos5 does not meet the requirement of the set second image recognition success rate, the value of the dimension reduction parameter ⁇ , the width parameter ⁇ or the first current I may be continuously adjusted, and according to the adjusted parameter value, the experimental data is according to FIG. 4 The method shown is identified.
  • the image recognition success rate and the system power consumption after each adjustment of the parameter values can be obtained in this way. For example, according to this manner, a plurality of sets of parameter values as shown in FIG. 10(a) and corresponding system power consumption and image recognition success rate can be obtained.
  • the value of the width parameter ⁇ is an integer
  • the value of the width parameter ⁇ can be preferentially adjusted, and the values of ⁇ and I are respectively adjusted according to the value of the adjusted width parameter ⁇ .
  • the value is such that the adjusted parameter value can satisfy the set image recognition success rate (for example, the second image recognition success rate) after the experimental data is recognized.
  • the embodiment of the present invention does not limit the specific adjustment order of the parameter values. It can be understood that, in the process of adjusting parameters, after adjusting the parameter values, a preset number of experimental data can be identified by the adjusted parameter values to obtain an image recognition success rate.
  • the recognition success rate of identifying a plurality of experimental data in the parameter adjustment process may be referred to as an adjusted recognition success rate.
  • an experimental library of image data may be preset, and experimental image data is stored in the experimental library for use as experimental data in the process of adjusting parameters.
  • ⁇ , ⁇ , I, E, and Qos of the header portion (the first row in FIG. 10(a)) in the list shown in FIG. 10(a) are used.
  • both ⁇ and ⁇ ' are used to represent the value of the width parameter
  • ⁇ and ⁇ ' are both used to represent the value of the dimensionality reduction parameter
  • I and I' are both A value used to represent the first current.
  • the parameter adjustment module 905 determines that the minimum function of the terminal device when the absolute value of the difference between the adjusted image recognition success rate and the set second image recognition success rate is not greater than the preset threshold. Consumption E'. It can be understood that in the process of adjusting parameters shown in step 900, a plurality of image recognition success rates corresponding to the adjusted parameters and a plurality of system power consumptions can be obtained. It can be understood by those skilled in the art that the smaller the value of the dimensionality reduction parameter ⁇ , the smaller the data amount of the dimensionally reduced image data, and the smaller the chance of error, but the amount of information contained in each value in the dimensionally reduced image data. Bigger. Therefore, in practical applications, there may be cases where the dimensionality reduction parameter ⁇ is reduced and the image recognition success rate is higher. Therefore, when selecting parameters, it is necessary to consider the trade-off between the dimensionality reduction parameter ⁇ and the image recognition success rate.
  • the image recognition success rate of the absolute value of the difference between the set second image recognition success rate and the preset threshold value may be regarded as the image recognition success that satisfies the requirement of the second image recognition success rate. rate. For example, if the second image recognition success rate is 90% and the preset threshold is 2%, the image recognition success rate between 88% and 92% can be considered as the image recognition success that satisfies the second image recognition success rate requirement. rate.
  • at least one image recognition success rate that satisfies the second image recognition success rate requirement may be determined among the recorded plurality of image recognition success rates. And determining a minimum system power consumption E' among the plurality of system power consumptions corresponding to the determined at least one image recognition success rate.
  • the parameter adjustment module 220 selects the dimensionality reduction parameter ⁇ , the width parameter ⁇ , and the first current I that obtain the maximum image recognition success rate when the minimum system power consumption E′ is satisfied as the adjusted The dimensionality reduction parameter ⁇ ', the width parameter ⁇ ', and the first current I'. It can be understood that there may be multiple image recognition success rates corresponding to the second image recognition success rate corresponding to the minimum system power consumption E′ determined in step 905. Therefore, in step 910, the parameter adjustment module 220 may select the dimension reduction parameter ⁇ , the width parameter ⁇ , and the first current I obtained as the adjusted drop when the minimum system power consumption E′ is satisfied. Dimension parameter ⁇ ', width parameter ⁇ ', and first current I'.
  • the width parameter ⁇ is increased by 1 bit, the obtained image recognition success rate is 88%, and the system power consumption E' is 10w.
  • the dimensionality reduction parameter ⁇ is reduced by 0.5, the obtained image recognition success rate is 90%, and the system power consumption E' is also 10w.
  • the current I is reduced by 500 ⁇ A, the obtained image recognition success rate is 92%, and the system power consumption E' is also 10 w.
  • the dimensionality reduction parameter ⁇ , the width parameter ⁇ , and the first current I in the third case may be used as the adjusted dimensionality reduction parameter ⁇ ′, the width parameter ⁇ ′, and the first current I′.
  • the embodiment of the present invention only performs a simple example of the process of adjusting the image recognition parameter by the parameter adjustment module 220.
  • the above three parameters may also be combined and adjusted, for example, the width parameter ⁇ may be simultaneously increased. 1 bit and reduce the dimensionality reduction parameter ⁇ by 0.5.
  • the specific adjustment form is not limited, as long as at least one of the above three image recognition parameters is adjusted.
  • the parameter adjustment module 220 may determine the adjusted dimensionality reduction parameter ⁇ ′, the width parameter ⁇ ′, and the first current I′ according to the greedy algorithm.
  • the statistics module 225 may also be located in the CPU 10 of the terminal device 100.
  • the parameter adjustment module 220 may adjust the image recognition parameter according to the instruction of the CPU 10.
  • FIG. 11 is a schematic structural diagram of still another terminal device according to an embodiment of the present invention.
  • the statistics module 225 (not shown in FIG. 11) may be located in the CPU 10, and the CPU 10 may count the image recognition accelerator 20 according to the matching result sent by the image matching module 215 during the preset statistical period. Image recognition success rate.
  • the CPU 10 may send a parameter adjustment instruction to the parameter adjustment module 220 to instruct the parameter adjustment module 220 to adjust the image recognition parameter.
  • the parameter adjustment instruction includes the second image recognition success rate.
  • the CPU 10 and the parameter adjustment module 220 jointly perform the function of adjusting the image recognition parameters. Specifically, the CPU 10 can perform the actions of steps 800-805 shown in FIG. 8 and instruct the parameter adjustment module 220 to perform the action of step 810.
  • FIG. 12 is a schematic structural diagram of still another terminal device according to an embodiment of the present invention.
  • FIG. 12 reduces the parameter adjustment module 220 on the basis of FIG. 11, and the function of the parameter adjustment module 220 in FIG. 11 is executed by the CPU 10.
  • the image matching module 215 can feed back the matching result to the CPU 10, and the CPU 10 can count the image of the image data recognized by the image recognition accelerator 20 during the preset statistical period according to the matching result. Identify the success rate.
  • the CPU 10 can determine whether the image recognition parameter needs to be adjusted according to the statistical image recognition success rate and the reset second image recognition success rate. When the CPU 10 determines that the absolute value of the difference between the statistical image recognition success rate and the set second image recognition success rate is greater than a preset threshold, the CPU 10 may identify the success rate according to the second image and the system power consumption of the terminal device. Adjusting at least one image recognition parameter: a dimension reduction parameter ⁇ , a width parameter ⁇ , and a first current I, and transmitting the adjusted dimensionality reduction parameter ⁇ ′, the width parameter ⁇ ′, and the first current I′ to the dimensionality reduction processing Module 205 and NVM 210. Alternatively, in the configuration shown in FIG. 12, the CPU 10 can execute the methods of steps 800-810 shown in FIG.
  • the adjusted dimensionality reduction parameter ⁇ can be transmitted to the dimensionality reduction processing module 205 and the NVM 210 in the image recognition accelerator 20 in the form of instructions. ', the width parameter ⁇ ' and the first current I', thereby controlling the dimensionality reduction processing module 205 and the NVM 210 to identify the image data in accordance with the adjusted image recognition parameters.
  • the image recognition accelerator provided by the embodiment of the present invention implements parameter adjustment.
  • the process is described, but the above example is only an example of a parameter adjustment function that can be implemented by the image recognition accelerator, thereby obtaining values of image recognition parameters (including a dimensionality reduction parameter, a width parameter, and a first current) that satisfy the condition.
  • image recognition parameters including a dimensionality reduction parameter, a width parameter, and a first current
  • the dimension reduction parameter ⁇ , the width parameter ⁇ , and the first current I used for identifying the first image data are also obtained according to the parameter adjustment manner.
  • the above parameter adjustment method is a description of how to adjust the parameter value to obtain an image recognition parameter value that satisfies the system power consumption and the image recognition success rate.
  • the parameter values satisfying the requirements can be obtained according to the above parameter adjustment method.
  • the image recognition accelerator 20 may follow the adjusted dimensionality reduction parameter ⁇ ′, the width parameter ⁇ ′, and the first current I′
  • the second image data that needs to be identified is subjected to image recognition. Specifically, as shown in FIG. 8, in step 815, the dimensionality reduction processing module 205 can reduce the dimension of the second image data according to the adjusted dimensionality reduction parameter ⁇ '.
  • the NVM 210 may store the low ⁇ ' bit of each value of the reduced dimensional second image data in the first storage area of the NVM according to the adjusted first current I′, and the dimension after the dimension reduction each of two high-value image data (N- ⁇ ') in accordance with said second bit current I s in the second storage region of the NVM 210, wherein the I' is less than the I s.
  • the image matching module 215 may determine whether image data stored in the image library stored in the NVM includes image data that matches the dimensionally reduced second image data.
  • the image matching module 215 may compare the reduced dimensional second image data with the image data in the image library stored in the NVM 210 to obtain the dimensionally reduced second image data and the NVM 210.
  • the image recognition accelerator performs image recognition on the second image data according to the adjusted dimensionality reduction parameter ⁇ ′, the width parameter ⁇ ′, and the first current I′, and the foregoing dimension reduction parameter ⁇ and width parameter ⁇ .
  • the process of the image recognition of the first image data by the first current I is similar. For details, refer to the foregoing description, and details are not described herein again.
  • the parameter adjustment method in the embodiment of the present invention triggers the adjustment parameter only when the parameter needs to be adjusted.
  • the image recognition accelerator may trigger to stop receiving the image data to be recognized (also referred to as service data), but adjust the parameter value in the manner shown in FIG. And the way in which the experimental data is identified to obtain the value of the image recognition parameter that satisfies the demand.
  • the parameter values satisfying the image recognition requirements are obtained, the parameter values satisfying the requirements are respectively sent to the dimensionality reduction processing module 205 and the NVM 210, so that the dimensionality reduction processing module 205, the NVM 210, and the image matching module 215 can obtain the images according to the adjustment.
  • the identification parameter values are image-recognized for image data to be recognized, such as first image data and second image data.
  • the operation of the image recognition accelerator 20 shown in FIG. 7 will be simplified in conjunction with the signaling diagram of the image recognition method shown in FIG. description of. In the embodiment of the present invention, the description is still made by taking the identification of the first image data as an example. As shown in Figure 13. After the dimension reduction processing module 205 receives the first image data 1300 to be identified sent by the CPU 10, the dimension reduction processing module 205 can reduce the dimension of the first image data 1300 according to the dimension reduction parameter ⁇ set by the parameter adjustment module 220.
  • the dimension reduction processing module 205 may perform dimensionality reduction on the first image data 1300 by using a Bernoulli matrix, so that the first image data 1300 can be dimensionally reduced based on the random mapping of the sparse representation.
  • the NVM 210 may select the broadband parameter ⁇ set by the parameter adjustment module 220 and the low ⁇ of each value of the first image data after the dimension reduction of the first current I.
  • the bits are stored in the first storage area 2104 according to the first current I, and the high N- ⁇ bits in each of the dimensionally reduced first image data are stored in the second storage area 2106 in accordance with the second current Is.
  • the image matching module 215 may identify the reduced-dimensional first image data 1305 based on the image data in the image library stored by the NVM 210, and determine whether the image library stored in the NVM includes the first dimension after the dimension reduction Image data Match the image data and output the matching result.
  • the image data in the image library and the reduced-dimensional first image data 1305 are collectively referred to as image data 1310 to be compared.
  • the image matching module 215 may output the recognition result of the first image data 1305 to the CPU 10.
  • the statistics module 225 can count the image matching result of the image matching module 215, thereby obtaining the image recognition success rate 1320 in the statistical period, and the parameter adjustment module 220 can thus obtain the image recognition success rate 1320 and the set according to the statistics module 225.
  • the second image recognition success rate determines whether the image recognition parameter needs to be adjusted.
  • the parameter adjustment module 220 may adjust the image recognition parameter according to the method shown in FIG. 9 and output the adjusted dimensionality reduction parameter ⁇ to the dimensionality reduction module 205 and the NVM 210, respectively. ', width parameter ⁇ ' and first current I'. Therefore, the dimensionality reduction processing module 205, the NVM 210, and the image matching module 215 can identify the subsequent second image data according to the adjusted dimensionality reduction parameter ⁇ ', the width parameter ⁇ ', and the first current I'.
  • FIG. 13 is only a schematic diagram of the signaling of the terminal device 100 provided by the embodiment of the present invention.
  • the image recognition accelerator 20 or the terminal device 100 provided by other embodiments refer to FIG. 13 and the foregoing embodiment. description of. I will not repeat them here.
  • the terminal device provided by the embodiment of the invention identifies the image by the image recognition accelerator, reduces the CPU data processing amount, reduces the data interaction between the CPU and the memory, reduces the burden on the CPU, and reduces the memory bandwidth for the image data recognition application. Limit and improve the recognition speed of image data.
  • the image recognition accelerator may reduce the dimension of the image data to be recognized according to the random mapping manner based on the sparse representation, and then write the image data after the dimension reduction according to different currents. Identify different storage areas in the accelerator's NVM. Since the set dimension reduction parameter ⁇ , the width parameter ⁇ , and the first current I are both obtained according to the system power consumption of the terminal device and the set image recognition success rate, the system power consumption of the terminal device can be reduced. Guarantee the accuracy of image recognition.
  • the embodiment of the invention further provides a computer program product for data processing, comprising a computer readable storage medium storing program code, the program code comprising instructions for executing the method flow described in any one of the foregoing method embodiments.
  • a person skilled in the art can understand that the foregoing storage medium includes: a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a random access memory (RAM), a solid state disk (SSD), or a nonvolatile.
  • a non-transitory machine readable medium that can store program code, such as a non-volatile memory.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Picture Signal Circuits (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

一种图像识别加速器(20)、终端设备及图像识别方法。图像识别加速器(20)包括了降维处理模块(205)、NVM(210)以及图像匹配模块(215)。在图像识别加速器(20)进行图像识别的过程中，先由降维处理模块(205)根据设置的降维参数γ降低第一图像数据的维度。NVM(210)将降维后的第一图像数据中的各个数值的低ω位按照设置的第一电流I写入NVM(210)中的第一存储区域(2104)，并将降维后的第一图像数据中的各个数值的高N-ω位按照设置的第二电流写入NVM (210) 中的第二存储区域(2106)。其中，第一电流小于第二电流。从而，匹配模块(215)可以确定所述NVM中存储的图像库中是否包含有与所述降维后的第一图像数据相匹配的图像数据。图像识别加速器(20)能够在降低终端设备的系统功耗的基础上保证图像识别的准确性。

Description

图像识别加速器、终端设备及图像识别方法 技术领域
本发明涉及计算机技术领域，尤其涉及一种图像识别加速器、终端设备及图像识别方法。
背景技术
图像识别技术是人工智能的一个重要领域。图像识别是指利用计算机对图像进行处理和分析，以识别各种不同目标和对像的技术。近年来，随着社交网络的普及，在移动设备中进行实时图像数据分析的需求逐渐增强。然而，由于实现图像数据分析会消耗较多的系统资源，因此，移动设备有限的电池寿命限制了图像数据分析在移动设备上的应用。
为了降低图像数据分析过程中的系统功耗，现有技术中的一种图像数据处理方法是通过降低图像数据写入静态随机存储器(Static Random-Access Memory,SRAM)中的写入电流的方式来降低系统功耗。然而，随着写入电流的降低，SRAM中存储的数据的错误率也随之上升。为了恢复错误，还需要通过解凸优化(convex optimization)处理等方式对存储的图像数据进行恢复，从而能够基于恢复后的图像数据进行图像识别。在这种方式中，虽然写入数据时系统功耗有所减少，然而，图像恢复过程中的CPU的计算复杂度高，也比较浪费系统资源。并且，为了保持存储于SRAM中的数据，SRAM需要保持通电状态，因此，SRAM还存在静态功耗，上述图像数据处理方式也并不能完全消除SRAM保持数据时所需的静态功耗。因此，总体上说，采用现有的图像数据处理方式处理图像数据时，系统功耗依然较大。
发明内容
本发明实施例中提供的一种图像识别加速器、终端设备及图像识别方法，能够在降低终端设备的系统功耗的基础上保证图像识别的准确性。
第一方面，本发明实施例提供了一种应用于终端设备中用于识别图像的图像识别加速器，包括：
降维处理模块，用于根据设置的降维参数γ降低第一图像数据的维度，其中，降维后的第一图像数据包括多个数值；
非易失性内存NVM，用于将降维后的第一图像数据的每一个数值的低ω位按照设置 的第一电流I存储于所述NVM的第一存储区域，将降维后的第一图像数据的每一个数值的高(N-ω)位按照设置的第二电流Is存储于所述NVM的第二存储区域，其中，N为每一个数值所占的比特位，ω为设置的宽度参数，所述第一电流I小于所述第二电流Is，所述降维参数γ、宽度参数ω以及第一电流I根据所述终端设备的系统功耗和设置的第一图像识别成功率获得；
图像匹配模块，用于确定所述NVM中存储的图像库中是否包含有与所述降维后的第一图像数据相匹配的图像数据。
结合第一方面，在第一方面的第一种可能的实现方式中，所述图像识别加速器还包括：参数调整模块，用于如果统计的图像识别成功率与设置的第二图像识别成功率之间的差值的绝对值大于预设阈值，则根据所述第二图像识别成功率以及所述终端设备的系统功耗调整下述至少一个参数：降维参数γ、宽度参数ω以及第一电流I，其中，所述第二图像识别成功率与所述第一图像识别成功率不同；
所述降维处理模块，还用于根据调整后的降维参数γ'降低第二图像数据的维度；
所述非易失性内存NVM，还用于将降维后的第二图像数据的每一个数值的低ω'位按照调整后的第一电流I'存储于NVM的第一存储区域，将降维后的第二图像数据的每一个数值的高(N-ω')位按照所述第二电流Is存储于所述NVM的第二存储区域，其中，ω'为调整后的宽度参数，所述I'小于所述Is；
所述图像匹配模块，还用于确定所述NVM中存储的图像库中是否包含有与所述降维后的第二图像数据相匹配的图像数据。
结合第一方面或第一方面的第一种实现方式，在第一方面的第二种可能的实现方式中，所述降维处理模块具体用于：
根据所述第一图像数据与设置的二进制矩阵的乘积获得所述降维后的第一图像数据，其中，所述第一图像数据为k行*m列的矩阵，所述二进制矩阵为m行*n列的矩阵，所述降维后的第一图像数据为k行*n列的矩阵，k、m和n为正整数，m的值大于n，n的值根据设置的降维参数γ确定，γ＝n/m。
结合第一方面的第一种或第二种可能的实现方式，在第一方面的第三种可能的实现方式中，所述参数调整模块具体用于：
如果统计的图像识别成功率与所述第二图像识别成功率之间的差值大于预设阈值，则分别调整降维参数γ、所述宽度参数ω或第一电流I的取值以降低系统功耗E，并分别获得调整后的图像识别成功率，其中，所述E的值与γ((N-ω)＊Is 2+ω*I)的值成正比；
确定在调整后的图像识别成功率与所述第二图像识别成功率之间的差值的绝对值不大于所述预设阈值时所述终端设备的最小功耗E'；
选择在满足所述最小功耗E'时获得最大图像识别成功率的降维参数γ、所述宽度参数ω以及第一电流I分别作为所述调整后的降维参数γ'、宽度参数ω'以及第一电流I'。
第二方面，本发明实施例提供了一种终端设备，所述终端设备包括CPU和图像识别加速器，其中，所述CPU，用于向所述图像识别加速器发送待识别的第一图像数据；
所述图像识别加速器，用于根据设置的降维参数γ降低所述第一图像数据的维度，其中，降维后的第一图像数据包括多个数值；
将所述降维后的第一图像数据的每一个数值的低ω位按照设置的第一电流I存储于NVM的第一存储区域，将所述降维后的第一图像数据的每一个数值的高(N-ω)位按照设置的第二电流Is存储于所述NVM的第二存储区域，其中，N为每一个数值所占的比特位，ω为设置的宽度参数，所述I小于所述Is，所述降维参数γ、宽度参数ω以及第一电流I根据所述终端设备的系统功耗和设置的第一图像识别成功率获得；
确定所述NVM中存储的图像库中是否包含有与所述降维后的第一图像数据相匹配的图像数据。
结合第二方面，在第二方面的第一种可能的实现方式中，所述图像识别加速器，还用于如果统计的图像识别成功率与设置的第二图像识别成功率之间的差值的绝对值大于预设阈值，则根据所述第二图像识别成功率以及所述终端设备的系统功耗调整下述至少一个参数：降维参数γ、宽度参数ω以及第一电流I，其中，所述第二图像识别成功率与所述第一图像识别成功率不同；
所述CPU，还用于向所述图像识别加速器发送第二图像数据；
所述图像识别加速器，还用于：
根据调整后的降维参数γ'降低第二图像数据的维度；
将降维后的第二图像数据的每一个数值的低ω'位按照调整后的第一电流I'存储于NVM的第一存储区域，将降维后的第二图像数据的每一个数值的高(N-ω')位按照所述第二电流Is存储于所述NVM的第二存储区域，其中，ω'为调整后的宽度参数，所述I'小于所述Is；
确定所述NVM中存储的图像库中是否包含有与所述降维后的第二图像数据相匹配的图像数据。
结合第二方面，在第二方面的第二种可能的实现方式中，所述CPU，还用于统 计在预设的统计期间内所述图像识别加速器输出的匹配结果，获取统计的图像识别成功率；确定所述统计的图像识别成功率与设置的第二图像识别成功率之间的差值的绝对值大于预设阈值；
所述图像识别加速器，还用于根据所述第二图像识别成功率以及所述终端设备的系统功耗调整下述至少一个参数：降维参数γ、宽度参数ω以及第一电流I，其中，所述第二图像识别成功率与所述第一图像识别成功率不同；
所述CPU，还用于向所述图像识别加速器发送第二图像数据；
所述图像识别加速器，还用于根据调整后的降维参数γ'降低第二图像数据的维度；
将降维后的第二图像数据的每一个数值的低ω'位按照调整后的第一电流I'存储于NVM的第一存储区域，将降维后的第二图像数据的每一个数值的高(N-ω')位按照所述第二电流Is存储于所述NVM的第二存储区域，其中，所述ω'为调整后的宽度参数，所述I'小于所述Is；
确定所述NVM中存储的图像库中是否包含有与所述降维后的第二图像数据相匹配的图像数据。
结合第二方面，在第二方面的第三种可能的实现方式中，所述CPU还用于：
统计在预设的统计期间内所述图像识别加速器输出的匹配结果，获取所述统计的图像识别成功率；
如果统计的图像识别成功率与设置的第二图像识别成功率之间的差值的绝对值大于预设阈值，则根据所述第二图像识别成功率以及所述终端设备的系统功耗调整下述至少一个参数：降维参数γ、宽度参数ω以及第一电流I，其中，所述第二图像识别成功率与所述第一图像识别成功率不同；
向所述图像识别加速器发送第二图像数据；
所述图像识别加速器，还用于：
根据调整后的降维参数γ'降低第二图像数据的维度；
将降维后的第二图像数据的每一个数值的低ω'位按照调整后的第一电流I'存储于NVM的第一存储区域，将降维后的第二图像数据的每一个数值的高(N-ω')位按照所述第二电流Is存储于所述NVM的第二存储区域，其中，ω'为调整后的宽度参数，所述I'小于所述Is；
确定所述NVM中存储的图像库中是否包含有与所述降维后的第二图像数据相匹配的图像数据。
结合第二方面或第二方面的第一种至第三种任意一种可能的实现方式中，在第二方面的第四种可能的实现方式中，所述图像识别加速器具体用于：
根据所述第一图像数据与设置的二进制矩阵的乘积获得所述降维后的第一图像数据，其中，所述第一图像数据为k行*m列的矩阵，所述二进制矩阵为m行*n列的矩阵，所述降维后的第一图像数据为k行*n列的矩阵，k、m和n为正整数，m的值大于n，n的值根据设置的降维参数γ确定，γ＝n/m。
结合第二方面的第一种或第二种可能的实现方式中，在第二方面的第五种可能的实现方式中，所述图像识别加速器具体用于：
分别调整降维参数γ、所述宽度参数ω或第一电流I的取值以降低所述终端设备的系统功耗E，并分别获得调整后的图像识别成功率，其中，所述E的值与γ((N-ω)＊Is 2+ω*I)的值成正比；
确定在调整后的图像识别成功率与所述第二图像识别成功率之间的差值的绝对值不大于所述预设阈值时所述终端设备的最小功耗E'；
选择在满足所述最小功耗E'时获得最大图像识别成功率的降维参数γ、所述宽度参数ω以及第一电流I分别作为所述调整后的降维参数γ'、宽度参数ω'以及第一电流I'。
结合第二方面的第三种可能的实现方式中，在第二方面的第六种可能的实现方式中，所述CPU具体用于：
分别调整降维参数γ、所述宽度参数ω或第一电流I的取值以降低所述终端设备的系统功耗E，并分别获得调整后的图像识别成功率，其中，所述E的值与γ((N-ω)＊Is 2+ω*I)的值成正比；
确定在调整后的图像识别成功率与所述第二图像识别成功率之间的差值的绝对值不大于所述预设阈值时所述终端设备的最小功耗E'；
选择在满足所述最小功耗E'时获得最大图像识别成功率的降维参数γ、所述宽度参数ω以及第一电流I分别作为所述调整后的降维参数γ'、宽度参数ω'以及第一电流I'。
第三方面，本发明实施例提供了一种应用于终端设备的图像识别方法，所述方法由所述终端设备中的图像识别加速器执行，所述方法包括：
根据设置的降维参数γ降低第一图像数据的维度，其中，降维后的第一图像数据包括多个数值；
将所述降维后的第一图像数据的每一个数值的低ω位按照设置的第一电流I存储于所述图像识别加速器中的NVM的第一存储区域，将所述降维后的第一图像数据的每一个 数值的高(N-ω)位按照设置的第二电流Is存储于所述NVM的第二存储区域，其中，N为每一个数值所占的比特位，ω为设置的宽度参数，所述I小于所述Is，所述降维参数γ、宽度参数ω以及第一电流I根据所述终端设备的系统功耗和设置的第一图像识别成功率获得；
确定所述NVM中存储的图像库中是否包含有与所述降维后的第一图像数据相匹配的图像数据。
结合第三方面，在第三方面第一种可能的实现方式中，所述方法还包括：
确定统计的图像识别成功率与设置的第二图像识别成功率的差值的绝对值大于预设阈值；
根据所述第二图像识别成功率以及所述终端设备的系统功耗调整下述至少一个参数：降维参数γ、宽度参数ω以及第一电流I，其中，所述第二图像识别成功率与所述第一图像识别成功率不同；
根据调整后的降维参数γ'降低第二图像数据的维度；
将降维后的第二图像数据的每一个数值的低ω'位按照调整后的第一电流I'存储于NVM的第一存储区域，将降维后的第二图像数据的每一个数值的高(N-ω')位按照所述第二电流Is存储于所述NVM的第二存储区域，其中，ω'为调整后的宽度参数，所述I'小于所述Is；
确定所述NVM中存储的图像库中是否包含有与所述降维后的第二图像数据相匹配的图像数据。
结合第三方面或第三方面的第一种可能的实现方式，在第三方面的第二种可能的实现方式中，所述根据设置的降维参数γ降低第一图像数据的维度包括：
根据所述第一图像数据与设置的二进制矩阵的乘积获得所述降维后的第一图像数据，其中，所述第一图像数据为k行*m列的矩阵，所述二进制矩阵为m行*n列的矩阵，所述降维后的第一图像数据为k行*n列的矩阵，k、m和n为正整数，m的值大于n，n的值根据设置的降维参数γ确定，γ＝n/m。
结合第三方面或第三方面的第一种至第二种中任意一种可能的实现方式，在第三方面的第三种可能的实现方式中，所述根据所述第二图像识别成功率以及所述终端设备的系统功耗调整下述至少一个参数：降维参数γ、宽度参数ω以及第一电流I，包括：
分别调整降维参数γ、所述宽度参数ω或第一电流I的取值以降低所述终端设备的系统功耗E，并分别获得调整后的图像识别成功率，其中，所述E的值与γ((N-ω)＊Is 2+ω*I)的值成正比；
确定在调整后的图像识别成功率与所述第二图像识别成功率之间的差值的绝对值不大于所述预设阈值时所述终端设备的最小功耗E'；
选择在满足所述最小功耗E'时获得最大图像识别成功率的降维参数γ、所述宽度参数ω以及第一电流I分别作为所述调整后的降维参数γ'、宽度参数ω'以及第一电流I'。
第四方面，本发明实施例提供了一种计算机程序产品，包括存储了程序代码的计算机可读存储介质，所述程序代码包括的指令用于执行前述第三方面中所述的方法。
第五方面，本申请提供了又一种应用于终端设备中用于识别图像的图像识别加速器。该图像识别加速器包括降维处理模块、非易失性内存NVM以及图像匹配模块。所述降维处理模块用于接收降维参数γ，根据接收的降维参数γ降低第一图像数据的维度，其中，降维后的第一图像数据包括多个数值，所述降维参数γ是根据所述终端设备的系统功耗和设置的第一图像识别成功率获得的。所述非易失性内存NVM用于接收宽度参数ω和第一电流I，并根据接收的宽度参数ω获得存储位数S，然后，将降维后的第一图像数据的每一个数值的低S位按照设置的第一电流I存储于所述NVM的第一存储区域，将降维后的第一图像数据的每一个数值的高(N-S)位按照设置的第二电流Is存储于所述NVM的第二存储区域.其中，N为每一个数值所占的比特位，所述第一电流I小于所述第二电流Is，所述宽度参数ω以及所述第一电流I是根据所述终端设备的系统功耗和设置的第一图像识别成功率获得的。所述图像匹配模块，用于确定所述NVM中存储的图像库中是否包含有与所述降维后的第一图像数据相匹配的图像数据。
结合第五方面，在一种可能的实现方式中，所述图像识别加速器还包括参数调整模块。所述参数调整模块用于根据设置的所述第一图像识别成功率以及所述终端设备的系统功耗调整下述至少一个参数：降维参数、宽度参数以及第一电流的值，以获得调整后的降维参数γ、宽度参数ω以及第一电流I，并将调整后的所述降维参数γ发送给所述降维处理模块，将调整后的所述宽度参数ω以及调整后的所述第一电流I发送给所述NVM。
结合上述第五方面以及可能的实现方式，在又一种可能的实现方式中，所述参数调整模块具体用于：分别调整所述降维参数、所述宽度参数或所述第一电流的值，并分别获得多个调整后的图像识别成功率和多个调整后的系统功耗，每个调整后的图像识别成功率对应于每个调整后的系统功耗；确定调整后的每个图像识别成功率与所述第一图像识别成功率之间的差值，选择所述差值的绝对值不大于所述预设阈值的至少一个调整后的图像识别成功率所对应的至少一个调整后的系统功耗中的最小系统功耗；选择在满足所述最小系统功耗时获得最大图像识别成功率的降维参数、宽度参数以及第一电流的值分别作为调整后的所述 降维参数γ、所述宽度参数ω以及所述第一电流I，并将调整后的所述降维参数γ发送给所述降维处理模块，将调整后的所述宽度参数ω以及所述第一电流I发送给所述NVM。
结合上述第五方面以及可能的实现方式，在又一种可能的实现方式中，所述降维处理模块具体用于：根据所述第一图像数据与设置的二进制矩阵的乘积获得所述降维后的第一图像数据，其中，所述第一图像数据为k行*m列的矩阵，所述二进制矩阵为m行*n列的矩阵，所述降维后的第一图像数据为k行*n列的矩阵，k、m和n为正整数，m的值大于n，n的值根据设置的降维参数γ确定，γ＝n/m。
本发明实施例提供的一种应用于终端设备中用于进行图像识别的图像识别加速器包括了降维处理模块、NVM以及图像匹配模块。在所述图像识别加速器对第一图像数据进行识别的过程中，先由降维处理模块根据设置的降维参数γ降低第一图像数据的维度。NVM可以将降维后的第一图像数据中的各个数值的低ω位按照设置的第一电流I写入NVM中的第一存储区域，并将降维后的第一图像数据中的各个数值的高N-ω位按照设置的第二电流Is写入NVM中的第二存储区域。其中，第一电流小于第二电流。从而，匹配模块可以确定所述NVM中存储的图像库中是否包含有与所述降维后的第一图像数据相匹配的图像数据，以获得第一图像数据的图像识别结果。由于设置的降维参数γ、宽度参数ω以及第一电流I均是根据所述终端设备的系统功耗和设置的第一图像识别成功率获得，因此能够保证存储于第一存储区域的数值中的低位部分在存储过程中出现的错误对第一图像数据的识别成功率的影响较小。本发明实施例提供的图像识别加速器能够在降低终端设备的系统功耗的基础上保证图像识别的准确性，并且能够提高图像数据的识别速度。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例中的附图。
图1为本发明实施例提供的一种终端设备的结构示意图；
图2为本发明实施例提供的另一种终端设备的结构示意图；
图3为本发明实施例提供的一种图像识别加速器的结构示意图；
图4为本发明实施例提供的一种图像识别方法的流程图；
图5为本发明实施例提供的一种降维处理模块的结构示意图；
图6为本发明实施例提供的一种NVM的硬件结构示意图；
图7为本发明实施例提供的又一种图像识别加速器的结构示意图；
图8为本发明实施例提供的又一种图像识别方法的流程图；
图9为本发明实施例提供的一种参数调整方法流程图；
图10(a)和图10(b)为本发明实施例提供的参数调整过程中记录参数的示意图；
图11为本发明实施例提供的又一种终端设备的结构示意图；
图12为本发明实施例提供的又一种终端设备结构示意图；
图13为本发明实施例提供的一种图像识别方法的信令图。
具体实施方式
为了使本技术领域的人员更好地理解本发明方案，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分的实施例，而不是全部的实施例。
本发明实施例提出了一种图像识别加速器，能够在降低系统功耗的基础上保证图像识别的准确性。图1为本发明实施例提供的一种终端设备的结构示意图。在图1所述的终端设备100中，中央处理器(Central Processing Unit，CPU)10与图像识别加速器20通过总线15直接交换数据。总线15可以为PCI、PCIE总线或图像加速接口AGP总线等系统总线，在本发明实施例中不对总线15的类型进行限定。需要说明的是，图1所示的终端设备100可以是计算机、手机、移动终端等终端设备，在此不做限定，只要是需要实现图像识别的终端设备即可。
如图1所示，CPU 10是终端设备100的运算核心(Core)和控制核心(Control Unit)。CPU 10可以是一块超大规模的集成电路。在CPU 10中安装有操作系统和其他软件程序，从而CPU 10能够实现对内存、缓存等存储空间的访问。可以理解的是，在本发明实施例中，CPU 10仅仅是处理器的一个示例。除了CPU 10外，处理器还可以是其他特定集成电路(Application Specific Integrated Circuit,ASIC)，或者是被配置成实施本发明实施例的一个或多个集成电路。
图像识别加速器20是硬件加速器(Hardware accelerator)中的一种。在本发明实施例中，图像识别加速器20是基于非易失内存(Non-Volatile Memory,NVM)的硬件加速器。硬件加速技术是利用硬件模块来替代软件算法，从而可以充分利用硬件所固有的快速特性，以提高计算机系统的处理速度。在传统的图像数据处理方法中，内存仅仅用于存储图像 数据，所有的图像数据处理、分析工作均由CPU完成，因此，CPU的处理速度以及内存的传输带宽成为图像识别技术发展的瓶颈。在本发明实施例中，通过在内存中增加简单的逻辑处理电路，以实现由专用的图像识别加速器来实现图像数据处理工作。在图1所示的终端设备100中，CPU 10只需要向图像识别加速器20发送待识别的图像数据并接受图像识别加速器20获得的识别结果，从而减少了CPU 10的负担，提高了终端设备识别图像的速度。并且，由于图1所示的系统结构减少了在CPU 10和图像识别加速器20的之间传输的数据量，从而可以解决因内存的传输带宽限制图像识别的速度的问题。
实际应用中，图像识别加速器20并不唯一与CPU 10进行数据交互。图2为本发明实施例提供的另一种终端设备的结构示意图。图2所示的终端设备100可以包括CPU10、图像识别加速器20以及图像数据采集器30。CPU 10和图像数据采集器30分别与图像识别加速器20连接。图像数据采集器30，用于采集图像数据信息，并将采集的图像数据信息发送给图像识别加速器20进行图像识别。本领域人员可以知道，图像数据采集器30可以采集人、物的图像，在此不对图像信息进行具体的限定。图像数据采集器30在采集到图像信息后，可以将采集的图像信息转换为图像数据。实际应用中，图像数据采集器可以包括实现照相或摄像等功能的器件。例如，图像数据采集器可以为手机上的摄像头。图像识别加速器20，用于对图像数据采集器30发送的图像数据信息与存储的图像数据信息进行识别，并将识别结果发送给CPU 10。可以理解的是，在图2中所示的CPU 10以及图像识别加速器20的功能及实现方式的描述可以参见上述图1的描述，在此不再赘述。
可以理解的是，上面描述的仅仅是本发明实施例中的终端设备100的两种示意性结构，说明了图像识别加速器20的两种应用场景。在又一种情形下，图像识别加速器20还可以接收CPU 10发送的图像数据信息进行图像识别后，将图像识别结果发送给其他器件或设备。在又一种情形下，图像识别加速器20还可以接收其他器件(例如图2中的图像数据采集器30)发送的图像数据信息，并向该器件反馈图像识别结果。在此不对与图像识别加速器20通信的器件进行限定。下面将对本发明实施例提供的图像识别加速器20的具体结构及操作过程进行详细描述。
图3为本发明实施例提供的一种图像识别加速器20的结构示意图，在图3中，对图像识别加速器20的结构进行了较为详细的图示。如图3所示，在本发明实施例中，图像识别加速器20可以包括降维处理模块205、非易失性存储器NVM 210、以及图像匹配模块215。需要说明的是，降维处理模块205以及图像匹配模块215均可以是逻辑电路的形式 存在，也可以是以集成电路的形式存在。实际应用中，图像识别加速器20可以是一种特定集成电路ASIC(Application Specific Integrated Circuit)或一种单板。在本发明实施例中，不对图像识别加速器20的具体存在形式进行限定。为了清楚的描述图像识别加速器20中各个器件的工作原理，下面将结合图4所示的图像识别方法的流程图对图3所示的图像识别加速器20中的各个器件的结构及工作过程进行详细的介绍。在下面的实施例中，以图像识别加速器20处理第一图像数据为例进行描述。
降维处理模块205，用于根据设置的降维参数γ降低图像数据的维度。具体的，如图4所示，在步骤400中，降维处理模块205可以根据设置的降维参数γ降低第一图像数据的维度。本领域技术人员可以知道，图像数据是用数值表示的各像素(pixel)的灰度值的集合。通常，图像数据是通过顺序地抽取图像的每一个像素的信息而获得的一个离散的阵列，该离散的阵列可以代表一副连续的图像。例如，第一图像数据可以表示为一个k行*m列的矩阵，其中该矩阵中的每一个数值用于代表第一图像数据中的一个像素的灰度值。换一种表达方式，第一图像数据是用数值表示的第一图像的各像素的灰度值的集合。为了提高图像识别的速度，在本发明实施例中降维处理模块205可以采用基于稀疏表示的随机映射方式降低第一图像数据的维度。在本发明实施例中并不对降维处理模块205实现的压缩算法进行限定，只要是能够实现通过稀疏表示的随机映射算法均可。
实际应用中，降维处理模块205可以采用矩阵乘法器来实现。具体的，降维处理模块205可以采用矩阵乘法器将第一图像数据与设置的低维的二进制矩阵相乘，从而实现降低第一图像数据的维度的目的。其中，二进制矩阵是指矩阵中的所有数值均为采用0或1表示的矩阵。采用二进制矩阵来实现降维的目的是为了减少降维过程中的计算复杂度。在本发明实施例中，设置的低维的二进制矩阵可以为伯努利矩阵，但本发明实施例不限定具体的二进制矩阵的形式，只要能够通过稀疏表示的方式实现降维目的的二进制矩阵即可。例如，第一图像数据为k行*m列的矩阵X，设置的二进制矩阵为m行*n列的伯努利矩阵Z，其中，k、m和n均为正整数，且m大于n。通过矩阵乘法器可以将第一图像数据与设置的伯努利矩阵Z相乘，从而可以获得一个k行*n列矩阵Y，矩阵Y即为降维后的第一图像数据。换一种表达方式，降低X矩阵的维度实际是为了降低X矩阵的列的数量。实际应用中，n的值可以根据m的值以及设置的降维参数γ来确定，其中，降维参数γ为降维后的第一图像数据的维度与第一图像数据的维度的比值，即γ＝n/m，则n＝m*γ。降维参数γ也可以被称为是降维率。
本领域人员可以知道，乘法器(multiplier)是一种用于完成两个互不相关的模拟 信号或数字信号相乘的作用的电子器件。乘法器可以将两个二进制数相乘。矩阵乘法器是由多个乘法器和加法器构成的用以实现矩阵与矩阵相乘的功能的器件。由于矩阵乘法器中不同列的乘法器和加法器的计算互不相关，能够实现并行运算，因此，可以通过增加或减少矩阵乘法器中乘法器及加法器的列数来调整矩阵的维数。为了描述方便，在本发明实施例中，将乘法器和加法器构成的用以实现矩阵乘法运算的电路简称为乘加器。
在本发明实施例中，降维处理模块205可以通过减少矩阵乘法器中的部分列的乘加器来实现降低图像数据的维度的目的。具体的，可以通过关闭降维处理模块205中的部分列的乘加器的电源来减少部分列的乘加器。图5为本发明实施例提供的一种降维处理模块205的结构示意图。如图5所示，降维模块205包括m列乘加器，各列乘加器之间的运算互相独立。各列乘加器通过独立的开关控制该列乘加器是否工作。例如，开关S1用于控制第1列乘加器，开关S2用于控制第2列乘加器，依次类推，开关Sm用于控制第m列乘加器。本领域技术人员可以知道，开关可以通过场效应晶体管或开关电路来实现，例如，开关可以为结型场效应管(junction field effect transistor,JFET)和金属氧化物半导体场效应管(metal-oxide semiconductor FET,MOS-FET)，在此不对开关的实现方式进行限制。
例如，在本发明实施例中，降维处理模块205可以接收CPU 10或图像数据采集器30发送的第一图像数据X，其中，X为k行*m列的矩阵。假设设置的伯努利矩阵Z为m行*n列的矩阵。降维处理模块205中可以设置m列乘加器。在一个周期内，可以将第一图像数据中的一个数值分别传输到矩阵乘法器的m列乘加器。矩阵乘加器中的各列乘加器可以分别将接收该数值与降维处理模块205中存储的伯努利矩阵Z中的一行数值中的一个数值进行乘法运算，并输出计算结果。换一种表达方式，每列乘加器一个周期内能够计算X矩阵中的一个数值与Z矩阵中的一个数值的计算结果，则在一个周期内，m列乘加器能够获得X矩阵中的该数值与Z矩阵中的一行数值的计算结果。可以理解的是，根据这种方式，经过m*k个循环后，可以获得X矩阵中的K行数值与伯努利矩阵Z的运算结果。在降低第一图像数据的维度的过程中，为了实现对第一图像数据的降维操作，降维处理模块205可以根据设定的降维参数γ以及第一图像数据中的m的值获得n的值，并根据获得的n的值关闭矩阵乘法器中控制m-n列乘加器的开关。例如，如图5所示，可以关闭矩阵乘法器中控制第n+1列至第m列乘加器的开关，使矩阵乘法器中的第n+1列至第m列乘加器在运算的过程中不进行运算。根据上述方式，降维处理模块205能够实现X矩阵和Z矩阵的乘法运算，获得降维后的第一图像数据，其中，降维后的第一图像数据以k行*n列的Y矩阵来表示。
非易失性内存(Non-Volatile Memory,NVM)210，用于存储待识别的图像数据 以及预设的图像库中的图像数据。具体的，如图4所示，在步骤410中，NVM 210可以将降维后的第一图像数据的每一个数值的低ω位按照设置的第一电流I存储于所述图像识别加速器20中的NVM 210的第一存储区域2104，将降维后的第一图像数据的每一个数值的高(N-ω)位按照设置的第二电流Is存储于所述NVM 210的第二存储区域2106。其中，所述降维参数γ、宽度参数ω以及第一电流I根据所述终端设备的系统功耗和设置的第一图像识别成功率获得。
在本发明实施例中，NVM 210是新一代的非易失性内存。NVM 210的存取速度与传统的易失性内存(例如，动态随机存取存储器DRAM或静态随机存取存储器SRAM)的存取速度相当。此外，NVM 210具有半导体产品的可靠性，使用寿命较长，能够实现按字节(Byte)寻址，将数据以位(bit)为单位写入存储介质中。因此，NVM 210能够被挂在内存总线上，作为内存被CPU 10直接访问。需要说明的是，NVM 210与传统的易失性内存不同，是非易失(Non-Volatile)的，当终端设备100关闭电源后，NVM 210中的信息依然存在。在本发明实施例中，NVM 210可以包括相变存储器(Phase Change Memory，PCM)、阻变存储器(Resistive Random Access Memory,RRAM)、磁性随机存储器(Magnetic Random Access Memory，MRAM)和铁电式随机存储器(Ferroelectric Random Access Memory，FRAM)等为代表的下一代非易失性存储器(Non-Volatile Memory,NVM)。具体的，由于自旋转移矩磁随机存取存储器(spin-transfer-torque magnetic RAM，STT-MRAM)的使用寿命较长且功耗较低，并且，由于STT-MRAM的写入成功率与写入电流关系较大，在本发明实施例中，NVM 210可以为STT-MRAM。
NVM 210可以包括NVM控制器2102、第一存储区域2104和第二存储区域2106。NVM控制器2102用于访问第一存储区域2104和第二存储区域2106。例如，NVM控制器2102可以将数据写入第一存储区域2104和第二存储区域2106，或者从第一存储区域2104和第二存储区域2106中读取数据。实际应用中，在NVM控制器2102中，可以包括处理器、特定集成电路(Application Specific Integrated Circuit，ASIC)或者被配置成实施本发明实施例的一个或多个集成电路。在NVM控制器2102中还可能包括缓存、通信接口等，在此不对NVM控制器2012的具体结构进行限定。
第一存储区域2104和第二存储区域2106可以是由多个存储单元构成的存储区域。在本发明实施例中，存储单元是指用于存储数据的最小存储介质单元，每个存储单元用于存储1比特(bit)的数据。例如，存储单元可以包括相变存储单元、磁性存储单元、阻变存储单元等非易失性存储单元。在本发明实施例中，以NVM 210为STT-MRAM为例，第一 存储区域2104和第二存储区域2106可以是由多个磁性存储单元构成的存储阵列。本领域技术人员可以知道，每个磁性存储单元包括两个磁性层和一个隧道层。其中，一个磁性层的电磁方向固定，另一磁性层的电磁方向可以通过外部的电磁场来改变。当两个磁性层的方向一样时，该磁性存储单元的电阻低，用于代表数据“0”；当两个磁性层的方向相反时，该磁性存储单元的电阻为高，用于代表数据“1”。通常，本领域技术人员将能够通过外部电磁场改变电磁方向的磁性层称为自由层。在本发明实施例中，可以通过将自旋偏振电流通过磁性存储单元来改变自由层的磁场方向。需要说明的是，在本发明实施例中，第一存储区域2104和第二存储区域2106并不一定是连续的地址空间。并且，在NVM 210中除了第一存储区域2104和第二存储区域2106外，还可以包括用于存储其他数据的存储空间(图中未示出)，在此不做限定。
本领域技术人员可以知道，与传统内存相比，非易失性内存基本不存在静态功耗，然而对非易失性内存执行读操作和写操作所造成的能量开销(也可称为动态功耗)较大。其中，静态功耗是指未对非易失性内存执行读操作和写操作期间所造成的能量开销。为了达到降低终端设备的系统功耗的目的，可以通过降低NVM的动态功耗来实现。具体的，可以通过控制写操作过程中的写入电流的大小来控制NVM的动态功耗。然而，本领域技术人员可以知道，在向磁性存储单元写数据的过程中，写入电流的强度要超过阈值电流才能保证磁性存储单元的电阻状态的翻转，因此，写入成功率与写入电流的大小也密切相关。在实现本发明的过程中，发明人发现，对于部分图像、视频等应用，数据中的低位部分在存储过程中出现的错误对识别成功率的影响较小。为了在降低写入功耗的同时不影响图像数据识别的成功率，在本发明实施例中，NVM 210采用了通过不同的写入电流结合存储的方式来存储图像数据。根据这种方式，NVM控制器2102可以通过控制写入电流将经过降维处理模块205降维处理后的第一图像数据中各个数值的低位部分和高位部分分别写入第一存储区域2104和第二存储区域2106。具体的，在本发明实施例中，第一存储区域2104的写入电流I低于第二存储区域2106的写入电流Is。例如，第一存储区域2104的写入电流可以为第一电流I，第二存储区域2106的写入电流Is可以为2I。本领域技术人员可以知道，NVM控制器2102可以通过控制写入电压来控制写入电流的大小。
图6为本发明实施例提供的一种NVM 210的硬件结构示意图。如图6所示，第一存储区域2104和第二存储区域2106包括多个磁性存储单元610构成的存储阵列。NVM控制器2102可以通过控制第一电压V来控制第一电流I，NVM控制器2102可以通过控制第二电压Vs来控制第二电流Is。同一列的磁性存储单元610可以连接一个多路选择器 (multiplexer,MUX)605。NVM控制器2102可以通过控制信号控制多路选择器605输出第一电压V还是输出第二电压Vs，以实现选择通过第一电流I将降维后的第一图像数据中的各数值的低ω位写入第一存储区域2104或通过第二电流Is将各数值的高(N-ω)位写入第二存储区域2106的目的。其中，N为每一个数值所占的比特位，ω为设置的宽度参数。例如，若待识别的图像数据的数值为64bit，则可以按照第一电流I将该数值的低16bit写入第一存储区域2104，按照第二电流Is将该数值的高48bit写入第二存储区域2106。为了描述方便，在本发明实施例中，将ω称为宽度参数。实际应用中，ω的值以及第一电流I的值均需要根据终端设备100的系统功耗以及设置的图像识别成功率来确定。可以理解的是，待识别的图像数据的类型不同，对图像识别成功率的要求也不相同，设置的宽度参数ω以及第一电流I的值也就不同，其中，ω的值为正整数。
可以理解的是，图6仅仅是为了阐述图像识别加速器20中的NVM 210如何将图像数据进行分区存储而对NVM 210中的部分结构做出的示意性图示。实际应用中，多路选择器MUX 605可能并不直接连接磁性存储单元610，而是通过STT-MRAM中的写装置(图6中未示出)将数据写入磁性存储单元610。此外，实际应用中，也可以为多列磁性存储单元610设置一个MUX 605，或者为一行或多行磁性存储单元610设置一个MUX 605。在此不对MUX 605的数量以及MUX 605与磁性存储单元610的连接关系进行限制，只要能够实现将图像数据中的数值的不同部分按照不同的电流分别写入不同的磁性存储单元610即可。
图像匹配模块215，用于确定所述NVM中存储的图像库中是否包含有与所述降维后的第一图像数据相匹配的图像数据，并输出匹配结果。具体的，结合图4，在步骤410中，图像匹配模块215可以确定所述NVM 210中存储的图像库中是否包含有与所述降维后的第一图像数据相匹配的图像数据，以获得所述降维后的第一图像数据与NVM 210中存储的图像库中的图像数据的匹配结果。例如，图像匹配模块215可以分别从第一存储区域2104和第二存储区域2106读取降维后的第一图像数据，并将降维后的第一图像数据直接与NVM210中存储的图像库中的图像数据进行匹配，以判断是否能够成功识别该第一图像数据。可以理解的是，为了识别图像，在NVM 210中需要预先存储包含有至少一个图像数据的图像库。在本发明实施例中，图像匹配模块215可以是逻辑电路或ASIC芯片。例如，图像匹配模块215可以通过逻辑电路或ASIC芯片将降维后的第一图像数据与图像库中的图像数据按照匹配追踪(Matching Pursuits，MP)算法进行计算，从而确定所述NVM中存储的图像库中是否包含有与所述降维后的第一图像数据相匹配的图像数据，以获得匹配结果。可以理解的是，图像库中的图像数据也可以是经过与第一图像数据相同的处理方法存储于NVM 210中的图 像数据。
需要说明的是，本发明实施例并不对图像匹配模块215的具体实现形式进行限制，只要能够实现图像数据的匹配过程即可。并且，本发明实施例也不对具体的匹配算法进行限定，实际应用中，可以采用正交匹配追踪(Orthogonal Matching Pursuit，OMP)算法，也可以采用其他匹配算法，在此，不对图像匹配模块215采用的匹配算法进行限定。实际应用中，匹配模块215获得匹配结果后，可以将匹配结果返回给CPU或者将匹配结果发生给其他数据处理模块，在此不进行限定。
在本发明实施例中，由于NVM 210将降维后的第一图像数据中的各个数值的不同部分按照不同的电流分别写入第一存储区域2104和第二存储区域2106，且第一电流I小于第二电流Is，因此，按照第一电流I将数据存储于第一存储区域2104与按照第二电流Is将数据存储于第二存储区域2106相比更节省系统功耗。本领域人员可以知道，通常，写入电流越低，存储的数据出现错误的机会将越大。或者换一种表达方式，随着写入电流的降低，图像数据的识别成功率将会降低。因此，现有技术中通常会先将存储的图像数据通过解凸优化等恢复方式进行恢复后再进行图像识别。在本发明实施例中，由于设置的宽度参数ω以及第一电流I是根据终端设备100的系统功耗和设置的第一图像识别成功率获得的，从而使得存储于第一存储区域2104的数值中的低位部分在存储过程中出现的错误对识别成功率的影响较小。因此，图像匹配模块215在实现图像数据匹配的过程中，并不需要将图像数据恢复后进行匹配，而可以直接将存储于NVM 210中的降维后的第一图像数据与图像库中的图像数据进行匹配。通过本发明实施例提供的这种图像识别方式，能够在节省系统功耗的情况下满足设置的图像识别成功率，保证存储的图像数据的准确性。
为了使终端设备100能够满足各种类型的图像数据的识别需求，并能够在节省系统功耗的情况下满足设置的图像识别成功率，在本发明实施例提供的图像识别加速器20中还可以设置有统计模块225以及参数调整模块220。如图7所示，图7为本发明实施例提供的又一种图像识别加速器20的结构示意图。如图7所示，在图3所示的结构基础上，参数调整模块220分别连接降维处理模块205以及NVM 210。统计模块225分别与匹配模块215以及参数调整模块220连接。下面将结合图8所示的又一种图像识别方法对图7所示的图像识别加速器20中的各个器件的结构和工作原理进行详细的介绍。
统计模块225用于统计在预设的统计期间内图像匹配模块215输出的匹配结果，以获得统计的图像识别成功率。从而参数调整模块220能够根据统计模块225统计的图像识 别成功率以及设置的第二图像识别成功率判断是否需要调整图像识别参数。具体的，如图8所示，在步骤800中，统计模块225可以统计在预设的统计期间内图像匹配模块215输出的匹配结果，获取所述统计的图像识别成功率。可以理解的是，统计模块225获得的图像识别成功率是根据多个图像数据的识别结果获得的。实际应用中，统计模块225可以是计数器等器件，在此不对统计模块225的具体实现形式进行限制。
可以理解的是，图7仅仅是对统计模块225的一种结构的示意，实际应用中，还可以将统计模块225单独设置于终端设备100中，也可以将统计模块225设置在CPU 10中，或者将统计模块225设置在与匹配模块215连接的其他设备中，本发明实施例不对统计模块225设置的具体位置进行限定。
参数调整模块220，用于如果统计的图像识别成功率与设置的第二图像识别成功率之间的差值的绝对值大于预设阈值，则根据所述第二图像识别成功率以及所述终端设备的系统功耗调整下述至少一个参数：降维参数γ、宽度参数ω以及第一电流I。为了描述方便，在本发明实施例中，可以将降维参数γ、宽度参数ω以及第一电流I统称为图像识别参数。具体的，参数调整模块220可以根据统计模块225统计的图像识别成功率与设置的第二图像识别成功率的差值的绝对值来判断是否需要调整图像识别参数的取值。其中，第二图像识别成功率为重新设置的图像识别成功率，第二图像识别成功率与前述的第一图像识别成功率不同。可以理解的是，第二图像识别成功率可以预先从CPU 10获得。结合图8所示，若在步骤805中，参数调整模块220确定统计的图像识别成功率与设置的第二图像识别成功率之间的差值的绝对值大于预设阈值，则在步骤810中，参数调整模块220可以根据所述第二图像识别成功率以及所述终端设备的系统功耗调整下述至少一个参数：降维参数γ、宽度参数ω以及第一电流I。
为了清楚阐述参数调整模块220具体如何调整图像识别参数，下面将结合图9所示的参数调整方法流程图对参数调整模块220如何平衡系统功耗和图像识别成功率，以获得合适的图像识别参数进行描述。图9为本发明实施例提供的一种参数调整方法流程图。在本发明实施例中，以需要将图像识别成功率从第一图像识别成功率调整为第二图像识别成功率为例进行描述。如图9所示，该参数调整方法可以包括下述步骤。
在步骤900中，参数调整模块220分别逐渐调整降维参数γ、所述宽度参数ω或第一电流I的取值以降低所述终端设备的系统功耗E，并分别通过统计模块225获得调整后的图像识别成功率。根据前面的描述可以知道，第一电流I的值越小，终端设备100存储图像数据时的动态功耗越小，则终端设备100的系统功耗越小。随着宽度参数ω的值增大，按 照第一电流I存储于第一存储区域2104的数据越多，则终端设备100的系统功耗越小。降维参数γ的值越小，降维后的图像数据的数据量越少，终端设备100的系统功耗也会越小。终端设备的系统功耗E的值与γ((N-ω)＊Is 2+ω*I)的值成正比，其中，Is为设置的标准写入电流，或者也可以被称为是安全电流，按照Is将数据写入NVM 210中时，可以保证图像数据的准确度。通常，Is的值可以根据NVM 210的物理参数获得，在使用NVM 210的过程中，NVM 210的Is不会改变。可以理解的是，不同工艺制造的NVM 210因其物理参数不同，因此Is也可能不同。所述第二图像数据是用数值表示的第二图像的各像素的灰度值的集合，所述第二图像数据可以包括多个数值。实际应用中，在调整图像识别参数的过程中，参数调整模块220可以分别通过逐渐增加宽度参数ω的值、或逐渐降低降维参数γ或逐渐第一电流I的值的方式降低终端设备100的系统功耗E，并记录下调整的过程中的各个参数的值、系统功耗以及获得的图像识别成功率。记录的形式可以是图10(a)所示的表格形式，也可以是如图10(b)所示的图形的形式，还可以是其他记录形式。其中，在图10(b)中仅对如何采用图表的形式记录ω和I的调整对图像识别成功率的值的改变进行了示例。可以看出，用图形的形式比用表格记录更加直观。在本发明实施例中，图像识别成功率也可以被称为服务质量QoS。可以理解的是，图像识别成功率可以采用确定的图像识别参数识别多个图像数据的试验来获得。系统功耗可以根据γ((N-ω)＊Is 2+ω*I)的公式计算获得，可以理解的是，系统功耗的值可以是一个估计值。
具体的，在调整参数的过程中，当调整了一次降维参数γ、所述宽度参数ω或第一电流I的值后，可以通过图3所示的图像识别加速器识别多个实验数据，以获得根据调整的降维参数γ、所述宽度参数ω或第一电流I的值识别多个实验数据的识别成功率，并根据γ((N-ω)＊Is 2+ω*I)计算获得每次调整参数后的系统功耗，以得到图10(a)所示的多组参数值以及对应的系统功耗和图像识别成功率的值。可以理解的是，在本发明实施例中，实验数据也为图像数据。例如，以调整过程中调整的参数值为图10(a)中的γ3、ω3和I3为例。参数调整模块220在将调整降维参数、所述宽度参数或第一电流的值调整为第一组参数值：γ3、ω3和I3后，参数调整模块220可以根据γ((N-ω)＊Is 2+ω*I)的公式计算获得与所述第一组参数值对应的系统功耗E5。并且，参数调整模块220可以将调整后的参数值γ3发送给降维处理模块205，将ω3和I3发送给NVM 210。降维处理模块205、NVM 210以及图像识别模块分别根据调整后的参数值γ3、ω3和I3按照图4所示的方法对实验数据进行识别，以得到相应的系统功耗和图像识别成功率。具体的，降维处理模块205根据接收的降维参数值γ3对实验数据进行降维处理。NVM 210根据I3将降维后的实验数据中的低ω3位存储在第一存 储区域2104中，并根据Is将降维后的实验数据中的高(N-ω3)位存储于第二存储区域2106中。图像匹配模块215可以分别从第一存储区域2104和第二存储区域2106读取降维后的实验数据，并将降维后的实验数据直接与NVM 210中存储的图像库中的图像数据进行匹配，以判断是否能够成功识别该实验数据。按照这种方式，按照γ3、ω3和I3对多个实验数据进行识别后，可以获得该组参数值对应的图像识别成功率Qos5。如果Qos5不满足设定的第二图像识别成功率的要求，可以继续调整降维参数γ、所述宽度参数ω或第一电流I的值，再根据调整后的参数值对实验数据按照图4所示的方法进行识别。从而在调整参数过程中，能够按照这种方式获得每次调整参数值后的图像识别成功率以及系统功耗。例如，根据这种方式可以得到如图10(a)所示的多组参数值以及对应的系统功耗和图像识别成功率。
实际应用中，由于宽度参数ω的值为整数，因此在调整过程中，为了方便调整，可以优先调整宽度参数ω的值，并以调整后的宽度参数ω的值为依据分别调整γ和I的值，以使调整后的参数值对实验数据识别后能够满足设定的图像识别成功率(例如第二图像识别成功率)的要求。本发明实施例不对参数值的具体调整顺序进行限定。可以理解的是，在调整参数的过程中，当调整参数值后，可以通过调整后的参数值对预设数量的实验数据进行识别以获得图像识别成功率。在本发明实施例中，可以将参数调整过程中识别多个实验数据的识别成功率称为调整后的识别成功率。可以理解的是，在本发明实施例中，可以预先设置图像数据的实验库，在实验库中存储有实验用的图像数据，以用于在调整参数过程中作为实验数据使用。需要说明的是，在本发明实施例中，除了图10(a)所示的列表中的表头部分(图10(a)中第一行)的ω、γ、I、E及Qos是用于表示参数外，表中除第一行之外的其他部分中的ω、γ、I、γ1、ω1、I2、E1、Qos1等均用于表示具体的参数值，本发明实施例中其他部分的ω、γ、I、γ'、ω'以及I'均用于表示具体的参数值。换一种表达方式，在本发明实施例中，如无特别说明，ω和ω'均用于表示宽度参数的值，γ和γ'均用于表示降维参数的值，I和I'均用于表示第一电流的值。
在步骤905中，参数调整模块905确定在调整后的图像识别成功率与设置的第二图像识别成功率之间的差值的绝对值不大于所述预设阈值时所述终端设备的最小功耗E'。可以理解的是，在步骤900所示的调整参数的过程中，可以获得与调整的参数对应的多个图像识别成功率以及多个系统功耗。本领域技术人员可以理解的是，降维参数γ的值越小，降维后的图像数据的数据量越小，出错机会更小，但降维后的图像数据中每个数值包含的信息量更大。因此，实际应用中，会存在降维参数γ减小而图像识别成功率更高的情况。从而，在选择参数时，需要考虑降维参数γ与图像识别成功率的折衷。
在本发明实施例中，可以将与设置的第二图像识别成功率的差值的绝对值不大于预设阈值的图像识别成功率都作为是满足第二图像识别成功率的要求的图像识别成功率。例如，若第二图像识别成功率为90％，预设阈值为2％，则在88％-92％之间的图像识别成功率均可以认为是满足第二图像识别成功率要求的图像识别成功率。在本步骤中，可以在记录的多个图像识别成功率中确定满足第二图像识别成功率要求的至少一个图像识别成功率。并在确定的至少一个图像识别成功率对应的多个系统功耗中确定最小的系统功耗E'。
在步骤910中，参数调整模块220选择在满足所述最小系统功耗E'时获得最大图像识别成功率的降维参数γ、所述宽度参数ω以及第一电流I分别作为所述调整后的降维参数γ'、宽度参数ω'以及第一电流I'。可以理解的是，在步骤905中确定的最小系统功耗E'对应的满足第二图像识别成功率要求的图像识别成功率可以有多个。因此，在步骤910中，参数调整模块220可以选择满足所述最小系统功耗E'时获得最大图像识别成功率的降维参数γ、所述宽度参数ω以及第一电流I作为调整后的降维参数γ'、宽度参数ω'以及第一电流I'。例如，在第一种情形下，宽度参数ω增加1bit，获得的图像识别成功率为88％，系统功耗E'为10w。在第二种情形下，降维参数γ减少0.5，获得的图像识别成功率为90％，系统功耗E'也为10w。在第三种情形下，电流I减少500μA，获得的图像识别成功率为92％，系统功耗E'也为10w。则可以将第三种情形下的降维参数γ、所述宽度参数ω以及第一电流I作为调整后的降维参数γ'、宽度参数ω'以及第一电流I'。
可以理解的是，本发明实施例仅仅对参数调整模块220调整图像识别参数的过程进行了一个简单的示例，实际应用中，还可以组合调整上述三个参数，例如，可以同时将宽度参数ω增加1bit并将降维参数γ减少0.5。在本发明实施例中不对具体的调整形式进行限定，只要调整上述三个图像识别参数中的至少一个参数即可。实际应用中，参数调整模块220可以根据贪心算法来确定调整后的降维参数γ'、宽度参数ω'以及第一电流I'。
需要说明的是，实际应用中，统计模块225还可以位于终端设备100的CPU 10中，则在这种情形下，参数调整模块220可以根据CPU10的指示调整图像识别参数。如图11所示，图11为本发明实施例提供的又一种终端设备的结构示意图。在图11所示的结构中，统计模块225(图11中未示出)可以位于CPU 10中，CPU 10可以根据图像匹配模块215在预设的统计期间发送的匹配结果统计图像识别加速器20的图像识别成功率。如果统计的图像识别成功率与所述第二图像识别成功率之间的差值大于预设阈值，则CPU 10可以向参数调整模块220发送参数调整指令，以指示参数调整模块220调整图像识别参数。所述参数调整指令中包含有所述第二图像识别成功率。换一种表达方式，在图11所示的结构中，由 CPU10和参数调整模块220共同完成调整图像识别参数的功能，具体的，可以由CPU 10执行图8中所示的步骤800-805的动作，并指示参数调整模块220执行步骤810的动作。
在又一种情形下，还可以由终端设备100中的CPU 10完成调整图像识别参数的功能。如图12所示，图12为本发明实施例提供的又一种终端设备结构示意图。图12在图11的基础上减少了参数调整模块220，图11中的参数调整模块220的功能改由CPU 10执行。具体的，在图12所示的终端设备100的结构示意图中，图像匹配模块215可以将匹配结果反馈给CPU 10，CPU 10可以根据匹配结果统计预设统计期间图像识别加速器20识别图像数据的图像识别成功率。CPU 10可以根据统计的图像识别成功率以及重新设置的第二图像识别成功率确定是否需要调整图像识别参数。当CPU 10确定统计的图像识别成功率与设置的第二图像识别成功率的差值的绝对值大于预设阈值时，CPU 10可以根据第二图像识别成功率以及所述终端设备的系统功耗调整下述至少一个图像识别参数：降维参数γ、宽度参数ω以及第一电流I，并将调整后的降维参数γ'、宽度参数ω'以及第一电流I'分别发送给降维处理模块205和NVM 210。换一种表达方式，在图12所示的结构中，CPU 10可以执行图8中所示的步骤800-810的方法。各器件具体功能的描述可以参见前述实施例的相关描述，在此不再赘述。可以理解的是，在图12所示的结构中，当CPU 10调整图像识别参数时，可以采用指令的形式向图像识别加速器20中的降维处理模块205和NVM 210发送调整的降维参数γ'、宽度参数ω'以及第一电流I'，从而控制降维处理模块205和NVM 210按照调整后的图像识别参数对图像数据进行识别。
本领域技术人员可以理解的是，虽然上述实施例是以将图像识别成功率从第一图像识别成功率调整为第二图像识别成功率为例对本发明实施例提供的图像识别加速器实现参数调整的过程进行描述，但上述示例只是为了对图像识别加速器能够实现的参数调整功能，从而获得满足条件的图像识别参数(包括降维参数、宽度参数以及第一电流)值的一个示例。可以理解的是，在本发明实施例中，用于对第一图像数据进行识别的降维参数γ、宽度参数ω以及第一电流I也是依据上述参数调整方式调整后获得的。换一种表达方式，上述参数调整方法是对如何通过调整参数值以获得满足系统功耗以及图像识别成功率的需求的图像识别参数值的一种方法的描述，实际应用中，当需要调整参数时，均可以按照上述参数调整方法获得满足需求的参数值。
在获得调整后的降维参数γ'、宽度参数ω'以及第一电流I'后，图像识别加速器20可以根据调整后的降维参数γ'、宽度参数ω'以及第一电流I'对后续需要识别的第二图像数据进行图像识别。具体的，如图8所示，在步骤815中，降维处理模块205可以根据调整后 的降维参数γ'降低第二图像数据的维度。在步骤820中，NVM 210可以将降维后的第二图像数据的每一个数值的低ω'位按照调整后的第一电流I'存储于NVM的第一存储区域，将降维后的第二图像数据的每一个数值的高(N-ω')位按照所述第二电流Is存储于NVM 210的第二存储区域，其中，所述I'小于所述Is。在步骤825中，图像匹配模块215可以确定所述NVM中存储的图像库中是否包含有与所述降维后的第二图像数据相匹配的图像数据。具体的，图像匹配模块215可以将降维后的第二图像数据与NVM 210中存储的图像库中的图像数据进行比较，以获得所述降维后的第二图像数据与NVM 210中存储的图像库中的图像数据的匹配结果。可以理解的是，图像识别加速器根据调整后的降维参数γ'、宽度参数ω'以及第一电流I'对第二图像数据进行图像识别的过程与前述的根据降维参数γ、宽度参数ω以及第一电流I对第一图像数据进行图像识别的过程类似，具体描述可以参见前面的描述，在此不再赘述。
可以理解的是，本发明实施例中的参数调整方法只在需要调整参数时才会触发调整参数。换一种表达方式，当需要调整图像识别参数的值时，图像识别加速器可以触发停止接收待识别的图像数据(也可称为业务数据)，而是按照图9所示的方式通过调整参数值并识别实验数据的方式获得满足需求的图像识别参数的值。在获得满足图像识别需求的参数值后，再将满足需求的参数值分别发送给降维处理模块205以及NVM 210，使得降维处理模块205、NVM 210以及图像匹配模块215能够根据调整获得的图像识别参数值对待识别的图像数据(例如第一图像数据和第二图像数据)进行图像识别。
为了更清晰的描述本发明实施例提供的图像识别加速器20如何识别图像数据，下面将结合图13所示的图像识别方法的信令图对图7所示的图像识别加速器20的工作过程进行简单的描述。在本发明实施例中，仍然以识别第一图像数据为例进行描述。如图13所示。降维处理模块205接收CPU 10发送的待识别的第一图像数据1300后，降维处理模块205可以根据参数调整模块220设置的降维参数γ降低第一图像数据1300的维度。其中降维处理模块205可以采用伯努利矩阵对第一图像数据1300进行降维，以便能够基于稀疏表示的随机映射的方式对第一图像数据1300进行降维。NVM 210接收到降维后的第一图像数据1305后，NVM 210可以根据参数调整模块220设置的宽带参数ω以及第一电流I将降维后的第一图像数据的每一个数值中的低ω位按照第一电流I存储于第一存储区域2104，并将降维后的第一图像数据的每一个数值中的高N-ω位按照第二电流Is存储于第二存储区域2106。图像匹配模块215可以基于NVM210存储的图像库中的图像数据对降维后的第一图像数据1305进行识别，确定所述NVM中存储的图像库中是否包含有与所述降维后的第一图像数据 相匹配的图像数据，并输出匹配结果。为了图示方便，在图13中，将图像库中的图像数据以及降维后的第一图像数据1305统称为待比较的图像数据1310。一方面，在图13中，图像匹配模块215可以向CPU 10输出第一图像数据1305的识别结果。另一方面，统计模块225可以统计图像匹配模块215的图像匹配结果，从而获得统计期间内的图像识别成功率1320，参数调整模块220从而可以根据统计模块225获得的图像识别成功率1320与设置的第二图像识别成功率确定是否需要调整图像识别参数。当参数调整模块220确定需要调整图像识别参数时，参数调整模块220可以根据图9所示的方法对图像识别参数进行调整，并向降维模块205和NVM 210分别输出调整后的降维参数γ'、宽度参数ω'以及第一电流I'。从而，降维处理模块205、NVM 210以及图像匹配模块215能够根据调整后的降维参数γ'、宽度参数ω'以及第一电流I'对后续的第二图像数据进行识别。
可以理解的是，图13仅仅是对本发明实施例提供的一种终端设备100的信令示意图，对其他实施例提供的图像识别加速器20或终端设备100的工作过程可以参见图13以及前述实施例的描述。在此不再赘述。
本发明实施例提供的终端设备，通过图像识别加速器识别图像，减少了CPU数据处理量，也减少了CPU与内存的数据交互，能够减少CPU的负担，且能够减少内存带宽对图像数据识别应用的限制，提高图像数据的识别速度。并且，在本发明实施例提供的终端设备中，图像识别加速器可以根据基于稀疏表示的随机映射的方式降低待识别的图像数据的维度，然后将降维后的图像数据按照不同的电流写入图像识别加速器的NVM中的不同存储区域。由于设置的降维参数γ、宽度参数ω以及第一电流I均是根据所述终端设备的系统功耗和设置的图像识别成功率获得的，因此能够在降低终端设备的系统功耗的基础上保证图像识别的准确性。
本发明实施例还提供一种数据处理的计算机程序产品，包括存储了程序代码的计算机可读存储介质，所述程序代码包括的指令用于执行前述任意一个方法实施例所述的方法流程。本领域普通技术人员可以理解，前述的存储介质包括：U盘、移动硬盘、磁碟、光盘、随机存储器(Random-Access Memory，RAM)、固态硬盘(Solid State Disk，SSD)或者非易失性存储器(non-volatile memory)等各种可以存储程序代码的非短暂性的(non-transitory)机器可读介质。
需要说明的是，本申请所提供的实施例仅仅是示意性的。所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其他实施例的相关描述。在本发明实施例、权利要求以及附图中揭示的特征可以独立存在也可以组合存在。在本发明实施例中以硬件形式描述的特征可以通过软件来执行，反之亦然。在此不做限定。

Claims (26)

  1. 一种应用于终端设备中用于识别图像的图像识别加速器，其特征在于，包括：
    降维处理模块，用于接收降维参数γ，根据接收的降维参数γ降低第一图像数据的维度，其中，降维后的第一图像数据包括多个数值，所述降维参数γ是根据所述终端设备的系统功耗和设置的第一图像识别成功率获得的；
    非易失性内存NVM，用于接收宽度参数ω和第一电流I，将降维后的第一图像数据的每一个数值的低ω位按照设置的第一电流I存储于所述NVM的第一存储区域，将降维后的第一图像数据的每一个数值的高(N-ω)位按照设置的第二电流Is存储于所述NVM的第二存储区域，其中，N为每一个数值所占的比特位，所述第一电流I小于所述第二电流Is，所述宽度参数ω以及所述第一电流I是根据所述终端设备的系统功耗和设置的第一图像识别成功率获得的；
    图像匹配模块，用于确定所述NVM中存储的图像库中是否包含有与所述降维后的第一图像数据相匹配的图像数据。
  2. 根据权利要求1所述的图像识别加速器，其特征在于，还包括：
    参数调整模块，用于根据设置的所述第一图像识别成功率以及所述终端设备的系统功耗调整下述至少一个参数：降维参数、宽度参数以及第一电流的值，以获得调整后的降维参数γ、宽度参数ω以及第一电流I，并将调整后的所述降维参数γ发送给所述降维处理模块，将调整后的所述宽度参数ω以及调整后的所述第一电流I发送给所述NVM。
  3. 根据权利要求2所述的图像识别加速器，其特征在于，所述参数调整模块具体用于：
    分别调整所述降维参数、所述宽度参数或所述第一电流的值，并分别获得多个调整后的图像识别成功率和多个调整后的系统功耗，每个调整后的图像识别成功率对应于每个调整后的系统功耗；
    确定调整后的每个图像识别成功率与所述第一图像识别成功率之间的差值，选择所述差值的绝对值不大于所述预设阈值的至少一个调整后的图像识别成功率所对应的至少一个调整后的系统功耗中的最小系统功耗；
    选择在满足所述最小系统功耗时获得最大图像识别成功率的降维参数、宽度参数以及第一电流的值分别作为调整后的所述降维参数γ、所述宽度参数ω以及所述第一电流I， 并将调整后的所述降维参数γ发送给所述降维处理模块，将调整后的所述宽度参数ω以及所述第一电流I发送给所述NVM。
  4. 根据权利要求1所述的图像识别加速器，其特征在于：
    所述参数调整模块，还用于如果统计的图像识别成功率与设置的第二图像识别成功率之间的差值的绝对值大于预设阈值，则根据所述第二图像识别成功率以及所述终端设备的系统功耗调整下述至少一个参数：降维参数γ、宽度参数ω以及第一电流I，以获得调整后的降维参数γ'、宽度参数ω'以及第一电流I'，其中，所述第二图像识别成功率与所述第一图像识别成功率不同；
    所述降维处理模块，还用于根据调整后的所述降维参数γ'降低第二图像数据的维度；
    所述非易失性内存NVM，还用于将降维后的第二图像数据的每一个数值的低ω'位按照调整后的第一电流I'存储于NVM的第一存储区域，将降维后的第二图像数据的每一个数值的高(N-ω')位按照所述第二电流Is存储于所述NVM的第二存储区域，其中，所述ω'为调整后的宽度参数，所述I'小于所述Is；
    所述图像匹配模块，还用于确定所述NVM中存储的图像库中是否包含有与所述降维后的第二图像数据相匹配的图像数据。
  5. 根据权利要求1-4任意一项所述的图像识别加速器，其特征在于，还包括：
    统计模块，用于统计在预设的统计期间内所述图像匹配模块输出的匹配结果，获取所述统计的图像识别成功率。
  6. 根据权利要求1-5任意一项所述的图像识别加速器，其特征在于，所述降维处理模块具体用于：
    根据所述第一图像数据与设置的二进制矩阵的乘积获得所述降维后的第一图像数据，其中，所述第一图像数据为k行*m列的矩阵，所述二进制矩阵为m行*n列的矩阵，所述降维后的第一图像数据为k行*n列的矩阵，k、m和n为正整数，m的值大于n，n的值根据设置的降维参数γ确定，γ＝n/m。
  7. 根据权利要求2或3所述的图像识别加速器，其特征在于，所述参数调整模块具体用于：
    如果统计的图像识别成功率与所述第二图像识别成功率之间的差值大于预设阈值，则分别调整降维参数γ、所述宽度参数ω或第一电流I的值，并分别获得多个调整后的图像识别成功率和多个调整后的系统功耗，其中，所述E的值与γ((N-ω)*Is 2+ω*I)的值成正比，每个调整后的图像识别成功率对应于每个调整后的系统功耗；
    确定调整后的每个图像识别成功率与所述第二图像识别成功率之间的差值，选择所述差值的绝对值不大于所述预设阈值时的至少一个调整后的图像识别成功率所对应的至少一个调整后的系统功耗中的最小系统功耗E'；
    选择在满足所述最小功耗E'时获得最大图像识别成功率的降维参数、宽度参数以及第一电流的值分别作为所述调整后的降维参数γ'、宽度参数ω'以及第一电流I'，并将所述调整后的降维参数γ'发送给所述降维处理模块，将所述调整后的宽度参数ω'以及第一电流I'发送给所述NVM。
  8. 根据权利要求6所述的图像识别加速器，其特征在于，所述二进制类型的矩阵包括伯努利映射矩阵。
  9. 一种终端设备，其特征在于，包括CPU和图像识别加速器，其中：
    所述CPU，用于向所述图像识别加速器发送待识别的第一图像数据；
    所述图像识别加速器，用于根据降维参数γ降低所述第一图像数据的维度，其中，降维后的第一图像数据包括多个数值，所述降维参数γ是根据所述终端设备的系统功耗和设置的第一图像识别成功率获得的；
    将所述降维后的第一图像数据的每一个数值的低ω位按照第一电流I存储于NVM的第一存储区域，将所述降维后的第一图像数据的每一个数值的高(N-ω)位按照设置的第二电流Is存储于所述NVM的第二存储区域，其中，N为每一个数值所占的比特位，ω为宽度参数，所述I小于所述Is，所述宽度参数ω以及所述第一电流I是根据所述终端设备的系统功耗和设置的第一图像识别成功率获得的；
    确定所述NVM中存储的图像库中是否包含有与所述降维后的第一图像数据相匹配的图像数据。
  10. 根据权利要求9所述的终端设备，其特征在于，所述图像识别加速器还用于：
    根据设置的所述第一图像识别成功率以及所述终端设备的系统功耗调整下述至少 一个参数：降维参数、宽度参数以及第一电流的值，以获得所述降维参数γ、所述宽度参数ω以及所述第一电流I。
  11. 根据权利要求10所述的终端设备，其特征在于，所述图像识别加速器具体用于：
    分别调整所述降维参数、所述宽度参数或所述第一电流的值，并分别获得多个调整后的图像识别成功率和多个调整后的系统功耗，每个调整后的图像识别成功率对应于每个调整后的系统功耗；
    确定调整后的每个图像识别成功率与所述第一图像识别成功率之间的差值，选择所述差值的绝对值不大于所述预设阈值的至少一个调整后的图像识别成功率所对应的至少一个调整后的系统功耗中的最小系统功耗；
    选择在满足所述最小系统功耗时获得最大图像识别成功率的降维参数、所述宽度参数以及第一电流的值分别作为所述降维参数γ、所述宽度参数ω以及所述第一电流I。
  12. 根据权利要求9所述的终端设备，其特征在于：
    所述图像识别加速器，还用于如果统计的图像识别成功率与设置的第二图像识别成功率之间的差值的绝对值大于预设阈值，则根据所述第二图像识别成功率以及所述终端设备的系统功耗调整下述至少一个参数：降维参数γ、宽度参数ω以及第一电流I，以获得调整后的降维参数γ'、宽度参数ω'以及第一电流I'，其中，所述第二图像识别成功率与所述第一图像识别成功率不同；
    所述CPU，还用于向所述图像识别加速器发送第二图像数据；
    所述图像识别加速器，还用于：
    根据调整后的所述降维参数γ'降低第二图像数据的维度；
    将降维后的第二图像数据的每一个数值的低ω'位按照调整后的所述第一电流I'存储于NVM的第一存储区域，将降维后的第二图像数据的每一个数值的高(N-ω')位按照所述第二电流Is存储于所述NVM的第二存储区域，其中，ω'为调整后的宽度参数，所述I'小于所述Is；
    确定所述NVM中存储的图像库中是否包含有与所述降维后的第二图像数据相匹配的图像数据。
  13. 根据权利要求9所述的终端设备，其特征在于：
    所述CPU，还用于统计在预设的统计期间内所述图像识别加速器输出的匹配结果，获取统计的图像识别成功率；
    确定所述统计的图像识别成功率与设置的第二图像识别成功率之间的差值的绝对值大于预设阈值；
    所述图像识别加速器，还用于根据所述第二图像识别成功率以及所述终端设备的系统功耗调整下述至少一个参数：降维参数γ、宽度参数ω以及第一电流I的值，以获得调整后的降维参数γ'、宽度参数ω'以及第一电流I'，其中，所述第二图像识别成功率与所述第一图像识别成功率不同；
    所述CPU，还用于向所述图像识别加速器发送第二图像数据；
    所述图像识别加速器，还用于根据调整后的所述降维参数γ'降低第二图像数据的维度；
    将降维后的第二图像数据的每一个数值的低ω'位按照调整后的所述第一电流I'存储于NVM的第一存储区域，将降维后的第二图像数据的每一个数值的高(N-ω')位按照所述第二电流Is存储于所述NVM的第二存储区域，其中，所述ω'为调整后的宽度参数，所述I'小于所述Is；
    确定所述NVM中存储的图像库中是否包含有与所述降维后的第二图像数据相匹配的图像数据。
  14. 根据权利要求9所述的终端设备，其特征在于，所述CPU还用于：
    统计在预设的统计期间内所述图像识别加速器输出的匹配结果，获取所述统计的图像识别成功率；
    如果统计的图像识别成功率与设置的第二图像识别成功率之间的差值的绝对值大于预设阈值，则根据所述第二图像识别成功率以及所述终端设备的系统功耗调整下述至少一个参数：降维参数γ、宽度参数ω以及第一电流I，以获得调整后的降维参数γ'、宽度参数ω'以及第一电流I'，其中，所述第二图像识别成功率与所述第一图像识别成功率不同；
    向所述图像识别加速器发送第二图像数据；
    所述图像识别加速器，还用于：
    根据调整后的所述降维参数γ'降低第二图像数据的维度；
    将降维后的第二图像数据的每一个数值的低ω'位按照调整后的所述第一电流I'存储于NVM的第一存储区域，将降维后的第二图像数据的每一个数值的高(N-ω')位按照所述 第二电流Is存储于所述NVM的第二存储区域，其中，所述ω'为调整后的宽度参数，所述I'小于所述Is；
    确定所述NVM中存储的图像库中是否包含有与所述降维后的第二图像数据相匹配的图像数据。
  15. 根据权利要求9-14任意一项所述的终端设备，其特征在于，所述图像识别加速器具体用于：
    根据所述第一图像数据与设置的二进制矩阵的乘积获得所述降维后的第一图像数据，其中，所述第一图像数据为k行*m列的矩阵，所述二进制矩阵为m行*n列的矩阵，所述降维后的第一图像数据为k行*n列的矩阵，k、m和n为正整数，m的值大于n，n的值根据设置的降维参数γ确定，γ＝n/m。
  16. 根据权利要求10或13所述的终端设备，其特征在于，所述图像识别加速器具体用于：
    分别调整降维参数γ、所述宽度参数ω或第一电流I的值，并分别获得多个调整后的图像识别成功率和多个调整后的系统功耗，其中，所述E的值与γ((N-ω)*Is 2+ω*I)的值成正比，每个调整后的图像识别成功率对应于每个调整后的系统功耗；
    确定调整后的每个图像识别成功率与所述第二图像识别成功率之间的差值，选择所述差值的绝对值不大于所述预设阈值时的至少一个调整后的图像识别成功率所对应的至少一个调整后的系统功耗中的最小系统功耗E'；
    选择在满足所述最小功耗E'时获得最大图像识别成功率的降维参数、宽度参数以及第一电流I的值分别作为所述调整后的降维参数γ'、宽度参数ω'以及第一电流I'。
  17. 根据权利要求14所述的终端设备，其特征在于，所述CPU具体用于：
    分别调整降维参数γ、所述宽度参数ω或第一电流I的值，并分别获得多个调整后的图像识别成功率和多个调整后的系统功耗，其中，所述E的值与γ((N-ω)*Is 2+ω*I)的值成正比，每个调整后的图像识别成功率对应于每个调整后的系统功耗；
    确定调整后的图像识别成功率与所述第二图像识别成功率之间的差值，选择所述差值的绝对值不大于所述预设阈值时的至少一个调整后的图像识别成功率所对应的至少一个调整后的系统功耗中的最小系统功耗E'；
    选择在满足所述最小功耗E'时获得最大图像识别成功率的降维参数、宽度参数以及第一电流分别作为所述调整后的降维参数γ'、宽度参数ω'以及第一电流I'。
  18. 根据权利要求15所述的终端设备，其特征在于：所述二进制类型的矩阵包括伯努利映射矩阵。
  19. 一种应用于终端设备的图像识别方法，其特征在于，所述方法由所述终端设备中的图像识别加速器执行，所述方法包括：
    根据降维参数γ降低第一图像数据的维度，其中，降维后的第一图像数据包括多个数值，所述降维参数γ是根据所述终端设备的系统功耗和设置的第一图像识别成功率获得的；
    将所述降维后的第一图像数据的每一个数值的低ω位按照设置的第一电流I存储于所述图像识别加速器中的NVM的第一存储区域，将所述降维后的第一图像数据的每一个数值的高(N-ω)位按照设置的第二电流Is存储于所述NVM的第二存储区域，其中，N为每一个数值所占的比特位，ω为宽度参数，所述I小于所述Is，所述宽度参数ω以及第一电流I是根据所述终端设备的系统功耗和设置的第一图像识别成功率获得的；
    确定所述NVM中存储的图像库中是否包含有与所述降维后的第一图像数据相匹配的图像数据。
  20. 根据权利要求19所述的图像识别方法，其特征在于，还包括：
    根据设置的所述第一图像识别成功率以及所述终端设备的系统功耗调整下述至少一个参数：降维参数、宽度参数以及第一电流的值，以获得调整后的降维参数γ、宽度参数ω以及第一电流I。
  21. 根据权利要求20所述的图像识别方法，其特征在于，所述根据设置的所述第一图像识别成功率以及所述终端设备的系统功耗调整下述至少一个参数：降维参数、宽度参数以及第一电流的值，以获得调整后的降维参数γ、宽度参数ω以及第一电流I具体包括：
    分别调整所述降维参数、所述宽度参数或所述第一电流的值，并分别获得多个调整后的图像识别成功率和多个调整后的系统功耗，每个调整后的图像识别成功率对应于每 个调整后的系统功耗；
    确定调整后的每个图像识别成功率与所述第一图像识别成功率之间的差值，选择所述差值的绝对值不大于所述预设阈值的至少一个调整后的图像识别成功率所对应的至少一个调整后的系统功耗中的最小系统功耗；
    选择在满足所述最小系统功耗时获得最大图像识别成功率的降维参数、宽度参数以及第一电流的值分别作为调整后的所述降维参数γ、所述宽度参数ω以及所述第一电流I。
  22. 根据权利要求19所述的图像识别方法，其特征在于，还包括：
    确定统计的图像识别成功率与设置的第二图像识别成功率的差值的绝对值大于预设阈值；
    根据所述第二图像识别成功率以及所述终端设备的系统功耗调整下述至少一个参数：降维参数γ、宽度参数ω以及第一电流I，以获得调整后的降维参数γ'、宽度参数ω'以及第一电流I'，其中，所述第二图像识别成功率与所述第一图像识别成功率不同；
    根据调整后的所述降维参数γ'降低第二图像数据的维度；
    将降维后的第二图像数据的每一个数值的低ω'位按照调整后的第一电流I'存储于NVM的第一存储区域，将降维后的第二图像数据的每一个数值的高(N-ω')位按照所述第二电流Is存储于所述NVM的第二存储区域，其中，所述ω'为调整后的宽度参数，所述I'小于所述Is；
    确定所述NVM中存储的图像库中是否包含有与所述降维后的第二图像数据相匹配的图像数据。
  23. 根据权利要求19-22任意一项所述的图像识别方法，其特征在于，还包括：
    统计在预设的统计期间内所述图像匹配模块输出的匹配结果，获取所述统计的图像识别成功率。
  24. 根据权利要求19-23任意一项所述的图像识别方法，其特征在于，所述根据设置的降维参数γ降低第一图像数据的维度包括：
    根据所述第一图像数据与设置的二进制矩阵的乘积获得所述降维后的第一图像数据，其中，所述第一图像数据为k行*m列的矩阵，所述二进制矩阵为m行*n列的矩阵，所述降维后的第一图像数据为k行*n列的矩阵，k、m和n为正整数，m的值大于n，n的 值根据设置的降维参数γ确定，γ＝n/m。
  25. 根据权利要求20-24任意一项所述的图像识别方法，其特征在于，所述根据所述第二图像识别成功率以及所述终端设备的系统功耗调整下述至少一个参数：降维参数γ、宽度参数ω以及第一电流I，包括：
    分别调整降维参数γ、所述宽度参数ω或第一电流I的值，并分别获得多个调整后的图像识别成功率和多个调整后的系统功耗，其中，所述E的值与γ((N-ω)*Is 2+ω*I)的值成正比；
    确定调整后的每个图像识别成功率与所述第二图像识别成功率之间的差值，选择所述差值的绝对值不大于所述预设阈值时的至少一个调整后的图像识别成功率所对应的至少一个调整后的系统功耗中的最小系统功耗E'；
    选择在满足所述最小功耗E'时获得最大图像识别成功率的降维参数、宽度参数以及第一电流的值分别作为所述调整后的降维参数γ'、宽度参数ω'以及第一电流I'。
  26. 根据权利要求24所述的图像识别方法，其特征在于：所述二进制类型的矩阵包括伯努利映射矩阵。
PCT/CN2016/074240 2015-03-06 2016-02-22 图像识别加速器、终端设备及图像识别方法 Ceased WO2016141803A1 (zh)

Priority Applications (7)

Application Number Priority Date Filing Date Title
KR1020177024724A KR102057471B1 (ko) 2015-03-06 2016-02-22 이미지 인식 가속기, 단말 기기 및 이미지 인식 방법
JP2017546832A JP6399534B2 (ja) 2015-03-06 2016-02-22 画像認識アクセラレータ、端末デバイス、および画像認識方法
SG11201706525RA SG11201706525RA (en) 2015-03-06 2016-02-22 Image recognition accelerator, terminal device, and image recognition method
BR112017018752-3A BR112017018752B1 (pt) 2015-03-06 2016-02-22 Acelerador e método de reconhecimento de imagem, dispositivo terminal e mídia legível por computador
EP16761030.2A EP3244345B1 (en) 2015-03-06 2016-02-22 Image recognition accelerator, terminal device and image recognition method
CN201680012483.4A CN107851175B (zh) 2015-03-06 2016-02-22 图像识别加速器、终端设备及图像识别方法
US15/695,681 US10346701B2 (en) 2015-03-06 2017-09-05 Image recognition accelerator, terminal device, and image recognition method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510101155.9A CN105989352B (zh) 2015-03-06 2015-03-06 图像识别加速器、终端设备及图像识别方法
CN201510101155.9 2015-03-06

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/695,681 Continuation US10346701B2 (en) 2015-03-06 2017-09-05 Image recognition accelerator, terminal device, and image recognition method

Publications (1)

Publication Number Publication Date
WO2016141803A1 true WO2016141803A1 (zh) 2016-09-15

Family

ID=56879170

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/074240 Ceased WO2016141803A1 (zh) 2015-03-06 2016-02-22 图像识别加速器、终端设备及图像识别方法

Country Status (7)

Country Link
US (1) US10346701B2 (zh)
EP (1) EP3244345B1 (zh)
JP (1) JP6399534B2 (zh)
KR (1) KR102057471B1 (zh)
CN (2) CN105989352B (zh)
SG (1) SG11201706525RA (zh)
WO (1) WO2016141803A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108230296B (zh) 2017-11-30 2023-04-07 腾讯科技（深圳）有限公司 图像特征的识别方法和装置、存储介质、电子装置
CN108256492A (zh) * 2018-01-26 2018-07-06 郑州云海信息技术有限公司 一种图像识别方法、装置及系统
CN110147792B (zh) * 2019-05-22 2021-05-28 齐鲁工业大学 基于内存优化的药品包装字符高速检测系统及方法
CN114359532A (zh) * 2021-12-31 2022-04-15 陕西科技大学 一种图像数据识别加速系统
US20260111484A1 (en) * 2024-10-18 2026-04-23 Charter Communications Operating, Llc System And Method For Semantic Video Metadata Search System at the Edge

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1577622A (zh) * 2003-07-18 2005-02-09 株式会社半导体能源研究所 存储器电路及包含其的显示装置和电子设备
CN1904907A (zh) * 2005-07-27 2007-01-31 上海明波通信技术有限公司 一种高速图像匹配检测系统及方法
CN103514432A (zh) * 2012-06-25 2014-01-15 诺基亚公司 人脸特征提取方法、设备和计算机程序产品
CN103810119A (zh) * 2014-02-28 2014-05-21 北京航空航天大学 一种利用三维集成电路片上温差降低stt-ram功耗的缓存设计方法

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3440617A (en) * 1967-03-31 1969-04-22 Andromeda Inc Signal responsive systems
US5339108A (en) * 1992-04-09 1994-08-16 Ampex Corporation Ordering and formatting coded image data and reconstructing partial images from the data
US6084600A (en) * 1996-03-15 2000-07-04 Micron Technology, Inc. Method and apparatus for high-speed block transfer of compressed and word-aligned bitmaps
JP3915855B2 (ja) * 1997-12-19 2007-05-16 ソニー株式会社 画像符号化装置および画像符号化方法、並びに学習装置および学習方法
JP4825644B2 (ja) * 2006-11-14 2011-11-30 ルネサスエレクトロニクス株式会社 画像復号装置、画像符号化装置、およびシステムｌｓｉ
CN101236601B (zh) * 2008-03-11 2010-10-06 马磊 图像识别加速装置及具有图像识别加速装置的微处理器芯片
JP5303325B2 (ja) 2009-03-18 2013-10-02 ルネサスエレクトロニクス株式会社 データ処理装置
US8427875B2 (en) * 2010-12-07 2013-04-23 Silicon Motion Inc. Method and memory controller for reading data stored in flash memory by referring to binary digit distribution characteristics of bit sequences read from flash memory
KR101180471B1 (ko) 2011-09-27 2012-09-07 (주)올라웍스 한정된 메모리 환경 하에서 얼굴 인식 성능 향상을 위한 참조 얼굴 데이터베이스 관리 방법, 장치 및 컴퓨터 판독 가능한 기록 매체
TWI455041B (zh) 2011-11-07 2014-10-01 Pixart Imaging Inc 人臉影像辨識方法及裝置
CN103106388B (zh) 2011-11-15 2017-02-08 中国科学院深圳先进技术研究院 图像识别方法和系统
KR20130057086A (ko) * 2011-11-23 2013-05-31 삼성전자주식회사 비휘발성 메모리 장치의 프로그램 방법
KR20140099295A (ko) * 2011-12-28 2014-08-11 인텔 코포레이션 파이프라인 이미지 프로세싱 시퀀서
US9645177B2 (en) * 2012-05-04 2017-05-09 Seagate Technology Llc Retention-drift-history-based non-volatile memory read threshold optimization
US8861270B2 (en) 2013-03-11 2014-10-14 Microsoft Corporation Approximate multi-level cell memory operations
US9118346B2 (en) * 2013-12-19 2015-08-25 Analog Devices, Inc. Complementary switches in current switching digital to analog converters
CN103824075A (zh) * 2014-02-18 2014-05-28 深圳天源迪科信息技术股份有限公司 图像识别系统及方法
US9213602B1 (en) * 2014-06-23 2015-12-15 Seagate Technology Llc Write mapping to mitigate hard errors via soft-decision decoding

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1577622A (zh) * 2003-07-18 2005-02-09 株式会社半导体能源研究所 存储器电路及包含其的显示装置和电子设备
CN1904907A (zh) * 2005-07-27 2007-01-31 上海明波通信技术有限公司 一种高速图像匹配检测系统及方法
CN103514432A (zh) * 2012-06-25 2014-01-15 诺基亚公司 人脸特征提取方法、设备和计算机程序产品
CN103810119A (zh) * 2014-02-28 2014-05-21 北京航空航天大学 一种利用三维集成电路片上温差降低stt-ram功耗的缓存设计方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3244345A4 *

Also Published As

Publication number Publication date
CN105989352B (zh) 2019-08-20
KR102057471B1 (ko) 2020-01-14
US10346701B2 (en) 2019-07-09
KR20170140162A (ko) 2017-12-20
EP3244345B1 (en) 2019-06-19
JP2018508904A (ja) 2018-03-29
EP3244345A4 (en) 2017-12-13
BR112017018752A2 (zh) 2018-07-24
SG11201706525RA (en) 2017-09-28
EP3244345A1 (en) 2017-11-15
CN107851175A (zh) 2018-03-27
CN105989352A (zh) 2016-10-05
US20180012095A1 (en) 2018-01-11
CN107851175B (zh) 2020-07-14
JP6399534B2 (ja) 2018-10-03

Similar Documents

Publication Publication Date Title
CN107466418B (zh) 用于多级别单元模式非易失性存储器的成本优化单级别单元模式非易失性存储器
WO2016141803A1 (zh) 图像识别加速器、终端设备及图像识别方法
US10211947B2 (en) System-on-chip using dynamic voltage frequency scaling and method of operating the same
US10528111B2 (en) Apparatuses and methods for indicating an operation type associated with a power management event
CN111245732A (zh) 一种流量控制方法、装置及设备
US20170344311A1 (en) Method of operating a memory device
KR102949947B1 (ko) 저장 장치 및 그의 동작 방법
US11783872B2 (en) Apparatuses and methods for performing operations using sense amplifiers and intermediary circuitry
CN112910890B (zh) 基于时间卷积网络的匿名网络流量指纹识别方法及设备
US11275521B2 (en) Image data based media type selection based on a first identified attribute of the initial image data
TW202004504A (zh) 記憶體裝置、記憶體裝置的控制方法及記錄媒體
US8966146B2 (en) Data processing method and data processing unit using the same
CN103765888A (zh) 分析辅助编码
CN106780415A (zh) 一种直方图统计电路及多媒体处理系统
US20250217082A1 (en) Media type selection for image data
CN113900813B (zh) 一种基于双口ram的盲元填充方法、系统及装置
US11733917B2 (en) High bandwidth controller memory buffer (CMB) for peer to peer data transfer
CN118963678B (zh) 一种存储设备的性能提升方法、装置、设备和介质
BR112017018752B1 (pt) Acelerador e método de reconhecimento de imagem, dispositivo terminal e mídia legível por computador
CN121598374A (zh) 计算存储系统、其操作方法以及电子装置
CN117873391A (zh) 芯片数据缓存方法、装置、计算机设备和存储介质
WO2020000735A1 (zh) 写数据方法、装置、计算机装置及存储介质
CN103679628A (zh) 影像处理记录系统及其影像处理记录方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16761030

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 11201706525R

Country of ref document: SG

REEP Request for entry into the european phase

Ref document number: 2016761030

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 20177024724

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2017546832

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

REG Reference to national code

Ref country code: BR

Ref legal event code: B01A

Ref document number: 112017018752

Country of ref document: BR

ENP Entry into the national phase

Ref document number: 112017018752

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20170831