WO2021143101A1 - 人脸识别方法和人脸识别装置 - Google Patents

人脸识别方法和人脸识别装置 Download PDF

Info

Publication number
WO2021143101A1
WO2021143101A1 PCT/CN2020/105772 CN2020105772W WO2021143101A1 WO 2021143101 A1 WO2021143101 A1 WO 2021143101A1 CN 2020105772 W CN2020105772 W CN 2020105772W WO 2021143101 A1 WO2021143101 A1 WO 2021143101A1
Authority
WO
WIPO (PCT)
Prior art keywords
face
feature
image
feature point
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/105772
Other languages
English (en)
French (fr)
Inventor
张红蕾
李志刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to US17/758,946 priority Critical patent/US12307815B2/en
Priority to EP20914749.5A priority patent/EP4075324B1/en
Publication of WO2021143101A1 publication Critical patent/WO2021143101A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification

Definitions

  • This application relates to the field of artificial intelligence, and in particular to a face recognition method and face recognition device.
  • Face recognition is a kind of biometric technology based on the facial feature information of the person to identify the identity, and it is an important application in the field of artificial intelligence (AI). Face recognition uses analysis and comparison of facial visual feature information for identity authentication. With the rapid development of computer and network technology, face recognition technology has been widely used in many industries and fields such as smart access control, smart door locks, mobile terminals, public safety, entertainment, and military.
  • Face recognition includes collecting face images, detecting faces in the images, preprocessing the detected faces, and then extracting image features and identifying and matching.
  • the extraction of image features refers to the extraction of image features including structure and texture from the image, and subsequent recognition and matching are completed based on the image features.
  • the integrity of the image features is a key factor that affects the success or failure of face recognition.
  • the face recognition effect depends on the integrity of the image features.
  • the face image is disturbed by the outside world, such as uneven light, or hats, scarves, and masks
  • some image features will be lost, and the image features will be incomplete, resulting in a low success rate of face recognition.
  • the embodiment of the present application provides a face recognition method for recognizing a face image, especially a face image with occlusion, which can improve the success rate of face recognition.
  • the first aspect of the embodiments of the present application provides a face recognition method, including: acquiring a face image to be recognized; extracting features of the face image through a pre-trained feature extraction network according to the face image; extracting the Multiple facial geometric feature points in the face image to determine multiple feature point sets, each feature point set in the multiple feature point sets corresponds to a face part, and the feature point set includes at least one facial geometry Feature points; acquiring face topological structure features according to the multiple feature point sets, where the face topological structure features are used to determine the relative positional relationship between the multiple feature point sets; according to the face topological structure features And the face image feature is matched in a preset face database to obtain a face recognition result.
  • the facial topological structure features are also extracted.
  • the facial topological structure features are used to characterize the topological structure between face parts, that is, the relative position. Relations, the topological structure has a low degree of dependence on the integrity of the image. Therefore, the combination of facial image features and facial topological structure features for matching recognition can improve the success rate of face recognition during occlusion.
  • the face topology feature includes: a feature vector set, and the feature vector in the feature vector set is used to indicate any two of the multiple feature point sets.
  • the face recognition method provided by the embodiment of the present application can directly obtain the topological structure features of the face according to multiple feature point sets, and express them in two forms: a feature vector set and a feature matrix.
  • obtaining the features of the face topology structure according to the plurality of feature point sets of face parts includes: constructing the plurality of feature point sets and multiple feature point sets of a standard face The mapping relationship is used to determine the relative position relationship between the multiple feature point sets; the mapping relationship is input to a pre-trained face topology feature extraction network to obtain the person Features of face topology.
  • the face recognition method provided by the embodiment of the application can indirectly acquire the features in the face image by constructing the corresponding relationship between the feature points of the face image and the standard face, and according to the mapping relationship data and the known standard face structure information
  • the relative positional relationship between points, and the relative positional relationship between multiple feature point sets, the mapping relationship data is input to the pre-training network, and the topological structure features of the face can be extracted.
  • the mapping relationship includes a distance and/or an angle between the multiple feature point sets and the multiple feature point sets of the standard face.
  • the face topology structure feature extraction network is obtained after the first network training, and the method further includes: extracting multiple facial geometric feature points from the face image training sample To determine multiple sample feature point sets, each feature point set in the multiple sample feature point sets corresponds to a face part of the training sample, and the sample feature point set includes at least one facial geometric feature point; Obtain the mapping relationship between the sample feature point set and the feature point set of the standard face, and input it into the first network for training to obtain a first loss value; update the first loss value according to the first loss value The weight parameters in the network are used to obtain the face topology feature extraction network.
  • the face recognition method provided by the embodiment of the present application can obtain a face topology feature extraction network by inputting topological structure data of a face image training sample for training.
  • extracting the face image features includes: inputting the face image into the pre-trained face The overall feature extraction network to extract the overall features of the face.
  • the features of the face image may include the overall feature of the face.
  • the overall feature of the face is the global feature, such as the image color feature or the image texture feature.
  • the extraction of the global feature depends on the integrity of the image. , When part of the face in the face image is occluded, the success rate of recognition based on the extracted global features is low.
  • the topological structure features of the face are combined and used for feature matching, which can provide the success rate of face recognition.
  • the method further includes: extracting a first face part image from the face image; according to the face image, using a pre-trained feature extraction network to extract The face image feature includes: inputting the first face part image to a pre-trained first part feature extraction network to extract the first part feature, and the first part feature is used in the face database Perform matching to obtain the face recognition result.
  • the face image feature may include the image feature of a face part, that is, the part feature, which can provide another form of face image feature, which improves the diversification of the solution.
  • the face part is, for example, the eye part, the nose part, or the mouth part.
  • the first part feature extraction network is obtained after training by the second network, and the method further includes: face part images extracted from face image training samples Input to the second network for training to obtain a second loss value; update the weight parameter in the second network according to the second loss value to obtain the first part feature extraction network.
  • the method further includes: extracting a plurality of face part images from the face image; and extracting a person according to the face image through a pre-trained feature extraction network
  • the face image feature includes: inputting the multiple face part images into a pre-trained multiple part feature extraction network to extract multiple part features; determining the target part feature of the face image according to the multiple part features .
  • the face recognition method provided by the embodiment of the present application extracts multiple face parts, and can separately extract multiple part features through the feature extraction network of each face part, which can improve the success rate of face recognition.
  • the target part feature is determined according to a weighted average of the multiple part features, and the weight of the multiple part features is a preset value.
  • the weights of features of different parts can be preset. Distinguish the importance of different face parts to improve the success rate of face recognition.
  • the method further includes: detecting whether a face part in the multiple face part images is occluded; if the first face part image in the first face part image is occluded; The face part is occluded, and the second face part in the second face part image is not occluded, and the second face part is a symmetrical part of the first face part, then The horizontally flipped image of the second face part image is determined to be the restored image of the first face part, and the restored image is used to input the part feature extraction network to extract the part feature.
  • the method further includes: based on the first face part being occluded, updating the weight of the part feature of the first face part, and the updated first face part A weight value is less than the preset first weight value of the first face part.
  • the face recognition method provided by the embodiments of the present application can reduce the weight of the feature of the occluded face, thereby effectively distinguishing the importance of the occluded part from the unoccluded part, and can improve the performance of the occluded scene. Success rate of face recognition.
  • the method further includes: preprocessing the face image to obtain a preprocessed face image, where the preprocessing includes face alignment, and The preprocessed face image is used to extract the face image features and extract the multiple facial geometric feature points.
  • preprocessing before the face image is used for feature extraction, preprocessing may be performed first to improve the feature extraction efficiency and feature accuracy.
  • the second aspect of the embodiments of the present application provides a face recognition device, including: an acquisition module for acquiring a face image to be recognized; an extraction module for extracting a network based on the face image through pre-trained features , Extracting facial image features; a determining module, used to extract multiple facial geometric feature points in the face image to determine multiple feature point sets, each of the multiple feature point sets corresponds to one For a human face part, the feature point set includes at least one facial geometric feature point; the acquiring module is further configured to acquire a face topological structure feature according to the multiple feature point sets, and the face topological structure feature is used for Determine the relative positional relationship between the multiple feature point sets; a matching module, configured to perform matching in a preset face database according to the face topology structure feature and the face image feature to obtain a face Recognition results.
  • the face topology feature includes: a feature vector set, and the feature vector in the feature vector set is used to indicate any two of the multiple feature point sets.
  • the determining module is further configured to: construct a mapping relationship between the multiple feature point sets and the multiple feature point sets of a standard face, and the mapping relationship is To determine the relative positional relationship between the multiple feature point sets; the acquisition module is specifically configured to: input the mapping relationship into a pre-trained face topology feature extraction network to acquire the face topology feature.
  • the mapping relationship includes a distance and/or an angle between the multiple feature point sets and the multiple feature point sets of the standard face.
  • the face topology structure feature extraction network is obtained after the first network is trained; the extraction module is also used to extract multiple facial geometry from the face image training sample Feature points to determine multiple sample feature point sets, each feature point set in the multiple sample feature point sets corresponds to a face part of the training sample, and the sample feature point set includes at least one facial geometric feature
  • the acquisition module is also used to acquire the mapping relationship between the sample feature point set and the feature point set of the standard face, and input it into the first network for training to obtain a first loss value; the The obtaining module is further configured to update the weight parameter in the first network according to the first loss value to obtain the face topology structure feature extraction network.
  • the extraction module is specifically configured to: input the face image into a pre-trained overall face feature extraction network to extract overall face features.
  • the extraction module is specifically configured to: extract a first face part image from the face image; and input the first face part image into a pre-
  • the trained first part feature extraction network is used to extract the first part feature, and the first part feature is used for matching in the face database to obtain the face recognition result.
  • the first part feature extraction network is obtained after training by the second network, and the acquisition module is further used for: the face extracted from the training sample of the face image The bit image is input to the second network for training to obtain a second loss value; the acquisition module is further configured to update the weight parameter in the second network according to the second loss value to obtain the first Location feature extraction network.
  • the extraction module is further configured to: extract multiple face part images from the face image; the extraction module is specifically configured to combine the multiple faces The part images are respectively input to a plurality of pre-trained part feature extraction networks to extract a plurality of part features; the determining module is further configured to determine the target part feature of the face image according to the multiple part features.
  • the target part feature is determined according to a weighted average of the multiple part features, and the weight of the multiple part features is a preset value.
  • the face recognition device further includes: a detection module, configured to detect whether the face parts in the multiple face part images are occluded; the determination module It is also used for, if the first face part in the first face part image is occluded, and the second face part in the second face part image is not occluded, the second face part Is a symmetrical part of the first face part, the horizontally flipped image of the second face part image is determined as the restored image of the first face part, and the restored image is used for input
  • the part feature extraction network is used to extract the part feature.
  • the face recognition device further includes: an update module, configured to update the position of the first face part based on the occlusion of the first face part The weight value of the feature, the updated first weight value is less than the preset first weight value of the first face part.
  • the acquisition module is further configured to: preprocess the face image to acquire a preprocessed face image, where the preprocessing includes face alignment,
  • the preprocessed face image is used to extract the face image features and extract the multiple facial geometric feature points.
  • the second aspect of the embodiments of the present application provides a face recognition device, including a processor and a memory, the processor and the memory are connected to each other, wherein the memory is used to store a computer program, and the computer program includes a program Instruction, the processor is used to call the program instruction to execute the method described in any one of the foregoing first aspect and various possible implementation manners.
  • the third aspect of the embodiments of the present application provides a computer program product containing instructions, which is characterized in that when it runs on a computer, the computer executes any one of the above-mentioned first aspect and various possible implementation manners. The method described in the item.
  • the fourth aspect of the embodiments of the present application provides a computer-readable storage medium, including instructions, characterized in that, when the instructions are run on a computer, the computer executes the first aspect and various possible implementation manners. Any of the methods.
  • a fifth aspect of the embodiments of the present application provides a chip including a processor.
  • the processor is used to read and execute the computer program stored in the memory to execute the method in any possible implementation manner of any one of the foregoing aspects.
  • the chip should include a memory, and the memory and the processor are connected to the memory through a circuit or a wire.
  • the chip further includes a communication interface, and the processor is connected to the communication interface.
  • the communication interface is used to receive data and/or information that needs to be processed, and the processor obtains the data and/or information from the communication interface, processes the data and/or information, and outputs the processing result through the communication interface.
  • the communication interface can be an input and output interface.
  • a face image is input to a pre-trained feature extraction network to obtain face image features.
  • multiple facial collection feature points in the face image are extracted to determine that they correspond to multiple people
  • a set of feature points of multiple face parts of the face and the topological structure features of the face are obtained according to the set of feature points of multiple face parts; feature matching is performed on the face data set according to the features of the topological structure of the face and the features of the face image, and finally Obtain the face recognition results. Since the face topology is constructed by the relative position relationship of multiple facial feature point sets, more structured information can be extracted, and the dependence on the integrity of the overall face image is reduced, which can effectively reduce The influence of occlusion on face recognition.
  • FIG. 1 is a schematic diagram of an artificial intelligence main body framework provided by an embodiment of this application.
  • Figure 2 is a schematic diagram of an application environment provided by an embodiment of the application.
  • FIG. 3 is a schematic diagram of a convolutional neural network structure provided by an embodiment of the application.
  • FIG. 4 is a schematic diagram of another convolutional neural network structure provided by an embodiment of the application.
  • Figure 5 is a system architecture diagram of an embodiment of the application
  • FIG. 6 is a schematic diagram of an embodiment of a face recognition method in an embodiment of this application.
  • FIG. 7 is a schematic diagram of feature points of a target set in an embodiment of this application.
  • FIG. 8 is a schematic diagram of a face topology structure in an embodiment of the application.
  • FIG. 9 is a schematic diagram of a standard face structure in an embodiment of the application.
  • FIG. 10 is a schematic diagram of a face part image and occlusion flip processing in an embodiment of the application.
  • FIG. 11 is a schematic diagram of an embodiment of a training method of a feature extraction network in an embodiment of the application.
  • FIG. 12 is a schematic diagram of a feature extraction network architecture in an embodiment of this application.
  • FIG. 13 is a schematic diagram of an embodiment of a face recognition device in an embodiment of the application.
  • FIG. 14 is a diagram of a chip hardware structure provided by an embodiment of the application.
  • FIG. 15 is a schematic diagram of another embodiment of a face recognition device in an embodiment of this application.
  • the embodiment of the present application provides a face recognition method for recognizing a face image, especially a face image with occlusion, which can improve the success rate of face recognition.
  • Face image an image containing face information
  • the human face is composed of parts such as eyes, nose, and mouth.
  • the geometric description of the shape and structure of these parts can be used as an important feature of face recognition. These features are the geometric features of the face.
  • Facial geometric feature points The human face is composed of eyes, nose, mouth and other parts. Through the detection of face images, the feature points used to characterize each face part can be extracted, that is, facial geometric feature points.
  • Face part image refers to an image that includes characteristic local areas in a face image, usually an image of the eyes, eyebrows, nose, or mouth.
  • Topology is a method of abstracting entities into "points” that have nothing to do with their size and shape, and abstracting lines connecting entities into “lines”, and then expressing the relationship between these points and lines in the form of graphs. Its purpose is to study the connection between these points and lines.
  • a graph showing the relationship between points and lines is called a topological structure graph.
  • Topological structure and geometric structure belong to two different mathematical concepts. In geometric structure, what we want to investigate is the positional relationship between points and lines, or the geometric structure emphasizes the shape and size of points and lines.
  • trapezoids, squares, parallelograms, and circles belong to different geometric structures, but from the perspective of topological structure, because the connection relationship between points and lines is the same, they have the same topological structure, that is, ring structure. In other words, different geometric structures may have the same topological structure.
  • the topological structure of a human face includes the connections between various parts of the human face.
  • Figure 1 shows a schematic diagram of an artificial intelligence main framework, which describes the overall workflow of the artificial intelligence system and is suitable for general artificial intelligence field requirements.
  • Intelligent Information Chain reflects a series of processes from data acquisition to processing. For example, it can be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, the data has gone through the condensing process of "data-information-knowledge-wisdom".
  • the infrastructure provides computing power support for the artificial intelligence system, realizes communication with the outside world, and realizes support through the basic platform.
  • smart chips hardware acceleration chips such as CPU, NPU, GPU, ASIC, FPGA
  • basic platforms include distributed computing frameworks and network related platform guarantees and support, which can include cloud storage and Computing, interconnection network, etc.
  • sensors communicate with the outside to obtain data, and these data are provided to the smart chip in the distributed computing system provided by the basic platform for calculation.
  • the data in the upper layer of the infrastructure is used to represent the data source in the field of artificial intelligence.
  • the data involves graphics, images, voice, and text, as well as the Internet of Things data of traditional devices, including business data of existing systems and sensory data such as force, displacement, liquid level, temperature, and humidity.
  • machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, training, etc.
  • Reasoning refers to the process of simulating human intelligent reasoning in a computer or intelligent system, using formal information to conduct machine thinking and solving problems based on reasoning control strategies.
  • the typical function is search and matching.
  • Decision-making refers to the process of making decisions after intelligent information is reasoned, and usually provides functions such as classification, ranking, and prediction.
  • some general capabilities can be formed based on the results of the data processing, such as an algorithm or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image Recognition and so on.
  • Intelligent products and industry applications refer to the products and applications of artificial intelligence systems in various fields. It is an encapsulation of the overall solution of artificial intelligence, productizing intelligent information decision-making and realizing landing applications. Its application fields mainly include: intelligent manufacturing, intelligent transportation, Smart home, smart medical, smart security, autonomous driving, safe city, smart terminal, etc.
  • an embodiment of the present application provides a system architecture 200.
  • the data collection device 260 is used to collect face image data and store it in the database 230, and the training device 220 generates a target model/rule 201 based on the face image data maintained in the database 230.
  • the following will describe in more detail how the training device 220 obtains the target model/rule 201 based on the face image data.
  • the target model/rule 201 can be used in application scenarios such as face recognition, image classification, and virtual reality.
  • the target model/rule 201 may be obtained based on a deep neural network, and the deep neural network will be introduced below.
  • the work of each layer in the deep neural network can be expressed in mathematical expressions To describe: From the physical level, the work of each layer in the deep neural network can be understood as the transformation of the input space to the output space (that is, the row space of the matrix to the column Space), these five operations include: 1. Dimension Up/Down; 2. Enlarge/Reduce; 3. Rotate; 4. Translation; 5. "Bend”. The operations of 1, 2, and 3 are determined by Completed, the operation of 4 is completed by +b, and the operation of 5 is realized by a(). The reason why the word "space” is used here is because the object to be classified is not a single thing, but a class of things. Space refers to the collection of all individuals of this kind of thing.
  • W is a weight vector, and each value in the vector represents the weight value of a neuron in the layer of neural network.
  • This vector W determines the spatial transformation from the input space to the output space described above, that is, the weight W of each layer controls how the space is transformed.
  • the purpose of training a deep neural network is to finally obtain the weight matrix of all layers of the trained neural network (the weight matrix formed by the vector W of many layers). Therefore, the training process of the neural network is essentially the way of learning to control the space transformation, and more specifically, the learning of the weight matrix.
  • the weight vector of the network (of course, there is usually an initialization process before the first update, which is to pre-configure parameters for each layer in the deep neural network). For example, if the predicted value of the network is high, adjust the weight vector to make it The prediction is lower and keep adjusting until the neural network can predict the target value you really want. Therefore, it is necessary to predefine "how to compare the difference between the predicted value and the target value".
  • This is the loss function or objective function, which is used to measure the difference between the predicted value and the target value. Important equation. Among them, take the loss function as an example. The higher the output value (loss) of the loss function, the greater the difference. Then the training of the deep neural network becomes a process of reducing this loss as much as possible.
  • the target model/rule obtained by the training device 220 can be applied to different systems or devices.
  • the execution device 210 is configured with an I/O interface 212 to perform data interaction with external devices.
  • the "user" can input data to the I/O interface 212 through the client device 240.
  • the execution device 210 can call data, codes, etc. in the data storage system 250, and can also store data, instructions, etc. in the data storage system 250.
  • the calculation module 211 uses the target model/rule 201 to process the input data. Taking face image recognition as an example, the calculation module 211 can analyze the input face image to obtain image features such as texture information in the face image.
  • the correlation function module 213 may preprocess the image data in the calculation module 211, for example, perform face image preprocessing, including face alignment.
  • the correlation function module 214 can preprocess the image data in the calculation module 211, for example, perform face image preprocessing, including face alignment.
  • the I/O interface 212 returns the processing result to the client device 240 and provides it to the user.
  • the training device 220 can generate corresponding target models/rules 201 based on different data for different targets, so as to provide users with better results.
  • the user can manually specify the input data in the execution device 210, for example, to operate in the interface provided by the I/O interface 212.
  • the client device 240 can automatically input data to the I/O interface 212 and obtain the result. If the client device 240 automatically inputs data and needs the user's authorization, the user can set the corresponding authority in the client device 240.
  • the user can view the result output by the execution device 210 on the client device 240, and the specific presentation form may be a specific manner such as display, sound, and action.
  • the client device 240 may also serve as a data collection terminal to store the collected training data in the database 230.
  • Fig. 2 is only a schematic diagram of a system architecture provided by an embodiment of the present application, and the positional relationship among the devices, devices, modules, etc. shown in the figure does not constitute any limitation.
  • the data storage system 250 is an external memory relative to the execution device 210. In other cases, the data storage system 250 may also be placed in the execution device 210.
  • Convolutional neural network (convolutional neural network, CNN) is a deep neural network with a convolutional structure. It is a deep learning architecture.
  • the deep learning architecture refers to the use of machine learning algorithms in different abstractions. There are multiple levels of learning at different levels.
  • CNN is a feed-forward artificial neural network. Taking image processing as an example, each neuron in the feed-forward artificial neural network responds to overlapping areas in the input image. .
  • a convolutional neural network (CNN) 100 may include an input layer 110, a convolutional layer/pooling layer 120, where the pooling layer is optional, and a neural network layer 130.
  • the convolutional layer/pooling layer 120 may include layers 121-126 as in the examples.
  • layer 121 is a convolutional layer
  • layer 122 is a pooling layer
  • layer 123 is a convolutional layer
  • 124 is a pooling layer
  • 121 and 122 are convolutional layers
  • 123 is a pooling layer
  • 124 and 125 are convolutional layers
  • 126 is a convolutional layer.
  • Pooling layer That is, the output of the convolutional layer can be used as the input of the subsequent pooling layer, or as the input of another convolutional layer to continue the convolution operation.
  • the convolutional layer 121 can include many convolution operators.
  • the convolution operator is also called a kernel. Its role in image processing is equivalent to a filter that extracts specific information from the input image matrix.
  • the convolution operator can be a weight matrix. This weight matrix is usually predefined. In the process of convolution on the image, the weight matrix is usually one pixel after another pixel in the horizontal direction on the input image ( Or two pixels followed by two pixels...It depends on the value of stride) to complete the work of extracting specific features from the image.
  • the size of the weight matrix should be related to the size of the image. It should be noted that the depth dimension of the weight matrix and the depth dimension of the input image are the same.
  • the weight matrix will extend to Enter the entire depth of the image. Therefore, convolution with a single weight matrix will produce a convolution output of a single depth dimension, but in most cases, a single weight matrix is not used, but multiple weight matrices with the same dimension are applied. The output of each weight matrix is stacked to form the depth dimension of the convolutional image. Different weight matrices can be used to extract different features in the image. For example, one weight matrix is used to extract edge information of the image, another weight matrix is used to extract specific colors of the image, and another weight matrix is used to eliminate unwanted noise in the image. Fuzzy... the dimensions of the multiple weight matrices are the same, and the dimension of the feature map extracted by the weight matrix of the same dimension is also the same, and then the extracted feature maps of the same dimension are merged to form the output of the convolution operation .
  • weight values in these weight matrices need to be obtained through a lot of training in practical applications, and each weight matrix formed by the weight values obtained through training can extract information from the input image, thereby helping the convolutional neural network 100 to make correct predictions.
  • the initial convolutional layer (such as 121) often extracts more general features, which can also be called low-level features; with the convolutional neural network
  • the subsequent convolutional layers for example, 126
  • features such as high-level semantics
  • the pooling layer can also be a multi-layer convolutional layer followed by one or more pooling layers.
  • the sole purpose of the pooling layer is to reduce the size of the image space.
  • the pooling layer may include an average pooling operator and/or a maximum pooling operator for sampling the input image to obtain an image with a smaller size.
  • the average pooling operator can calculate the pixel values in the image within a specific range to generate an average value.
  • the maximum pooling operator can take the pixel with the largest value within a specific range as the result of the maximum pooling.
  • the operators in the pooling layer should also be related to the image size.
  • the size of the image output after processing by the pooling layer can be smaller than the size of the image of the input pooling layer, and each pixel in the image output by the pooling layer represents the average value or the maximum value of the corresponding sub-region of the image input to the pooling layer.
  • the convolutional neural network 100 After processing by the convolutional layer/pooling layer 120, the convolutional neural network 100 is not enough to output the required output information. Because as mentioned above, the convolutional layer/pooling layer 120 only extracts features and reduces the parameters brought by the input image. However, in order to generate the final output information (required class information or other related information), the convolutional neural network 100 needs to use the neural network layer 130 to generate one or a group of required classes of output. Therefore, the neural network layer 130 may include multiple hidden layers (131, 132 to 13n as shown in FIG. 3) and an output layer 140. The parameters contained in the multiple hidden layers may be based on specific task types. The relevant training data of the, for example, the task type can include image recognition, image classification, image super-resolution reconstruction and so on.
  • the output layer 140 After the multiple hidden layers in the neural network layer 130, that is, the final layer of the entire convolutional neural network 100 is the output layer 140.
  • the output layer 140 has a loss function similar to the classification cross entropy, which is specifically used to calculate the prediction error.
  • the convolutional neural network 100 shown in FIG. 3 is only used as an example of a convolutional neural network.
  • the convolutional neural network may also exist in the form of other network models, for example,
  • the multiple convolutional layers/pooling layers shown in FIG. 4 are in parallel, and the respectively extracted features are input to the full neural network layer 130 for processing.
  • the face recognition method provided in the embodiments of this application is suitable for face recognition in various scenarios such as home and security, including robots, smart phones, desktop computers, tablet computers, TVs, home or public security surveillance cameras, cameras, access control, Use scenarios such as identity verification, personalized customization, and facial expression simulation of door locks, attendance machines, smart glasses and other products.
  • the process of face recognition can be completed by the above-mentioned entities, or it can be connected to a dedicated server through the network and completed by the server, and the details are not limited here.
  • the network includes one or more of multiple types of wireless or partial wireless communication networks, such as local area network (LAN), wireless local area network (WLAN), personal area network (PAN), and wide area network (WAN), Intranet, Internet, peer-to-peer network, peer-to-peer network or mesh network, etc.
  • LAN local area network
  • WLAN wireless local area network
  • PAN personal area network
  • WAN wide area network
  • Intranet Internet
  • peer-to-peer network peer-to-peer network or mesh network, etc.
  • the specifics are not limited here.
  • the face recognition device obtains a face image.
  • the face recognition device captures an image through a built-in or external camera, and then detects a face image including face information from the captured image.
  • the robot collects an image through a camera and detects that a human face is included in the image to obtain a human face image.
  • the face recognition device preprocesses the face image. Due to various conditions and random interference, the original face image cannot be used directly. It must be gray-scale correction and noise filtering in the early stage of image processing. Image preprocessing, to obtain preprocessed images for subsequent feature extraction.
  • the face image preprocessing may include: face alignment, light compensation, gray scale transformation, histogram equalization, normalization processing, geometric correction, median filtering, sharpening, etc.
  • the specific processing flow is not limited here.
  • the normalization process is used to obtain standardized face images with the same size and the same gray value range, and the median filter can be used to smooth the image to eliminate noise.
  • face alignment processing is performed on the face image, and faces of different scales and directions are normalized to a uniform scale according to the positions of the face feature points to obtain images with correct face positions.
  • Face rotation because the person in the face image may have different postures, for example, the detected face may be a front face or a side face.
  • the face can be rotated through face alignment, and the faces of different poses Rotate to the same angle as much as possible for easy identification.
  • Face alignment can reduce the influence of distance or pose on subsequent feature extraction, and recognize faces on a uniform scale.
  • the face recognition device detects facial geometric feature points in a face image to be recognized, and the facial geometric feature points include feature points used to characterize various parts of the face, such as eyes, nose, mouth, and facial contours.
  • the face recognition device may detect facial geometric feature points by using a preset algorithm, and the specific type of the preset algorithm is not limited here.
  • the number of facial geometric feature points to be extracted is not limited, 68 feature points or 128 feature points can be extracted.
  • FIG. 7, is a schematic diagram of facial geometric feature points in an embodiment of this application. 68 facial geometric feature points are shown.
  • multiple facial feature point sets can be determined.
  • the face feature point set composed of facial geometric feature points 18 to 22 is used to indicate the left eyebrow part 701; facial geometric feature points
  • the face feature point set consisting of 23 to 27 is used to indicate the right eyebrow part 702;
  • the face feature point set consisting of facial geometric feature points 37 to 42 is used to indicate the left eye part 703; facial geometric feature points 43
  • the face feature point set consisting of to 48 is used to indicate the right eye part 704;
  • the face feature point set consisting of facial geometric feature points 28 to 36 is used to indicate the nose part 705; facial geometric feature points 49 to 68
  • the formed feature point set of the face part is used to indicate the mouth part 706.
  • the connection relationship between the feature point sets of each face part is calculated to construct the face Topological structure, to obtain the topological structure features of the face.
  • the face topological structure feature is expressed as a feature vector set, and the feature vector in the feature vector set is used to indicate the relative position between any two face part feature point sets in the plurality of face part feature point sets relation.
  • FIG. 8 is a schematic diagram of a face topology structure in an embodiment of this application.
  • P represents a set of feature points of a face part
  • vectors ci, j represent the connection relationship between a set of feature points of a face part i and j.
  • the face topology is represented as [c0,1,c0,2,c0,5,c1,2,c1,3,c3,4,c2,4,c2,5,c1,5], exemplary, vector c0 ,1 can be obtained according to the position coordinate of P0 and the position coordinate of P1.
  • the vector notation only lists the characteristic relationships that have the connection relationship. It is understandable that the connection relationship of two face parts that do not directly have the connection relationship can also be obtained indirectly according to the topology information.
  • Normalize the facial structure features refers to the use of a unified measurement unit and measurement standard for the features. For example, if the nose is used as the origin, the distance from the eyes to the nose is C0,1. This distance can be the image coordinate system To calculate the system, the distance can also be further normalized to between 0 and 1.
  • connection relationships such as Euclidean distance, curvature, or angle, can also be defined, which are not specifically limited here.
  • mapping relationship between the multiple face part feature point sets and the face part feature point set of a standard face construct a mapping relationship between the multiple face part feature point sets and the face part feature point set of a standard face, and input the mapping relationship into a pre-trained face topology feature extraction network , In order to obtain the topological structure features of the face.
  • the mapping relationship between each face part in the face image and the corresponding face part in the standard face can be expressed by a metric relationship such as distance and/or angle, which is used to indicate the characteristics of the multiple face parts The relative positional relationship between point sets.
  • the standard face is used as a reference standard face image. Please refer to Figure 9.
  • the middle standard face at least one feature point is marked for the eyes, nose, eyebrows, mouth and other parts.
  • the nose part can be used
  • One feature point can be labeled, or multiple feature points can be used for labeling.
  • the number of feature points labeled for each part is not specifically limited and can be specified by the developer.
  • the mapping relationship can be expressed by measurement relationships such as distance and/or angle. For example, among the facial geometric feature points detected in the face image to be recognized, a feature point in the nose part feature point set is compared with the nose part feature point of the standard face. The metric relationship between a feature point in the set is a.
  • a feature point can be identified by two-dimensional data.
  • the data amount of the positional relationship between a feature point in the face image to be recognized and the 68 points in the standard face image is 1*68*2, and the 68 feature points in the face image to be recognized are compared with those in the standard face image.
  • the data amount of the positional relationship between 68 points is a data block of 68*68*2.
  • the data representing the mapping relationship is input into the pre-trained face topology feature extraction network, and the face topology feature can be obtained.
  • the face recognition device can determine each face part from the face image.
  • the face parts extracted from the image include: mouth, nose, left eye, left eyebrow, right eye, and right eyebrow.
  • the face parts to be distinguished can be preset and are not limited here.
  • FIG. 10 is a schematic diagram of the image of the human face and the occlusion flip processing in the embodiment of the application.
  • each face part determines whether each face part is occluded according to a preset part discrimination model for each face part. If the occluded part is one of the paired parts, and the other of the paired parts is not covered, for example, as shown in Figure 10, the eyebrows or eyes are covered by one, then the other part of the paired part that is not covered Flip horizontally as a restored image of the currently blocked part.
  • step 604 may be executed first, and then step 604 may be executed, or step 604 may be executed first, and then step 603 may be executed.
  • the face part image obtained in step 604 is input into the pre-trained part feature extraction network for feature extraction, and the target part feature is output; it should be noted that each face part can be preset with a corresponding feature extraction network. For multiple parts of the face, different part feature extraction networks can be preset for feature extraction, and then the individual part features extracted for each part can be synthesized to obtain the target part feature.
  • the target part feature is a weighted average of the multiple part features, and the weight of the multiple part features is a preset value.
  • the face recognition device detects whether the face parts in the multiple face part images are occluded. For the occluded parts, the weight can be reduced on the basis of the initial preset weight of the part, by Therefore, the influence of part occlusion on face recognition can be reduced.
  • the face topological structure feature extraction network For the training process of the overall face feature extraction network, the face topological structure feature extraction network, and the part feature extraction network, please refer to the embodiment corresponding to FIG. 11, which will not be described in detail here.
  • the feature matching module matches the features of the current face with the face features in the face database, and obtains the result of face recognition according to the similarity measurement between the features.
  • Extract features for parts reduce the impact of low-response areas, enhance the receptive fields of high-response areas, and learn more texture information to enhance the judgment and robustness of features.
  • the strategy with weight distribution is adopted for part recognition to reduce the influence of low-response parts on the final discrimination result and increase the influence of high-noise areas, which is conducive to face recognition;
  • the face part to be matched is turned over, features are extracted, and the occlusion discriminator is added to reduce the influence of the occlusion part on face recognition.
  • Propose a feature extraction unit that specializes in learning the topological structure of a face, learn the difference between a face and a standard face, and the link relationship between each topological node, and obtain more structure by linking the information between each layer This information can reduce the impact of occlusion on face recognition without affecting normal face recognition.
  • the following describes the training method of the feature extraction network in the face recognition method provided by this application. Please refer to Fig. 11.
  • the method includes:
  • the face recognition device obtains a face image from a face database.
  • the face database can be downloaded from the Internet, or a part of the database can be built by yourself, and the details are not limited here. Commonly used face databases include LFW dataset, Yale series, etc.
  • the face database includes occluded face images, or occlusion processing is performed on the face images in the face database to generate occluded face images, which is not specifically limited here.
  • Steps 1102 to 1103 are similar to the methods of step 602 and step 604 in the embodiment corresponding to FIG. 6, and will not be repeated here.
  • step 603 in the embodiment corresponding to FIG. 6 to construct the mapping relationship between the multiple face feature point sets and the face feature point set of the standard face, and obtain data representing the mapping relationship, representing a person Face topology.
  • FIG. 12 is a schematic diagram of a feature extraction network architecture in an embodiment of this application.
  • the first network can be various existing face extraction networks, which are not specifically limited here.
  • the input image data is a face image of H*W*3, where H is the height of the image, W is the width of the image, and 3 represents 3 channels of RGB (red green blue).
  • H is the height of the image
  • W is the width of the image
  • 3 represents 3 channels of RGB (red green blue).
  • the image is used as the input of training, which can make the image robust at multiple scales.
  • the input image passes through 9 convolutional layers, each block in Figure 12 includes 3 convolutional layers, and the output [T*1] vector.
  • the fully connected layer is followed by the softmax layer (not shown in the figure).
  • the input of this layer is to input the [T*1] vector into the softmax layer, and the output is also the [T*1] vector. Normalize each output vector to [0,1].
  • the vector output by softmax here is the probability that the sample belongs to each class.
  • the first network loss function L1 is as follows:
  • Ls is the softmax loss of the inter-class difference
  • Lc is the center loss of the individual difference within the class
  • m is the batch size (batch_size)
  • Xi is the i-th value of the vector of [T*1]
  • Xj is the j-th value
  • wi Is the weight of the convolution, learned from back propagation
  • bi is the corresponding bias.
  • the input image passes through the network 1, and the feature matrix (1*W dimension) is obtained; the score corresponding to each image block is a real number in the range of [0,1].
  • the training process of the part feature extraction network obtained by the training of network 2 corresponds to the parts 2 and 3 in Fig. 12.
  • the input of each part network is the H*W*3 part image.
  • the picture of each part is processed, and the result of whether each face part is occluded is obtained.
  • the above figure shows an example of a part network.
  • the part network can use a small network to extract features for the part, reduce the influence of the low response area, enhance the receptive field of the high response area, and learn more high-resolution texture information to enhance the features Judgment and robustness.
  • the initial weights are set for each part, which can be defined by the product developer.
  • the weight of eyes is 0.4
  • the weight of nose and mouth is 0.25
  • the weight of eyebrows is 0.1.
  • the image data of the current part processed by the part discriminator and the result of whether the part is occluded are obtained through the above-mentioned part network to obtain an N*1 dimensional feature vector and the final part weight wi of each part.
  • the part is occluded, perform a drop adjustment based on the initial weight. For example, if the eye is occluded, the weight of the eye part is adjusted to 0 after passing through the part network.
  • the loss function is as follows:
  • wi is the weight of the convolution
  • k represents the number of extracted face parts
  • Li is the calculated loss function for each part.
  • ⁇ j,i is the angle between vectors W j and x i , ⁇ i,j ⁇ (0, ⁇ )
  • cosine represents the angle cosine value of the feature vector and the weight vector.
  • the meaning of the parameter m is to make the distance between classes large enough and the distance within classes small enough.
  • the face topology feature extraction network obtained by the training of network 3 corresponds to the part marked 4 in Figure 12;
  • the network 3 training can obtain the face topology feature extraction network, with 68 facial feature points in 2 dimensions to measure the relationship distance, the network input is a data block of 68*68*2 size, after the network convolutional layer and the fully connected layer, The differential feature between the constructed face and the standard face is extracted, and the output is an M*1 dimensional feature vector.
  • the loss function L3 of network 3 is as follows:
  • m is the batch size (batch_size)
  • n is XX
  • Xi is the i-th value of the vector of [T*1]
  • Xj is the j-th value
  • wi is the weight of the convolution
  • wj represents XX.
  • Bi is the bias corresponding to the i-th value
  • bj is the bias corresponding to the j-th value.
  • the training process may not train the part features (network 2).
  • the overall face features and the face topological structure features are used to complete face recognition.
  • the training process may not train the overall face features (network 1).
  • the face part features and the face topological structure features are used to complete the face recognition.
  • FIG. 13 is a schematic diagram of an embodiment of the face recognition device in the embodiment of the application.
  • the face recognition device includes:
  • the obtaining module 1301 is used to obtain the face image to be recognized
  • the extraction module 1302 is configured to extract features of the face image through a pre-trained feature extraction network according to the face image;
  • the determining module 1303 is used to extract multiple facial geometric feature points in the face image to determine multiple feature point sets.
  • Each feature point set in the multiple feature point sets corresponds to a face part.
  • the set includes at least one facial geometric feature point;
  • the obtaining module 1301 is further configured to obtain the topological structure feature of the face according to the plurality of feature point sets, and the topological structure feature of the face is used to determine the relative position relationship between the plurality of feature point sets;
  • the matching module 1304 is configured to perform matching in a preset face database according to the face topological structure feature and the face image feature to obtain a face recognition result.
  • the features of the face topology include:
  • a feature vector set where the feature vector in the feature vector set is used to indicate the relative positional relationship between any two feature point sets in the multiple feature point sets; or,
  • Feature matrix the elements in the feature matrix are used to indicate the relative positional relationship between any two feature point sets in the multiple feature point sets.
  • the determining module 1303 is also used to:
  • mapping relationship Constructing a mapping relationship between the multiple feature point sets and multiple feature point sets of a standard face, and the mapping relationship is used to determine the relative position relationship between the multiple feature point sets;
  • the obtaining module 1301 is specifically configured to input the mapping relationship into a pre-trained face topology structure feature extraction network to obtain the face topology structure feature.
  • the mapping relationship includes the distance and/or angle between the multiple feature point sets and the multiple feature point sets of the standard face.
  • the face topology feature extraction network is obtained after the first network is trained
  • the extraction module 1302 is also used to extract multiple facial geometric feature points from the face image training samples to determine multiple sample feature point sets.
  • Each feature point set in the multiple sample feature point sets corresponds to the training sample A face part, the sample feature point set includes at least one facial geometric feature point;
  • the acquiring module 1301 is further configured to acquire the mapping relationship between the sample feature point set and the feature point set of the standard face, and input it into the first network for training to obtain the first loss value;
  • the obtaining module 1301 is further configured to update the weight parameter in the first network according to the first loss value to obtain the face topology structure feature extraction network.
  • the extraction module 1302 is specifically used for:
  • the face image is input to the pre-trained overall face feature extraction network to extract the overall face feature.
  • the extraction module 1302 is specifically used for:
  • the first part feature extraction network is obtained after training of the second network, and the acquisition module 1301 is further configured to:
  • the obtaining module 1301 is further configured to update the weight parameter in the second network according to the second loss value to obtain the first part feature extraction network.
  • the extraction module 1302 is also used to:
  • the extraction module 1302 is specifically configured to input the multiple face part images into a pre-trained multiple part feature extraction network to extract multiple part features;
  • the determining module 1303 is further configured to determine the target part feature of the face image according to the multiple part features.
  • the target part feature is determined according to a weighted average of the multiple part features, and the weight of the multiple part features is a preset value.
  • the face recognition device further includes:
  • the detection module 1305 is configured to detect whether the face parts in the multiple face part images are occluded;
  • the determining module 1303 is also used to: if the first face part in the first face part image is occluded, and the second face part in the second face part image is not occluded, the second face part image
  • the face part is a symmetrical part of the first face part, then the horizontally flipped image of the second face part image is determined as the restored image of the first face part, and the restored image is used to input the
  • the part feature extraction network is used to extract the part feature.
  • the face recognition device further includes:
  • the update module 1306 is configured to update the weight value of the part feature of the first face part based on the occlusion of the first face part, and the updated first weight value is less than the preset first face part of the first face part. One weight.
  • the acquisition module 1301 is also used to:
  • the face image is preprocessed to obtain a preprocessed face image, the preprocessing includes face alignment, and the preprocessed face image is used to extract features of the face image and extract the multiple facial geometry Feature points.
  • FIG. 14 is a diagram of a chip hardware structure provided by an embodiment of the application.
  • the algorithm based on the convolutional neural network shown in FIG. 3 and FIG. 4 can be implemented in the NPU chip shown in FIG. 14.
  • Neural Network Processor NPU 50 The NPU is mounted as a coprocessor to the host CPU (Host CPU), and the Host CPU assigns tasks.
  • the core part of the NPU is the arithmetic circuit 50.
  • the arithmetic circuit 503 is controlled by the controller 504 to extract matrix data from the memory and perform multiplication operations.
  • the arithmetic circuit 503 includes multiple processing units (process engines, PE). In some implementations, the arithmetic circuit 503 is a two-dimensional systolic array. The arithmetic circuit 503 may also be a one-dimensional systolic array or other electronic circuit capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 503 is a general-purpose matrix processor.
  • the arithmetic circuit fetches the corresponding data of matrix B from the weight memory 502 and caches it on each PE in the arithmetic circuit.
  • the arithmetic circuit takes the matrix A data and matrix B from the input memory 501 to perform matrix operations, and the partial result or final result of the obtained matrix is stored in the accumulator 508.
  • the unified memory 506 is used to store input data and output data.
  • the weight data is directly transferred to the weight memory 502 through the storage unit access controller 505 (direct memory access controller, DMAC).
  • the input data is also transferred to the unified memory 506 through the DMAC.
  • the BIU is the Bus Interface Unit, that is, the bus interface unit 510, which is used for the interaction between the AXI bus and the DMAC and the instruction fetch buffer 509.
  • the bus interface unit 510 (bus interface unit, BIU for short) is used for the instruction fetch memory 509 to obtain instructions from an external memory, and is also used for the storage unit access controller 505 to obtain the original data of the input matrix A or the weight matrix B from the external memory.
  • the DMAC is mainly used to transfer the input data in the external memory DDR to the unified memory 506 or to transfer the weight data to the weight memory 502 or to transfer the input data to the input memory 501.
  • the vector calculation unit 507 may include multiple arithmetic processing units, if necessary, further processing the output of the arithmetic circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison and so on.
  • the vector calculation unit 507 can store the processed output vector in the unified buffer 506.
  • the vector calculation unit 507 may apply a nonlinear function to the output of the arithmetic circuit 503, such as a vector of accumulated values, to generate the activation value.
  • the vector calculation unit 507 generates a normalized value, a combined value, or both.
  • the processed output vector can be used as an activation input to the arithmetic circuit 503, for example for use in a subsequent layer in a neural network.
  • the instruction fetch buffer 509 connected to the controller 504 is used to store instructions used by the controller 504;
  • the unified memory 506, the input memory 501, the weight memory 502, and the fetch memory 509 are all On-Chip memories.
  • the external memory is private to the NPU hardware architecture.
  • each layer in the convolutional neural network shown in FIG. 3 and FIG. 4 may be executed by the matrix calculation unit 212 or the vector calculation unit 507.
  • FIG. 15 is a schematic diagram of another embodiment of a face recognition device in an embodiment of the application.
  • the face recognition apparatus provided in this embodiment may include a terminal or a server, and the specific device form is not limited in the embodiment of the present application.
  • the face recognition device 1500 may have relatively large differences due to different configurations or performances, and may include one or more processors 1501 and a memory 1502, and the memory 1502 stores programs or data.
  • the memory 1502 may be volatile storage or non-volatile storage.
  • the processor 1501 is one or more central processing units (CPU, Central Processing Unit, the CPU may be a single-core CPU or a multi-core CPU.
  • the processor 1501 may communicate with the memory 1502, in the face recognition device 1500 executes a series of instructions in the memory 1502.
  • the face recognition device 1500 also includes one or more wired or wireless network interfaces 1503, such as an Ethernet interface.
  • the face recognition apparatus 1500 may also include one or more power supplies; one or more input and output interfaces, which may be used to connect a display, a mouse, a keyboard, a touch screen device,
  • the input and output interfaces are optional components, which may or may not exist, and are not limited here.
  • the disclosed system, device, and method can be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of this application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium.
  • a computer device which may be a personal computer, a server, or a network device, etc.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disks or optical disks and other media that can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Geometry (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)

Abstract

本申请实施例公开了一种人脸识别方法，该方法可用于人工智能领域中。本申请实施例方法包括：获取待识别的人脸图像；根据所述人脸图像，通过预训练的特征提取网络，提取人脸图像特征；提取所述人脸图像中的多个面部几何特征点，以确定多个特征点集合，所述多个特征点集合中每个特征点集合对应一个人脸部位，所述特征点集合包括至少一个面部几何特征点；根据所述多个特征点集合获取人脸拓扑结构特征，所述人脸拓扑结构特征用于确定所述多个特征点集合之间的相对位置关系；根据所述人脸拓扑结构特征和所述人脸图像特征在预设的人脸数据库中进行匹配，以获取人脸识别结果。

Description

人脸识别方法和人脸识别装置
本申请要求于2020年01月16日提交中国专利局、申请号为202010051193.9、发明名称为“人脸识别方法和人脸识别装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及人工智能领域，特别涉及一种人脸识别方法和人脸识别装置。
背景技术
人脸识别是基于人的脸部特征信息进行身份识别的一种生物识别技术，是人工智能(artificial intelligence,AI)领域的重要应用。人脸识别利用分析比较人脸视觉特征信息进行身份鉴别。随着计算机和网络技术的飞速发展，人脸识别技术已广泛地应用于智能门禁、智能门锁、移动终端、公共安全、娱乐、军事等诸多行业及领域。
人脸识别包括，采集人脸图像，在图像中检测人脸，对检测到的人脸进行脸部的图像预处理，进而提取图像特征和识别匹配。其中，提取图像特征是指从图像提取包含结构和纹理的图像特征，后续的识别匹配基于该图像特征完成，图像特征的完整性是影响人脸识别成败的关键因素。
由于现有技术中图像特征基于整体人脸图像的结构和纹理信息提取，人脸识别效果依赖图像特征的完整性，当人脸图像受到外界干扰，例如发生光线不均，或帽子、围巾和口罩等遮挡时，将失去部分图像特征，图像特征不完整，导致人脸识别的成功率较低。
发明内容
本申请实施例提供了一种人脸识别方法，用于识别人脸图像，尤其是存在遮挡的人脸图像，可以提升人脸识别的成功率。
本申请实施例第一方面提供了一种人脸识别方法，包括：获取待识别的人脸图像；根据所述人脸图像，通过预训练的特征提取网络，提取人脸图像特征；提取所述人脸图像中的多个面部几何特征点，以确定多个特征点集合，所述多个特征点集合中每个特征点集合对应一个人脸部位，所述特征点集合包括至少一个面部几何特征点；根据所述多个特征点集合获取人脸拓扑结构特征，所述人脸拓扑结构特征用于确定所述多个特征点集合之间的相对位置关系；根据所述人脸拓扑结构特征和所述人脸图像特征在预设的人脸数据库中进行匹配，以获取人脸识别结果。
本申请实施例提供的人脸识别方法，在提取人脸图像特征之外，还提取了人脸拓扑结构特征，人脸拓扑结构特征用于表征人脸部位之间的拓扑结构，即相对位置关系，拓扑结构对图像的完整性依赖程度低，因此，基于人脸图像特征和人脸拓扑结构特征共同用于匹配识别，可以提高遮挡时的人脸识别成功率。
在第一方面的一种可能的实现方式中，所述人脸拓扑结构特征包括：特征向量集合，所述特征向量集合中的特征向量用于指示所述多个特征点集合中任意两个所述特征点集合之间的相对位置关系；或者，特征矩阵，所述特征矩阵中的元素用于指示所述多个特征点集合中 任意两个所述特征点集合之间的相对位置关系。
本申请实施例提供的人脸识别方法，可以直接根据多个特征点集合获取人脸拓扑结构特征，并通过特征向量集合和特征矩阵两种形式表达。
在第一方面的一种可能的实现方式中，根据所述多个人脸部位特征点集合获取人脸拓扑结构特征包括：构建所述多个特征点集合与标准人脸的多个特征点集合之间的映射关系，所述映射关系用于确定所述多个特征点集合之间的相对位置关系；将所述映射关系输入到预训练的人脸拓扑结构特征提取网络，以获取所述人脸拓扑结构特征。
本申请实施例提供的人脸识别方法，通过构建人脸图像和标准人脸的特征点的对应关系，根据映射关系数据以及已知的标准人脸结构信息，可以间接获取人脸图像中的特征点之间的相对位置关系，以及多个特征点集合之间的相对位置关系，将该映射关系数据输入到预训练网络，可以提取人脸拓扑结构特征。
在第一方面的一种可能的实现方式中，所述映射关系包括所述多个特征点集合与所述标准人脸的多个特征点集合之间的距离和/或角度。
在第一方面的一种可能的实现方式中，所述人脸拓扑结构特征提取网络为第一网络训练后得到，所述方法还包括：从人脸图像训练样本中提取多个面部几何特征点，以确定多个样本特征点集合，所述多个样本特征点集合中每个特征点集合对应所述训练样本的一个人脸部位，所述样本特征点集合包括至少一个面部几何特征点；获取所述样本特征点集合与标准人脸的特征点集合之间的映射关系，并输入到所述第一网络进行训练，获取第一损失值；根据所述第一损失值更新所述第一网络中的权重参数，以获取所述人脸拓扑结构特征提取网络。
本申请实施例提供的人脸识别方法，通过输入人脸图像训练样本的拓扑结构数据进行训练，可以获取人脸拓扑结构特征提取网络。
在第一方面的一种可能的实现方式中，其特征在于，根据所述人脸图像，通过预训练特征提取网络，提取人脸图像特征包括：将述人脸图像输入到预训练的人脸整体特征提取网络，以提取人脸整体特征。
本申请实施例提供的人脸识别方法，人脸图像特征可以包括人脸整体特征，该人脸整体特征即全局特征，例如图像颜色特征或图像纹理特征等，全局特征的提取依赖图像的完整性，当人脸图像中的部分人脸部位被遮挡时，根据提取的全局特征进行识别的成功率较低。本申请中结合人脸拓扑结构特征，共同用于特征匹配，可以提供人脸识别的成功率。
在第一方面的一种可能的实现方式中，所述方法还包括：从所述人脸图像中提取第一人脸部位图像；根据所述人脸图像，通过预训练特征提取网络，提取人脸图像特征包括：将所述第一人脸部位图像输入到预训练的第一部位特征提取网络，以提取第一部位特征，所述第一部位特征用于在所述人脸数据库中进行匹配，以获取所述人脸识别结果。
本申请实施例提供的人脸识别方法，人脸图像特征可以包括一个人脸部位的图像特征，即部位特征，由此可以提供人脸图像特征的又一种形式，提升了方案实现的多样性，该人脸部位例如为眼睛部位、鼻子部位或者嘴巴部位等。
在第一方面的一种可能的实现方式中，所述第一部位特征提取网络为第二网络训练后得到，所述方法还包括：将从人脸图像训练样本中提取的人脸部位图像输入到所述第二网络进行训练，获取第二损失值；根据所述第二损失值更新所述第二网络中的权重参数，以获取所 述第一部位特征提取网络。
本申请实施例提供的人脸识别方法，通过输入训练样本的人脸部位图像进行训练，可以获取人脸部位特征提取网络。
在第一方面的一种可能的实现方式中，所述方法还包括：从所述人脸图像中提取多个人脸部位图像；根据所述人脸图像，通过预训练特征提取网络，提取人脸图像特征包括：将所述多个人脸部位图像分别输入预训练的多个部位特征提取网络，以提取多个部位特征；根据所述多个部位特征确定所述人脸图像的目标部位特征。
本申请实施例提供的人脸识别方法，提取多个人脸部位，可以通过各个人脸部位的特征提取网络，分别提取多个部位特征，可以提升人脸识别的成功率。
在第一方面的一种可能的实现方式中，所述目标部位特征根据所述多个部位特征的加权平均值确定，所述多个部位特征的权值为预设值。
本申请实施例提供的人脸识别方法，由于人脸部位在人脸识别中的重要性不同，例如眼睛部位的重要程度高于眉毛部位，本方案中通过预设不同部位特征的权重，可以区分不同人脸部位的重要程度，提高人脸识别的成功率。
在第一方面的一种可能的实现方式中，所述方法还包括：检测所述多个人脸部位图像中的人脸部位是否被遮挡；若第一人脸部位图像中的第一人脸部位被遮挡，且第二人脸部位图像中的第二人脸部位未被遮挡，所述第二人脸部位为所述第一人脸部位的对称部位，则将所述第二人脸部位图像的水平翻转图像确定为所述第一人脸部位的恢复图像，所述恢复图像用于输入所述部位特征提取网络以提取所述部位特征。
本申请实施例提供的人脸识别方法，可以检查人脸部位是否被遮挡，对于一个部位被遮挡，且该部位的对称部位未被遮挡的情况，可以将图像进行翻转恢复，用于特征提取，提高了遮挡场景下的人脸识别成功率。
在第一方面的一种可能的实现方式中，所述方法还包括：基于所述第一人脸部位被遮挡，更新所述第一人脸部位的部位特征的权值，更新的第一权值小于所述第一人脸部位的预设第一权值。
本申请实施例提供的人脸识别方法，对于被遮挡的人脸部位，可以降低其部位特征的权值，由此，有效区分遮挡部位与未遮挡部位的重要程度，可以提高遮挡场景下的人脸识别成功率。
在第一方面的一种可能的实现方式中，所述方法还包括：对所述人脸图像进行预处理，以获取预处理后的人脸图像，所述预处理包括人脸对齐，所述预处理后的人脸图像用于提取所述人脸图像特征和提取所述多个面部几何特征点。
本申请实施例提供的人脸识别方法，人脸图像被用于特征提取前，可以先进行预处理，提高特征提取效率和特征准确程度。
本申请实施例第二方面提供了一种人脸识别装置，包括：获取模块，用于获取待识别的人脸图像；提取模块，用于根据所述人脸图像，通过预训练的特征提取网络，提取人脸图像特征；确定模块，用于提取所述人脸图像中的多个面部几何特征点，以确定多个特征点集合，所述多个特征点集合中每个特征点集合对应一个人脸部位，所述特征点集合包括至少一个面部几何特征点；所述获取模块，还用于根据所述多个特征点集合获取人脸拓扑结构特征，所 述人脸拓扑结构特征用于确定所述多个特征点集合之间的相对位置关系；匹配模块，用于根据所述人脸拓扑结构特征和所述人脸图像特征在预设的人脸数据库中进行匹配，以获取人脸识别结果。
在第二方面的一种可能的实现方式中，所述人脸拓扑结构特征包括：特征向量集合，所述特征向量集合中的特征向量用于指示所述多个特征点集合中任意两个所述特征点集合之间的相对位置关系；或者，特征矩阵，所述特征矩阵中的元素用于指示所述多个特征点集合中任意两个所述特征点集合之间的相对位置关系。
在第二方面的一种可能的实现方式中，所述确定模块还用于：构建所述多个特征点集合与标准人脸的多个特征点集合之间的映射关系，所述映射关系用于确定所述多个特征点集合之间的相对位置关系；所述获取模块具体用于：将所述映射关系输入到预训练的人脸拓扑结构特征提取网络，以获取所述人脸拓扑结构特征。
在第二方面的一种可能的实现方式中，所述映射关系包括所述多个特征点集合与所述标准人脸的多个特征点集合之间的距离和/或角度。
在第二方面的一种可能的实现方式中，所述人脸拓扑结构特征提取网络为第一网络训练后得到；所述提取模块还用于，从人脸图像训练样本中提取多个面部几何特征点，以确定多个样本特征点集合，所述多个样本特征点集合中每个特征点集合对应所述训练样本的一个人脸部位，所述样本特征点集合包括至少一个面部几何特征点；所述获取模块还用于，获取所述样本特征点集合与标准人脸的特征点集合之间的映射关系，并输入到所述第一网络进行训练，获取第一损失值；所述获取模块还用于，根据所述第一损失值更新所述第一网络中的权重参数，以获取所述人脸拓扑结构特征提取网络。
在第二方面的一种可能的实现方式中，所述提取模块具体用于：将述人脸图像输入到预训练的人脸整体特征提取网络，以提取人脸整体特征。
在第二方面的一种可能的实现方式中，所述提取模块具体用于：从所述人脸图像中提取第一人脸部位图像；将所述第一人脸部位图像输入到预训练的第一部位特征提取网络，以提取第一部位特征，所述第一部位特征用于在所述人脸数据库中进行匹配，以获取所述人脸识别结果。
在第二方面的一种可能的实现方式中，所述第一部位特征提取网络为第二网络训练后得到，所述获取模块还用于：将从人脸图像训练样本中提取的人脸部位图像输入到所述第二网络进行训练，获取第二损失值；所述获取模块还用于，根据所述第二损失值更新所述第二网络中的权重参数，以获取所述第一部位特征提取网络。
在第二方面的一种可能的实现方式中，所述提取模块还用于：从所述人脸图像中提取多个人脸部位图像；所述提取模块具体用于，将所述多个人脸部位图像分别输入预训练的多个部位特征提取网络，以提取多个部位特征；所述确定模块还用于，根据所述多个部位特征确定所述人脸图像的目标部位特征。
在第二方面的一种可能的实现方式中，所述目标部位特征根据所述多个部位特征的加权平均值确定，所述多个部位特征的权值为预设值。
在第二方面的一种可能的实现方式中，所述人脸识别装置还包括：检测模块，用于检测所述多个人脸部位图像中的人脸部位是否被遮挡；所述确定模块还用于，若第一人脸部位图 像中的第一人脸部位被遮挡，且第二人脸部位图像中的第二人脸部位未被遮挡，所述第二人脸部位为所述第一人脸部位的对称部位，则将所述第二人脸部位图像的水平翻转图像确定为所述第一人脸部位的恢复图像，所述恢复图像用于输入所述部位特征提取网络以提取所述部位特征。
在第二方面的一种可能的实现方式中，所述人脸识别装置还包括：更新模块，用于基于所述第一人脸部位被遮挡，更新所述第一人脸部位的部位特征的权值，更新的第一权值小于所述第一人脸部位的预设第一权值。
在第二方面的一种可能的实现方式中，所述获取模块还用于：对所述人脸图像进行预处理，以获取预处理后的人脸图像，所述预处理包括人脸对齐，所述预处理后的人脸图像用于提取所述人脸图像特征和提取所述多个面部几何特征点。
本申请实施例第二方面提供了一种人脸识别装置，包括处理器和存储器，所述处理器和所述存储器相互连接，其中，所述存储器用于存储计算机程序，所述计算机程序包括程序指令，所述处理器用于调用所述程序指令，执行如上述第一方面以及各种可能的实现方式中任一项所述的方法。
本申请实施例第三方面提供了一种包含指令的计算机程序产品，其特征在于，当其在计算机上运行时，使得所述计算机执行如上述第一方面以及各种可能的实现方式中任一项所述的方法。
本申请实施例第四方面提供了一种计算机可读存储介质，包括指令，其特征在于，当所述指令在计算机上运行时，使得计算机执行如上述第一方面以及各种可能的实现方式中任一项所述的方法。
本申请实施例第五方面提供了一种一种芯片，包括处理器。处理器用于读取并执行存储器中存储的计算机程序，以执行上述任一方面任意可能的实现方式中的方法。可选地，该芯片该包括存储器，该存储器与该处理器通过电路或电线与存储器连接。进一步可选地，该芯片还包括通信接口，处理器与该通信接口连接。通信接口用于接收需要处理的数据和/或信息，处理器从该通信接口获取该数据和/或信息，并对该数据和/或信息进行处理，并通过该通信接口输出处理结果。该通信接口可以是输入输出接口。
从以上技术方案可以看出，本申请实施例具有以下优点：
本申请实施例提出的人脸识别方法，将人脸图像输入预训练的特征提取网络获取人脸图像特征，此外，还通过提取人脸图像中的多个面部集合特征点，确定对应于多个人脸部位的多个人脸部位特征点集合，根据多个人脸部位特征点集合获取人脸拓扑结构特征；根据人脸拓扑结构特征和人脸图像特征在人脸数据集进行特征匹配，最终获取人脸识别结果，由于人脸拓扑结构由多个人脸部位特征点集合的相对位置关系构建，可以提取获得更多的结构化信息，对整体人脸图像的完整性依赖降低，可以有效降低遮挡对人脸识别的影响。
附图说明
图1为本申请实施例提供的一种人工智能主体框架示意图；
图2为本申请实施例提供的一种应用环境示意图；
图3为本申请实施例提供的一种卷积神经网络结构示意图；
图4为本申请实施例提供的另一种卷积神经网络结构示意图；
图5为本申请实施例的系统架构图；
图6为本申请实施例中一种人脸识别方法的实施例示意图；
图7为本申请实施例中目标集合特征点的示意图；
图8为本申请实施例中人脸拓扑结构的示意图；
图9为本申请实施例中标准人脸结构的示意图；
图10为本申请实施例中人脸部位图像以及遮挡翻转处理的示意图；
图11为本申请实施例中特征提取网络的训练方法的实施例示意图；
图12为本申请实施例中特征提取网络架构的示意图；
图13为本申请实施例中人脸识别装置的一个实施例示意图；
图14为本申请实施例提供的一种芯片硬件结构图；
图15为本申请实施例中人脸识别装置的另一个实施例示意图。
具体实施方式
本申请实施例提供了一种人脸识别方法，用于识别人脸图像，尤其是存在遮挡的人脸图像，可以提升人脸识别的成功率。
本申请涉及的术语简介：
人脸图像：包含人脸信息的图像；
人脸几何特征：人脸由眼睛、鼻子和嘴巴等部位构成，对这些部位的形状和结构关系的几何描述，可以做为人脸识别的重要特征，这些特征即人脸几何特征。
面部几何特征点：人脸由眼睛、鼻子和嘴巴等部位构成，通过对人脸图像的检测，可以提取用于表征各个人脸部位的特征点，即面部几何特征点。
人脸部位图像：指人脸图像中包括特征性的局部区域的图像，通常指眼睛、眉毛、鼻子或嘴巴等部位的图像。
人脸拓扑结构：拓扑就是把实体抽象成与其大小、形状无关的“点”，而把连接实体的线路抽象成“线”，进而以图的形式来表示这些点与线之间关系的方法，其目的在于研究这些点、线之间的相连关系。表示点和线之间关系的图被称为拓扑结构图。拓扑结构与几何结构属于两个不同的数学概念。在几何结构中，我们要考察的是点、线之间的位置关系，或者说几何结构强调的是点与线所构成的形状及大小。如梯形、正方形、平行四边形及圆都属于不同的几何结构，但从拓扑结构的角度去看，由于点、线间的连接关系相同，从而具有相同的拓扑结构即环型结构。也就是说，不同的几何结构可能具有相同的拓扑结构。人脸拓扑结构包括人脸的各个部位之间的相连关系。
下面结合附图，对本申请的实施例进行描述，显然，所描述的实施例仅仅是本申请一部分的实施例，而不是全部的实施例。本领域普通技术人员可知，随着技术的发展和新场景的出现，本申请实施例提供的技术方案对于类似的技术问题，同样适用。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的实施例能够以除了在这里图示或描述的内容以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了 一系列步骤或模块的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或模块，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或模块。在本申请中出现的对步骤进行的命名或者编号，并不意味着必须按照命名或者编号所指示的时间/逻辑先后顺序执行方法流程中的步骤，已经命名或者编号的流程步骤可以根据要实现的技术目的变更执行次序，只要能达到相同或者相类似的技术效果即可。
图1示出一种人工智能主体框架示意图，该主体框架描述了人工智能系统总体工作流程，适用于通用的人工智能领域需求。
下面从“智能信息链”(水平轴)和“IT价值链”(垂直轴)两个维度对上述人工智能主题框架进行阐述。
“智能信息链”反映从数据的获取到处理的一列过程。举例来说，可以是智能信息感知、智能信息表示与形成、智能推理、智能决策、智能执行与输出的一般过程。在这个过程中，数据经历了“数据—信息—知识—智慧”的凝练过程。
“IT价值链”从人智能的底层基础设施、信息(提供和处理技术实现)到系统的产业生态过程，反映人工智能为信息技术产业带来的价值。
(1)基础设施：
基础设施为人工智能系统提供计算能力支持，实现与外部世界的沟通，并通过基础平台实现支撑。通过传感器与外部沟通；计算能力由智能芯片(CPU、NPU、GPU、ASIC、FPGA等硬件加速芯片)提供；基础平台包括分布式计算框架及网络等相关的平台保障和支持，可以包括云存储和计算、互联互通网络等。举例来说，传感器和外部沟通获取数据，这些数据提供给基础平台提供的分布式计算系统中的智能芯片进行计算。
(2)数据
基础设施的上一层的数据用于表示人工智能领域的数据来源。数据涉及到图形、图像、语音、文本，还涉及到传统设备的物联网数据，包括已有系统的业务数据以及力、位移、液位、温度、湿度等感知数据。
(3)数据处理
数据处理通常包括数据训练，机器学习，深度学习，搜索，推理，决策等方式。
其中，机器学习和深度学习可以对数据进行符号化和形式化的智能信息建模、抽取、预处理、训练等。
推理是指在计算机或智能系统中，模拟人类的智能推理方式，依据推理控制策略，利用形式化的信息进行机器思维和求解问题的过程，典型的功能是搜索与匹配。
决策是指智能信息经过推理后进行决策的过程，通常提供分类、排序、预测等功能。
(4)通用能力
对数据经过上面提到的数据处理后，进一步基于数据处理的结果可以形成一些通用的能力，比如可以是算法或者一个通用系统，例如，翻译，文本的分析，计算机视觉的处理，语音识别，图像的识别等等。
(5)智能产品及行业应用
智能产品及行业应用指人工智能系统在各领域的产品和应用，是对人工智能整体解决方案的封装，将智能信息决策产品化、实现落地应用，其应用领域主要包括：智能制造、智能 交通、智能家居、智能医疗、智能安防、自动驾驶，平安城市，智能终端等。
参见附图2，本申请实施例提供了一种系统架构200。数据采集设备260用于采集人脸图像数据并存入数据库230，训练设备220基于数据库230中维护的人脸图像数据生成目标模型/规则201。下面将更详细地描述训练设备220如何基于人脸图像数据得到目标模型/规则201，目标模型/规则201能够用于人脸识别、图像分类和虚拟现实等应用场景。
本申请实施例中，可以基于人脸图像数据进行训练，具体的，可以通过数据采集设备260采集各种人脸图像，包括存在遮挡的人脸图像，并存入数据库230。此外，还可以直接从常用的数据库，如LFW、YaleB、CMU PIE、CFW、Celeba等数据库获取人脸图像数据。
该目标模型/规则201可以是基于深度神经网络得到的，下面对深度神经网络进行介绍。
深度神经网络中的每一层的工作可以用数学表达式
Figure PCTCN2020105772-appb-000001
来描述：从物理层面深度神经网络中的每一层的工作可以理解为通过五种对输入空间(输入向量的集合)的操作，完成输入空间到输出空间的变换(即矩阵的行空间到列空间)，这五种操作包括：1、升维/降维；2、放大/缩小；3、旋转；4、平移；5、“弯曲”。其中1、2、3的操作由
Figure PCTCN2020105772-appb-000002
完成，4的操作由+b完成，5的操作则由a()来实现。这里之所以用“空间”二字来表述是因为被分类的对象并不是单个事物，而是一类事物，空间是指这类事物所有个体的集合。其中，W是权重向量，该向量中的每一个值表示该层神经网络中的一个神经元的权重值。该向量W决定着上文所述的输入空间到输出空间的空间变换，即每一层的权重W控制着如何变换空间。训练深度神经网络的目的，也就是最终得到训练好的神经网络的所有层的权重矩阵(由很多层的向量W形成的权重矩阵)。因此，神经网络的训练过程本质上就是学习控制空间变换的方式，更具体的就是学习权重矩阵。
因为希望深度神经网络的输出尽可能的接近真正想要预测的值，所以可以通过比较当前网络的预测值和真正想要的目标值，再根据两者之间的差异情况来更新每一层神经网络的权重向量(当然，在第一次更新之前通常会有初始化的过程，即为深度神经网络中的各层预先配置参数)，比如，如果网络的预测值高了，就调整权重向量让它预测低一些，不断的调整，直到神经网络能够预测出真正想要的目标值。因此，就需要预先定义“如何比较预测值和目标值之间的差异”，这便是损失函数(loss function)或目标函数(objective function)，它们是用于衡量预测值和目标值的差异的重要方程。其中，以损失函数举例，损失函数的输出值(loss)越高表示差异越大，那么深度神经网络的训练就变成了尽可能缩小这个loss的过程。
训练设备220得到的目标模型/规则可以应用不同的系统或设备中。在附图2中，执行设备210配置有I/O接口212，与外部设备进行数据交互，“用户”可以通过客户设备240向I/O接口212输入数据。
执行设备210可以调用数据存储系统250中的数据、代码等，也可以将数据、指令等存入数据存储系统250中。
计算模块211使用目标模型/规则201对输入的数据进行处理，以人脸图像识别为例，计算模块211可以对输入的人脸图像进行解析，得到人脸图像中的纹理信息等图像特征。
关联功能模块213可以对计算模块211中的图像数据进行预处理，例如进行人脸图像预处理，包括人脸对齐等。
关联功能模块214可以对计算模块211中的图像数据进行预处理，例如进行人脸图像预处理，包括人脸对齐等。
最后，I/O接口212将处理结果返回给客户设备240，提供给用户。
更深层地，训练设备220可以针对不同的目标，基于不同的数据生成相应的目标模型/规则201，以给用户提供更佳的结果。
在附图2中所示情况下，用户可以手动指定输入执行设备210中的数据，例如，在I/O接口212提供的界面中操作。另一种情况下，客户设备240可以自动地向I/O接口212输入数据并获得结果，如果客户设备240自动输入数据需要获得用户的授权，用户可以在客户设备240中设置相应权限。用户可以在客户设备240查看执行设备210输出的结果，具体的呈现形式可以是显示、声音、动作等具体方式。客户设备240也可以作为数据采集端将采集到训练数据存入数据库230。
值得注意的，附图2仅是本申请实施例提供的一种系统架构的示意图，图中所示设备、器件、模块等之间的位置关系不构成任何限制，例如，在附图2中，数据存储系统250相对执行设备210是外部存储器，在其它情况下，也可以将数据存储系统250置于执行设备210中。
卷积神经网络(convolutional neural network，CNN)是一种带有卷积结构的深度神经网络，是一种深度学习(deep learning)架构，深度学习架构是指通过机器学习的算法，在不同的抽象层级上进行多个层次的学习。作为一种深度学习架构，CNN是一种前馈(feed-forward)人工神经网络，以图像处理为例，该前馈人工神经网络中的各个神经元对输入其中的图像中的重叠区域作出响应。
如图3所示，卷积神经网络(CNN)100可以包括输入层110，卷积层/池化层120，其中池化层为可选的，以及神经网络层130。
卷积层/池化层120：
卷积层：
如图3所示卷积层/池化层120可以包括如示例121-126层，在一种实现中，121层为卷积层，122层为池化层，123层为卷积层，124层为池化层，125为卷积层，126为池化层；在另一种实现方式中，121、122为卷积层，123为池化层，124、125为卷积层，126为池化层。即卷积层的输出可以作为随后的池化层的输入，也可以作为另一个卷积层的输入以继续进行卷积操作。
以卷积层121为例，卷积层121可以包括很多个卷积算子，卷积算子也称为核，其在图像处理中的作用相当于一个从输入图像矩阵中提取特定信息的过滤器，卷积算子本质上可以是一个权重矩阵，这个权重矩阵通常被预先定义，在对图像进行卷积操作的过程中，权重矩阵通常在输入图像上沿着水平方向一个像素接着一个像素(或两个像素接着两个像素……这取决于步长stride的取值)的进行处理，从而完成从图像中提取特定特征的工作。该权重矩阵的大小应该与图像的大小相关，需要注意的是，权重矩阵的纵深维度(depth dimension)和输入图像的纵深维度是相同的，在进行卷积运算的过程中，权重矩阵会延伸到输入图像的整个深度。因此，和一个单一的权重矩阵进行卷积会产生一个单一纵深维度的卷积化输出，但是大多数情况下不使用单一权重矩阵，而是应用维度相同的多个权重矩阵。每个权重矩阵 的输出被堆叠起来形成卷积图像的纵深维度。不同的权重矩阵可以用来提取图像中不同的特征，例如一个权重矩阵用来提取图像边缘信息，另一个权重矩阵用来提取图像的特定颜色，又一个权重矩阵用来对图像中不需要的噪点进行模糊化……该多个权重矩阵维度相同，经过该多个维度相同的权重矩阵提取后的特征图维度也相同，再将提取到的多个维度相同的特征图合并形成卷积运算的输出。
这些权重矩阵中的权重值在实际应用中需要经过大量的训练得到，通过训练得到的权重值形成的各个权重矩阵可以从输入图像中提取信息，从而帮助卷积神经网络100进行正确的预测。
当卷积神经网络100有多个卷积层的时候，初始的卷积层(例如121)往往提取较多的一般特征，该一般特征也可以称之为低级别的特征；随着卷积神经网络100深度的加深，越往后的卷积层(例如126)提取到的特征越来越复杂，比如高级别的语义之类的特征，语义越高的特征越适用于待解决的问题。
池化层：
由于常常需要减少训练参数的数量，因此卷积层之后常常需要周期性的引入池化层，即如图3中120所示例的121-126各层，可以是一层卷积层后面跟一层池化层，也可以是多层卷积层后面接一层或多层池化层。在图像处理过程中，池化层的唯一目的就是减少图像的空间大小。池化层可以包括平均池化算子和/或最大池化算子，以用于对输入图像进行采样得到较小尺寸的图像。平均池化算子可以在特定范围内对图像中的像素值进行计算产生平均值。最大池化算子可以在特定范围内取该范围内值最大的像素作为最大池化的结果。另外，就像卷积层中用权重矩阵的大小应该与图像大小相关一样，池化层中的运算符也应该与图像的大小相关。通过池化层处理后输出的图像尺寸可以小于输入池化层的图像的尺寸，池化层输出的图像中每个像素点表示输入池化层的图像的对应子区域的平均值或最大值。
神经网络层130：
在经过卷积层/池化层120的处理后，卷积神经网络100还不足以输出所需要的输出信息。因为如前所述，卷积层/池化层120只会提取特征，并减少输入图像带来的参数。然而为了生成最终的输出信息(所需要的类信息或别的相关信息)，卷积神经网络100需要利用神经网络层130来生成一个或者一组所需要的类的数量的输出。因此，在神经网络层130中可以包括多层隐含层(如图3所示的131、132至13n)以及输出层140，该多层隐含层中所包含的参数可以根据具体的任务类型的相关训练数据进行预先训练得到，例如该任务类型可以包括图像识别，图像分类，图像超分辨率重建等等。
在神经网络层130中的多层隐含层之后，也就是整个卷积神经网络100的最后层为输出层140，该输出层140具有类似分类交叉熵的损失函数，具体用于计算预测误差，一旦整个卷积神经网络100的前向传播(如图3由110至140的传播为前向传播)完成，反向传播(如图3由140至110的传播为反向传播)就会开始更新前面提到的各层的权重值以及偏差，以减少卷积神经网络100的损失及卷积神经网络100通过输出层输出的结果和理想结果之间的误差。
需要说明的是，如图3所示的卷积神经网络100仅作为一种卷积神经网络的示例，在具体的应用中，卷积神经网络还可以以其他网络模型的形式存在，例如，如图4所示的多个卷 积层/池化层并行，将分别提取的特征均输入给全神经网络层130进行处理。
下面结合图5介绍本申请实施例中人脸识别方法的系统架构图；
本申请实施例提供的人脸识别方法适用于居家、安防等各种场景下的人脸识别，包括机器人、智能手机、台式电脑、平板电脑、电视、家用或公共安全监控摄像头、照相机、门禁、门锁、考勤机、智能眼镜等产品的身份验证、个性化定制、表情模拟等使用场景。人脸识别的过程可以由上述实体完成，也可以通过网络，与专用的服务器连接，由服务器完成，具体此处不做限定。
对于通过服务器进行人脸识别的场景，其中，网络包括多种类型的无线或部分无线通信网络中的一个或多个，例如局域网(LAN)、无线局域网(WLAN)、个人局域网(PAN)、广域网(WAN)、内联网、互联网、对等网络、点对点网络或网状网络等，具体此处不做限定。
下面结合图6对本申请实施例的人脸识别方法进行详细的介绍。
601、获取人脸图像；
人脸识别装置获取人脸图像，可选的，人脸识别装置通过内置或外设的摄像头拍摄图像，然后从拍摄图像中检测到包括人脸信息的人脸图像。
示例性的，机器人通过摄像头采集图像并探测图像中包括人脸，即可获取人脸图像。
602、对人脸图像进行预处理；
人脸识别装置对人脸图像进行预处理，原始的人脸图像由于受到各种条件的限制和随机干扰，往往不能直接使用，必须在图像处理的早期阶段对它进行灰度矫正、噪声过滤等图像预处理，获取预处理后的图像，用于后续特征提取。
人脸图像预处理可以包括：人脸对齐、光线补偿、灰度变换、直方图均衡化、归一化处理、几何校正、中值滤波以及锐化等。具体处理流程此处不做限定，其中，归一化处理用于取得尺寸一致、灰度取值范围相同的标准化人脸图像，中值滤波可用于图片的平滑操作以消除噪声。
可选的，对人脸图像进行人脸对齐处理，根据人脸特征点的位置，将不同尺度，不同方向的人脸归一化到统一尺度，得到人脸位置端正的图像。包括：
1、人脸框归一化到统一尺度，图像中人脸的范围可以由人脸框区域标识，人脸框大小与图像拍摄时人体与摄像头之间的距离有关，距离近，则人脸大，距离远，则人脸小。通过人脸对齐可以将人脸框归一化到统一尺度，目的是便于后续的特征提取；
2、人脸旋转，由于人脸图像中的人可能有不同的姿态，例如检测到的人脸可能是正脸，可能是侧脸，通过人脸对齐，进行人脸旋转，将不同姿态的人脸尽可能转到同一角度，便于识别，可选的，旋转至鼻子正向；
人脸对齐可以减少距离或姿态对后续特征提取的影响，在统一尺度上识别人脸。
603、获取人脸拓扑结构特征；
人脸识别装置检测待识别人脸图像中的面部几何特征点，面部几何特征点包括用于表征各个人脸部位的特征点，例如眼睛、鼻子和嘴巴和面部轮廓等。
人脸识别装置可以通过预设算法检测面部几何特征点，此处对于预设算法的具体类型不做限定。此外，提取面部几何特征点的数量不做限定，可以提取68个特征点，或者128个特征点，示例性的，请参阅图7，为本申请实施例中面部几何特征点的示意图，图中示出了68 个面部几何特征点。
根据目标几何特征点，可以确定多个人脸部位特征点集合，例如图7中，面部几何特征点18至22组成的人脸部位特征点集合用于指示左眉部位701；面部几何特征点23至27组成的人脸部位特征点集合用于指示右眉部位702；面部几何特征点37至42组成的人脸部位特征点集合用于指示左眼部位703；面部几何特征点43至48组成的人脸部位特征点集合用于指示右眼部位704；面部几何特征点28至36组成的人脸部位特征点集合用于指示鼻子部位705；面部几何特征点49至68组成的人脸部位特征点集合用于指示嘴巴部位706。
由于人脸各个部位的相对位置关系固定，以鼻子为中心发散分布，利用此先验信息和检测到的面部几何特征点，计算各个人脸部位特征点集合之间的连接关系，构建人脸拓扑结构，获取人脸拓扑结构特征。
获取人脸拓扑结构特征的方法有多种：
可选的，人脸拓扑结构特征表示为特征向量集合，特征向量集合中的特征向量用于指示所述多个人脸部位特征点集合中任意两个人脸部位特征点集合之间的相对位置关系。请参阅图8，为本申请实施例中人脸拓扑结构的示意图。
P表示人脸部位特征点集合，向量ci，j表示人脸部位特征点集合i和j之间的连接关系。
人脸拓扑结构表示为[c0,1，c0,2，c0,5，c1,2，c1,3，c3,4，c2,4，c2,5，c1,5]，示例性的，向量c0,1可以根据P0的位置坐标，P1的位置坐标得到。向量表示法仅列出了具有连接关系的特征关系，可以理解的是，不直接具有连接关系的两个人脸部位，也可以根据拓扑结构信息间接获取其连接关系。
对人脸结构特征进行归一化，归一化是指将特征采用统一的度量单位，度量标准，如采用以鼻子为原点，眼睛到鼻子的距离为C0,1，这个距离可以是图像坐标系为计算体系，也可进一步将距离归一化到0至1之间。
可选的，人脸拓扑结构特征表示为特征矩阵，所述特征矩阵中的元素用于指示所述多个人脸部位特征点集合中任意两个人脸部位特征点集合之间的相对位置关系，如下所示，其中没有连接关系的cij＝0，例如上图中的p1和p4之间没有连接关系，那C1,4可以表示成0。
Figure PCTCN2020105772-appb-000003
需要说明的是，除了图8中示意的连接关系之外，还可以定义其他的连接关系，如欧氏距离、曲率或角度等，具体此处不做限定。
可选的，构建所述多个人脸部位特征点集合与标准人脸的人脸部位特征点集合之间的映射关系，将所述映射关系输入到预训练的人脸拓扑结构特征提取网络，以获取人脸拓扑结构特征。人脸图像中各人脸部位与标准人脸中对应的各人脸部位之间的映射关系可以用距离和/或角度等度量关系来表达，用于指示所述多个人脸部位特征点集合之间的相对位置关系。
标准人脸是用来作参考标准的人脸图像，请参阅图9，在中间的标准人脸上，对眼睛、鼻子、眉毛、嘴巴等部位都标注了至少一个特征点，例如鼻子部位可以用1个特征点进行标注，也可以用多个特征点进行标注，每个部位标注的特征点数量具体不做限定，可以由开发 人员指定。在已知标准人脸的特征点拓扑结构的情况下，根据待识别人脸图像中人脸部位特征点集合与标准人脸上各个部位的特征点集合的映射关系，可以确定待识别人脸图像中各个特征点的位置关系，多个人脸部位特征点集合之间的相对位置关系。
映射关系可以用距离和/或角度等度量关系来表达，如待识别人脸图像中检测的面部几何特征点中，鼻子部位特征点集合中的一个特征点，与标准人脸的鼻子部位特征点集合中的一个特征点之间的度量关系是a。
示例性的，假设标准人脸中面部几何特征点数量为68，对于待识别人脸图像中检测到面部几何特征点数量为68，在平面坐标系中，一个特征点可以用二维数据标识。待识别人脸图像中一个特征点，与标准人脸图像中68个点之间的位置关系的数据量为1*68*2，待识别人脸图像中68个特征点与标准人脸图像中68个点之间的位置关系的数据量为68*68*2的数据块。
将表征映射关系的数据输入到预训练的人脸拓扑结构特征提取网络，可以获取人脸拓扑结构特征。
604、提取人脸部位图像；
人脸识别装置可以从人脸图像中确定各人脸部位。
从人脸图像中确定各人脸部位的方法有多种，可以先提取几何特征点，进而从人脸图像中分割出人脸部位。也可以通过预训练的人脸部位判别模型，用于提取人脸部位。对于从脸图像中确定各人脸部位的方法，具体此处不做限定。
可选的，从图像中提取的人脸部位包括：嘴、鼻、左眼、左眉、右眼和右眉，进行区分的人脸部位可以预先设定，此处不做限定。请参阅图10，为本申请实施例中人脸部位图像以及遮挡翻转处理的示意图。
可选的，根据预设的针对每个人脸部位的部位判别模型确定各个人脸部位是否被遮挡。若被遮挡部位为成对部位中的一个，且成对部位中的另一个未被遮挡，例如，图10所示，眉毛或者眼睛被遮挡一个，则将成对部位中未被遮挡的另一个部位水平翻转，作为当前被遮挡部位的恢复图像。
需要说明的是，步骤604与步骤603之间的执行顺序不做限定，可以先执行步骤603，再执行步骤604，也可以先执行步骤604，再执行步骤603。
605、获取人脸图像特征；
将步骤602中经过预处理的完整人脸图像输入由第一网络训练得到的人脸整体特征提取网络进行特征提取，输出人脸整体特征；
将步骤604中获取的人脸部位图像输入预训练的部位特征提取网络进行特征提取，输出目标部位特征；需要说明的是，每个人脸部位都可以预设对应的特征提取网络，对于人脸的多个部位，可以分别预设不同的部位特征提取网络进行特征提取，然后综合针对各个部位提取的单个部位特征得到目标部位特征。
可选的，目标部位特征为所述多个部位特征的加权平均值，所述多个部位特征的权值为预设值。可选的，人脸识别装置检测将多个人脸部位图像中的人脸部位是否被遮挡，对于被遮挡的部位，可以在该部位的初始预设权值的基础上降低权值，由此，可以降低部位遮挡对人脸识别带来的影响。
人脸整体特征提取网络、人脸拓扑结构特征提取网络和部位特征提取网络的训练过程请参考图11对应的实施例，此处不做具体介绍。
606、进行特征比对，获取人脸识别结果。
特征匹配模块将当前人脸的特征与人脸数据库中的人脸特征进行匹配，根据特征之间的相似性度量，获取人脸识别的结果。
本申请实施例提供的人脸识别方法，把人脸识别问题，划分为整体脸部识别、部位识别和基于结构化信息的特征识别；将任务解剖，能一定程度的减小网络体量，减少网络深度，增大宽度，使特征提取过程更加有效率，有效果；
针对部位提取特征，降低低响应区的影响，增强高响应区的感受野，学习更多的纹理信息，来增强特征的判断性和鲁棒性。部位识别时采用具有权重分布的策略，降低低响应部位对最终判别结果的影响，提高高响区的影响，利于人脸的识别；
当部位遮挡，该部位信息无效时，寻求匹配的人脸部位翻转，提取特征，遮挡判别器的加入，降低遮挡部位对人脸识别的影响。
提出专门学习人脸拓扑结构的特征提取单元，学习人脸与标准人脸之间的差异性，及其各个拓扑节点之间的链接关系，通过链接各层之间的信息，获得更多的结构化信息，可以在不影响正常的人脸识别的前提下，降低遮挡对人脸识别的影响。
下面介绍本申请提供的人脸识别方法中特征提取网络的训练方法，请参阅图11，该方法包括：
1101、从人脸数据库获取人脸图像；
人脸识别装置从人脸数据库获取人脸图像，人脸数据库可以从网上下载，也可以自己建立一部分数据库，具体此处不做限定。常用的人脸数据库有LFW数据集、Yale系列等。
可选的，人脸数据库中包括有遮挡的人脸图像，或者对人脸数据库中的人脸图像进行遮挡处理，生成有遮挡的人脸图像，具体此处不做限定。
1102、人脸图像预处理；
1103、提取人脸部位图像；
步骤1102至步骤1103与图6对应的实施例中步骤602和步骤604的方法类似，此处不再赘述。
1104、构建人脸拓扑结构；
请参考图6对应的实施例中步骤603，构建所述多个人脸部位特征点集合与标准人脸的人脸部位特征点集合之间的映射关系，获取表征映射关系的数据，代表人脸拓扑结构。
1105、将人脸图像、部位图像和人脸拓扑结构分别输入不同网络进行训练；
分别将人脸图像、部位图像、人脸拓扑数据分别输入到网络1、网络2、网络3中进行训练。请参阅图12，为本申请实施例中特征提取网络架构的示意图。
1、网络1训练得到的人脸整体特征提取网络的训练过程，对应于图12中标识1部分；
第一网络可以为现有的各种人脸整体特征提取网络，具体此处不做限定。
示例性的，输入图像数据为H*W*3的人脸图像，其中H为图像的高，W为图像宽，3代表RGB(red green blue)3通道，可选的，将多个尺度的图像作为训练的输入，可以使得图像在多尺度下具有鲁棒性。
输入图像经过9个卷积层，图12中每个块包括3个卷积层，输出[T*1]向量。
全连接层后面接softmax层(图中未示出)。这个层的输入是将[T*1]向量输入softmax层，输出也是[T*1]向量。将输出的每个向量都归一化到[0,1]之间。这里的softmax输出的向量是该样本属于每一类的概率。
第一网络损失函数L1如下：
Figure PCTCN2020105772-appb-000004
其中Ls是类间差异softmax loss，Lc是类内个体差异center loss，其中m为批大小(batch_size)，Xi是[T*1]的向量的第i个值，Xj是第j个值，wi是卷积的权值，由反向传播学习而来，bi是对应的偏置。
输入图像经过网络1，得到特征矩阵(1*W维)；每张图像块对应的得分，该得分为[0,1]范围内的实数。
2、网络2训练得到的部位特征提取网络的训练过程，对应于图12中标识2、3部分。
不同人脸部位对人脸识别的重要性是不一样的，一般而言，眼睛的权重最大，鼻子、嘴巴次之，眉毛的权重最小。
同时，部位是否被遮挡也影响人脸识别的效果，如果某个部位被遮挡，则应降低该部位的权重。
在本步骤中，每个部位网络的输入为H*W*3的部位图像，经过部位判别器，将每个部位的图片进行处理，并得到各人脸部位是否被遮挡的结果。
如上图所示为部位网络的示例，部位网络可采用小网络，针对部位提取特征，降低低响应区的影响，增强高响应区的感受野，学习更多的高分辨率纹理信息，来增强特征的判断性和鲁棒性。
根据不同人脸部位的响应特性，为每个部位设置初始权值，可以由产品开发人员自行定义，如眼睛的权重为0.4，鼻子、嘴巴为0.25，眉毛的权重为0.1。
经过部位判别器处理后的当前部位的图像数据和部位是否被遮挡的结果经过上述部位网络得到N*1维的特征向量和每个部位最后的部位权值wi。
如果部位被遮挡，在初始权值的基础上进行下降调整，例如：若眼睛被遮挡，则经过部位网络后，眼睛部位的权值被调整为0。
损失函数如下：
Figure PCTCN2020105772-appb-000005
其中，wi是卷积的权值，k代表提取的人脸部位数量，Li是每个部位的计算loss函数，计算公式如下：
Figure PCTCN2020105772-appb-000006
其中θ j，i为向量W j，x i之间的角度，θ i，j∈(0，π)，余弦代表特征向量与权重向量的角度余弦值。参数m的意义是使得类间的距离足够大，使类内距离足够小。
3、网络3训练得到的人脸拓扑结构特征提取网络，对应于图12中标识4部分；
采用标准人脸方式构建人脸拓扑结构时，获取表征映射关系的数据，将该数据输入到网络3进行训练，获取人脸拓扑结构特征提取网络，可以用于获取人脸拓扑结构特征。
网络3训练可以得到人脸拓扑结构特征提取网络，以68点人脸特征点2维度量关系距离，网络输入就是是68*68*2大小的数据块，经过网络卷积层和全连接层，提取构建人脸和标准人脸之间的差异化特征，输出是M*1维的特征向量。
网络3的损失函数L3如下：
Figure PCTCN2020105772-appb-000007
其中其中m为批大小(batch_size)，n为XX，Xi是[T*1]的向量的第i个值，Xj是第j个值，wi是卷积的权值，wj代表XX，由反向传播学习而来，bi是第i个值对应的偏置，bj是第j个值对应的偏置。
4、将网络1的输出Si，网络2的输出Sj，网络3的输出Sk，整合成一个输出。
L total＝S iL 1+S jL 2+S kL 3
可选的，训练过程也可以不训练部位特征(网络2)，在识别的过程中，使用人脸整体特征和人脸拓扑结构特征来完成人脸识别。
可选的，训练过程也可以不训练人脸整体特征(网络1)，在识别的过程中，使用人脸部位特征和人脸拓扑结构特征来完成人脸识别。
下面对实现人脸识别的人脸识别装置进行介绍，请参阅图13，为本申请实施例中人脸识别装置的一个实施例示意图；
该人脸识别装置包括：
获取模块1301，用于获取待识别的人脸图像；
提取模块1302，用于根据该人脸图像，通过预训练的特征提取网络，提取人脸图像特征；
确定模块1303，用于提取该人脸图像中的多个面部几何特征点，以确定多个特征点集合，该多个特征点集合中每个特征点集合对应一个人脸部位，该特征点集合包括至少一个面部几何特征点；
该获取模块1301，还用于根据该多个特征点集合获取人脸拓扑结构特征，该人脸拓扑结构特征用于确定该多个特征点集合之间的相对位置关系；
匹配模块1304，用于根据该人脸拓扑结构特征和该人脸图像特征在预设的人脸数据库中进行匹配，以获取人脸识别结果。
可选的，该人脸拓扑结构特征包括：
特征向量集合，该特征向量集合中的特征向量用于指示该多个特征点集合中任意两个该特征点集合之间的相对位置关系；或者，
特征矩阵，该特征矩阵中的元素用于指示该多个特征点集合中任意两个该特征点集合之 间的相对位置关系。
可选的，该确定模块1303还用于：
构建该多个特征点集合与标准人脸的多个特征点集合之间的映射关系，该映射关系用于确定该多个特征点集合之间的相对位置关系；
该获取模块1301具体用于：将该映射关系输入到预训练的人脸拓扑结构特征提取网络，以获取该人脸拓扑结构特征。
可选的，该映射关系包括该多个特征点集合与该标准人脸的多个特征点集合之间的距离和/或角度。
可选的，该人脸拓扑结构特征提取网络为第一网络训练后得到；
该提取模块1302还用于，从人脸图像训练样本中提取多个面部几何特征点，以确定多个样本特征点集合，该多个样本特征点集合中每个特征点集合对应该训练样本的一个人脸部位，该样本特征点集合包括至少一个面部几何特征点；
该获取模块1301还用于，获取该样本特征点集合与标准人脸的特征点集合之间的映射关系，并输入到该第一网络进行训练，获取第一损失值；
该获取模块1301还用于，根据该第一损失值更新该第一网络中的权重参数，以获取该人脸拓扑结构特征提取网络。
可选的，该提取模块1302具体用于：
将述人脸图像输入到预训练的人脸整体特征提取网络，以提取人脸整体特征。
可选的，该提取模块1302具体用于：
从该人脸图像中提取第一人脸部位图像；
将该第一人脸部位图像输入到预训练的第一部位特征提取网络，以提取第一部位特征，该第一部位特征用于在该人脸数据库中进行匹配，以获取该人脸识别结果。
可选的，该第一部位特征提取网络为第二网络训练后得到，该获取模块1301还用于：
将从人脸图像训练样本中提取的人脸部位图像输入到该第二网络进行训练，获取第二损失值；
该获取模块1301还用于，根据该第二损失值更新该第二网络中的权重参数，以获取该第一部位特征提取网络。
可选的，该提取模块1302还用于：
从该人脸图像中提取多个人脸部位图像；
该提取模块1302具体用于，将该多个人脸部位图像分别输入预训练的多个部位特征提取网络，以提取多个部位特征；
该确定模块1303还用于，根据该多个部位特征确定该人脸图像的目标部位特征。
可选的，该目标部位特征根据该多个部位特征的加权平均值确定，该多个部位特征的权值为预设值。
可选的，该人脸识别装置还包括：
检测模块1305，用于检测该多个人脸部位图像中的人脸部位是否被遮挡；
该确定模块1303还用于，若第一人脸部位图像中的第一人脸部位被遮挡，且第二人脸部位图像中的第二人脸部位未被遮挡，该第二人脸部位为该第一人脸部位的对称部位，则将该 第二人脸部位图像的水平翻转图像确定为该第一人脸部位的恢复图像，该恢复图像用于输入该部位特征提取网络以提取该部位特征。
可选的，该人脸识别装置还包括：
更新模块1306，用于基于该第一人脸部位被遮挡，更新该第一人脸部位的部位特征的权值，更新的第一权值小于该第一人脸部位的预设第一权值。
可选的，该获取模块1301还用于：
对该人脸图像进行预处理，以获取预处理后的人脸图像，该预处理包括人脸对齐，该预处理后的人脸图像用于提取该人脸图像特征和提取该多个面部几何特征点。
图14为本申请实施例提供的一种芯片硬件结构图。
图3和图4所示的基于卷积神经网络的算法可以在图14所示的NPU芯片中实现。
神经网络处理器NPU 50 NPU作为协处理器挂载到主CPU(Host CPU)上，由Host CPU分配任务。NPU的核心部分为运算电路50，通过控制器504控制运算电路503提取存储器中的矩阵数据并进行乘法运算。
在一些实现中，运算电路503内部包括多个处理单元(process engine,PE)。在一些实现中，运算电路503是二维脉动阵列。运算电路503还可以是一维脉动阵列或者能够执行例如乘法和加法这样的数学运算的其它电子线路。在一些实现中，运算电路503是通用的矩阵处理器。
举例来说，假设有输入矩阵A，权重矩阵B，输出矩阵C。运算电路从权重存储器502中取矩阵B相应的数据，并缓存在运算电路中每一个PE上。运算电路从输入存储器501中取矩阵A数据与矩阵B进行矩阵运算，得到的矩阵的部分结果或最终结果，保存在累加器508accumulator中。
统一存储器506用于存放输入数据以及输出数据。权重数据直接通过存储单元访问控制器505(direct memory access controller，DMAC)被搬运到权重存储器502中。输入数据也通过DMAC被搬运到统一存储器506中。
BIU为Bus Interface Unit即，总线接口单元510，用于AXI总线与DMAC和取指存储器509 Instruction Fetch Buffer的交互。
总线接口单元510(bus interface unit，简称BIU)，用于取指存储器509从外部存储器获取指令，还用于存储单元访问控制器505从外部存储器获取输入矩阵A或者权重矩阵B的原数据。
DMAC主要用于将外部存储器DDR中的输入数据搬运到统一存储器506或将权重数据搬运到权重存储器502中或将输入数据数据搬运到输入存储器501中。
向量计算单元507可以包括多个运算处理单元，在需要的情况下，对运算电路的输出做进一步处理，如向量乘，向量加，指数运算，对数运算，大小比较等等。主要用于神经网络中非卷积/FC层网络计算，如Pooling(池化)，Batch Normalization(批归一化)，Local Response Normalization(局部响应归一化)等。
在一些实现种，向量计算单元能507将经处理的输出的向量存储到统一缓存器506。例如，向量计算单元507可以将非线性函数应用到运算电路503的输出，例如累加值的向量，用以生成激活值。在一些实现中，向量计算单元507生成归一化的值、合并值，或二者均有。 在一些实现中，处理过的输出的向量能够用作到运算电路503的激活输入，例如用于在神经网络中的后续层中的使用。
控制器504连接的取指存储器(instruction fetch buffer)509，用于存储控制器504使用的指令；
统一存储器506，输入存储器501，权重存储器502以及取指存储器509均为On-Chip存储器。外部存储器私有于该NPU硬件架构。
其中，图3和图4所示的卷积神经网络中各层的运算可以由矩阵计算单元212或向量计算单元507执行。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
请参阅图15，为本申请实施例中一种人脸识别装置的另一个实施例示意图。本实施例提供的人脸识别装置可以包括终端或服务器，本申请实施例中对其具体设备形态不做限定。
该人脸识别装置1500可因配置或性能不同而产生比较大的差异，可以包括一个或一个以上处理器1501和存储器1502，该存储器1502中存储有程序或数据。
其中，存储器1502可以是易失性存储或非易失性存储。可选地，处理器1501是一个或多个中央处理器(CPU，Central Processing Unit，该CPU可以是单核CPU，也可以是多核CPU。处理器1501可以与存储器1502通信，在人脸识别装置1500上执行存储器1502中的一系列指令。
该人脸识别装置1500还包括一个或一个以上有线或无线网络接口1503，例如以太网接口。
可选地，尽管图15中未示出，人脸识别装置1500还可以包括一个或一个以上电源；一个或一个以上输入输出接口，输入输出接口可以用于连接显示器、鼠标、键盘、触摸屏设备、图像采集模块或其他传感设备等，输入输出接口为可选部件，可以存在也可以不存在，此处不做限定。
本实施例中人脸识别装置1500中的处理器1501所执行的流程可以参考前述方法实施例中描述的方法流程，此处不加赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。

Claims (29)

  1. 一种人脸识别方法，其特征在于，包括：
    获取待识别的人脸图像；
    根据所述人脸图像，通过预训练的特征提取网络，提取人脸图像特征；
    提取所述人脸图像中的多个面部几何特征点，以确定多个特征点集合，所述多个特征点集合中每个特征点集合对应一个人脸部位，所述特征点集合包括至少一个面部几何特征点；
    根据所述多个特征点集合获取人脸拓扑结构特征，所述人脸拓扑结构特征用于确定所述多个特征点集合之间的相对位置关系；
    根据所述人脸拓扑结构特征和所述人脸图像特征在预设的人脸数据库中进行匹配，以获取人脸识别结果。
  2. 根据权利要求1所述的方法，其特征在于，所述人脸拓扑结构特征包括：
    特征向量集合，所述特征向量集合中的特征向量用于指示所述多个特征点集合中任意两个所述特征点集合之间的相对位置关系；或者，
    特征矩阵，所述特征矩阵中的元素用于指示所述多个特征点集合中任意两个所述特征点集合之间的相对位置关系。
  3. 根据权利要求1所述的方法，其特征在于，根据所述多个人脸部位特征点集合获取人脸拓扑结构特征包括：
    构建所述多个特征点集合与标准人脸的多个特征点集合之间的映射关系，所述映射关系用于确定所述多个特征点集合之间的相对位置关系；
    将所述映射关系输入到预训练的人脸拓扑结构特征提取网络，以获取所述人脸拓扑结构特征。
  4. 根据权利要求3所述的方法，其特征在于，所述映射关系包括所述多个特征点集合与所述标准人脸的多个特征点集合之间的距离和/或角度。
  5. 根据权利要求3或4所述的方法，其特征在于，所述人脸拓扑结构特征提取网络为第一网络训练后得到，所述方法还包括：
    从人脸图像训练样本中提取多个面部几何特征点，以确定多个样本特征点集合，所述多个样本特征点集合中每个特征点集合对应所述训练样本的一个人脸部位，所述样本特征点集合包括至少一个面部几何特征点；
    获取所述样本特征点集合与标准人脸的特征点集合之间的映射关系，并输入到所述第一网络进行训练，获取第一损失值；
    根据所述第一损失值更新所述第一网络中的权重参数，以获取所述人脸拓扑结构特征提取网络。
  6. 根据权利要求1至5中任一项所述的方法，其特征在于，根据所述人脸图像，通过预训练特征提取网络，提取人脸图像特征包括：
    将述人脸图像输入到预训练的人脸整体特征提取网络，以提取人脸整体特征。
  7. 根据权利要求1至6中任一项所述的方法，其特征在于，所述方法还包括：
    从所述人脸图像中提取第一人脸部位图像；
    根据所述人脸图像，通过预训练特征提取网络，提取人脸图像特征包括：
    将所述第一人脸部位图像输入到预训练的第一部位特征提取网络，以提取第一部位特征，所述第一部位特征用于在所述人脸数据库中进行匹配，以获取所述人脸识别结果。
  8. 根据权利要求7所述的方法，其特征在于，所述第一部位特征提取网络为第二网络训练后得到，所述方法还包括：
    将从人脸图像训练样本中提取的人脸部位图像输入到所述第二网络进行训练，获取第二损失值；
    根据所述第二损失值更新所述第二网络中的权重参数，以获取所述第一部位特征提取网络。
  9. 根据权利要求1至6中任一项所述的方法，其特征在于，所述方法还包括：
    从所述人脸图像中提取多个人脸部位图像；
    根据所述人脸图像，通过预训练特征提取网络，提取人脸图像特征包括：
    将所述多个人脸部位图像分别输入预训练的多个部位特征提取网络，以提取多个部位特征；
    根据所述多个部位特征确定所述人脸图像的目标部位特征。
  10. 根据权利要求9所述的方法，其特征在于，
    所述目标部位特征根据所述多个部位特征的加权平均值确定，所述多个部位特征的权值为预设值。
  11. 根据权利要求9或10所述的方法，其特征在于，所述方法还包括：
    检测所述多个人脸部位图像中的人脸部位是否被遮挡；
    若第一人脸部位图像中的第一人脸部位被遮挡，且第二人脸部位图像中的第二人脸部位未被遮挡，所述第二人脸部位为所述第一人脸部位的对称部位，则将所述第二人脸部位图像的水平翻转图像确定为所述第一人脸部位的恢复图像，所述恢复图像用于输入所述部位特征提取网络以提取所述部位特征。
  12. 根据权利要求11所述的方法，其特征在于，所述方法还包括：
    基于所述第一人脸部位被遮挡，更新所述第一人脸部位的部位特征的权值，更新的第一权值小于所述第一人脸部位的预设第一权值。
  13. 根据权利要求1至12中任一项所述的方法，其特征在于，所述方法还包括：
    对所述人脸图像进行预处理，以获取预处理后的人脸图像，所述预处理包括人脸对齐，所述预处理后的人脸图像用于提取所述人脸图像特征和提取所述多个面部几何特征点。
  14. 一种人脸识别装置，其特征在于，包括：
    获取模块，用于获取待识别的人脸图像；
    提取模块，用于根据所述人脸图像，通过预训练的特征提取网络，提取人脸图像特征；
    确定模块，用于提取所述人脸图像中的多个面部几何特征点，以确定多个特征点集合，所述多个特征点集合中每个特征点集合对应一个人脸部位，所述特征点集合包括至少一个面部几何特征点；
    所述获取模块，还用于根据所述多个特征点集合获取人脸拓扑结构特征，所述人脸拓扑结构特征用于确定所述多个特征点集合之间的相对位置关系；
    匹配模块，用于根据所述人脸拓扑结构特征和所述人脸图像特征在预设的人脸数据库中 进行匹配，以获取人脸识别结果。
  15. 根据权利要求14所述的人脸识别装置，其特征在于，所述人脸拓扑结构特征包括：
    特征向量集合，所述特征向量集合中的特征向量用于指示所述多个特征点集合中任意两个所述特征点集合之间的相对位置关系；或者，
    特征矩阵，所述特征矩阵中的元素用于指示所述多个特征点集合中任意两个所述特征点集合之间的相对位置关系。
  16. 根据权利要求14所述的人脸识别装置，其特征在于，所述确定模块还用于：
    构建所述多个特征点集合与标准人脸的多个特征点集合之间的映射关系，所述映射关系用于确定所述多个特征点集合之间的相对位置关系；
    所述获取模块具体用于：将所述映射关系输入到预训练的人脸拓扑结构特征提取网络，以获取所述人脸拓扑结构特征。
  17. 根据权利要求16所述的人脸识别装置，其特征在于，所述映射关系包括所述多个特征点集合与所述标准人脸的多个特征点集合之间的距离和/或角度。
  18. 根据权利要求16或17所述的人脸识别装置，其特征在于，所述人脸拓扑结构特征提取网络为第一网络训练后得到；
    所述提取模块还用于，从人脸图像训练样本中提取多个面部几何特征点，以确定多个样本特征点集合，所述多个样本特征点集合中每个特征点集合对应所述训练样本的一个人脸部位，所述样本特征点集合包括至少一个面部几何特征点；
    所述获取模块还用于，获取所述样本特征点集合与标准人脸的特征点集合之间的映射关系，并输入到所述第一网络进行训练，获取第一损失值；
    所述获取模块还用于，根据所述第一损失值更新所述第一网络中的权重参数，以获取所述人脸拓扑结构特征提取网络。
  19. 根据权利要求14至18中任一项所述的人脸识别装置，其特征在于，所述提取模块具体用于：
    将述人脸图像输入到预训练的人脸整体特征提取网络，以提取人脸整体特征。
  20. 根据权利要求14至19中任一项所述的人脸识别装置，其特征在于，所述提取模块具体用于：
    从所述人脸图像中提取第一人脸部位图像；
    将所述第一人脸部位图像输入到预训练的第一部位特征提取网络，以提取第一部位特征，所述第一部位特征用于在所述人脸数据库中进行匹配，以获取所述人脸识别结果。
  21. 根据权利要求20所述的人脸识别装置，其特征在于，所述第一部位特征提取网络为第二网络训练后得到，所述获取模块还用于：
    将从人脸图像训练样本中提取的人脸部位图像输入到所述第二网络进行训练，获取第二损失值；
    所述获取模块还用于，根据所述第二损失值更新所述第二网络中的权重参数，以获取所述第一部位特征提取网络。
  22. 根据权利要求14至19中任一项所述的人脸识别装置，其特征在于，所述提取模块还用于：
    从所述人脸图像中提取多个人脸部位图像；
    所述提取模块具体用于，将所述多个人脸部位图像分别输入预训练的多个部位特征提取网络，以提取多个部位特征；
    所述确定模块还用于，根据所述多个部位特征确定所述人脸图像的目标部位特征。
  23. 根据权利要求22所述的人脸识别装置，其特征在于，
    所述目标部位特征根据所述多个部位特征的加权平均值确定，所述多个部位特征的权值为预设值。
  24. 根据权利要求22或23所述的人脸识别装置，其特征在于，所述人脸识别装置还包括：
    检测模块，用于检测所述多个人脸部位图像中的人脸部位是否被遮挡；
    所述确定模块还用于，若第一人脸部位图像中的第一人脸部位被遮挡，且第二人脸部位图像中的第二人脸部位未被遮挡，所述第二人脸部位为所述第一人脸部位的对称部位，则将所述第二人脸部位图像的水平翻转图像确定为所述第一人脸部位的恢复图像，所述恢复图像用于输入所述部位特征提取网络以提取所述部位特征。
  25. 根据权利要求24所述的人脸识别装置，其特征在于，所述人脸识别装置还包括：
    更新模块，用于基于所述第一人脸部位被遮挡，更新所述第一人脸部位的部位特征的权值，更新的第一权值小于所述第一人脸部位的预设第一权值。
  26. 根据权利要求14至25中任一项所述的人脸识别装置，其特征在于，
    所述获取模块还用于：
    对所述人脸图像进行预处理，以获取预处理后的人脸图像，所述预处理包括人脸对齐，所述预处理后的人脸图像用于提取所述人脸图像特征和提取所述多个面部几何特征点。
  27. 一种人脸识别装置，其特征在于，包括处理器和存储器，所述处理器和所述存储器相互连接，其中，所述存储器用于存储计算机程序，所述计算机程序包括程序指令，所述处理器用于调用所述程序指令，执行如权利要求1至13中任一项所述的方法。
  28. 一种包含指令的计算机程序产品，其特征在于，当其在计算机上运行时，使得所述计算机执行如权利要求1至13中任一项所述的方法。
  29. 一种计算机可读存储介质，包括指令，其特征在于，当所述指令在计算机上运行时，使得计算机执行如权利要求1至13中任一项所述的方法。
PCT/CN2020/105772 2020-01-16 2020-07-30 人脸识别方法和人脸识别装置 Ceased WO2021143101A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US17/758,946 US12307815B2 (en) 2020-01-16 2020-07-30 Face recognition method and face recognition apparatus
EP20914749.5A EP4075324B1 (en) 2020-01-16 2020-07-30 Face recognition method and face recognition device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010051193.9A CN111274916B (zh) 2020-01-16 2020-01-16 人脸识别方法和人脸识别装置
CN202010051193.9 2020-01-16

Publications (1)

Publication Number Publication Date
WO2021143101A1 true WO2021143101A1 (zh) 2021-07-22

Family

ID=70997337

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/105772 Ceased WO2021143101A1 (zh) 2020-01-16 2020-07-30 人脸识别方法和人脸识别装置

Country Status (4)

Country Link
US (1) US12307815B2 (zh)
EP (1) EP4075324B1 (zh)
CN (1) CN111274916B (zh)
WO (1) WO2021143101A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113688849A (zh) * 2021-08-30 2021-11-23 中国空空导弹研究院 一种用于卷积神经网络的灰度图像序列特征提取方法
CN115424353A (zh) * 2022-09-07 2022-12-02 杭银消费金融股份有限公司 基于ai模型的业务用户特征识别方法及系统
CN117173758A (zh) * 2022-12-23 2023-12-05 南昌工学院 一种基于多维度特征融合网络的学习注意力状态评估方法

Families Citing this family (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111274916B (zh) * 2020-01-16 2024-02-02 华为技术有限公司 人脸识别方法和人脸识别装置
CN111783601B (zh) * 2020-06-24 2024-04-26 北京百度网讯科技有限公司 人脸识别模型的训练方法、装置、电子设备及存储介质
CN111931628B (zh) * 2020-08-04 2023-10-24 腾讯科技（深圳）有限公司 人脸识别模型的训练方法、装置及相关设备
CN111985360A (zh) * 2020-08-05 2020-11-24 上海依图网络科技有限公司 一种人脸识别的方法、装置、设备和介质
US11741751B2 (en) * 2020-10-01 2023-08-29 Vinai AI Application and Research Joint Stock Co. Masked face recognition method
CN112307947A (zh) * 2020-10-29 2021-02-02 北京沃东天骏信息技术有限公司 用于生成信息的方法和装置
CN114639134A (zh) * 2020-12-16 2022-06-17 富士通株式会社 用于人脸识别的方法和装置
CN115035560B (zh) * 2021-03-04 2025-12-23 武汉Tcl集团工业研究院有限公司 口罩佩戴识别方法、装置、终端设备及存储介质
CN113657227A (zh) * 2021-08-06 2021-11-16 姜政毫 人脸识别方法及基于深度学习算法的人脸识别系统
CN115546856A (zh) * 2021-11-30 2022-12-30 青岛海尔智能家电科技有限公司 用于人脸识别的方法、装置及电子设备
CN114463806A (zh) * 2021-12-29 2022-05-10 四川新网银行股份有限公司 一种基于模型可视化的新特征自动预警方法
CN115393482A (zh) * 2022-08-08 2022-11-25 网易（杭州）网络有限公司 一种表情动画重定向方法、装置及电子设备
TWI818824B (zh) * 2022-12-07 2023-10-11 財團法人工業技術研究院 用於計算遮蔽人臉影像的人臉擺動方向的裝置及方法
CN116645715A (zh) * 2023-05-30 2023-08-25 中国工商银行股份有限公司 面部检测方法、装置、设备、介质及产品
CN117238018B (zh) * 2023-09-20 2024-06-21 华南理工大学 基于多粒度的可增量深宽网络活体检测方法、介质及设备
CN117391968A (zh) * 2023-10-16 2024-01-12 南京邮电大学 一种人脸图像复原方法、系统、存储介质及设备
CN117369349B (zh) * 2023-12-08 2024-02-23 如特数字科技(苏州)有限公司 一种远程监测智能机器人的管理系统
CN117636446B (zh) * 2024-01-25 2024-05-07 江汉大学 脸部穴位定位方法、针灸方法、针灸机器人及存储介质
CN118334758B (zh) * 2024-06-12 2024-08-23 广东博科电子科技有限公司 应用于楼宇门禁系统的图像识别方法及系统
CN120126201B (zh) * 2025-05-12 2025-07-25 华中科技大学 基于非线性映射的面部机器人表情协同控制方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106022317A (zh) * 2016-06-27 2016-10-12 北京小米移动软件有限公司 人脸识别方法及装置
CN108898068A (zh) * 2018-06-06 2018-11-27 腾讯科技（深圳）有限公司 一种人脸图像的处理方法和装置以及计算机可读存储介质
CN110263673A (zh) * 2019-05-31 2019-09-20 合肥工业大学 面部表情识别方法、装置、计算机设备及存储介质
CN111274916A (zh) * 2020-01-16 2020-06-12 华为技术有限公司 人脸识别方法和人脸识别装置

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7236615B2 (en) * 2004-04-21 2007-06-26 Nec Laboratories America, Inc. Synergistic face detection and pose estimation with energy-based models
CN101980305A (zh) * 2010-10-21 2011-02-23 青岛科技大学 基于人脸识别与3g通信的家庭智能门禁系统
CN105684038B (zh) * 2013-10-28 2019-06-11 谷歌有限责任公司 用于替换图像的部分的图像缓存
CN103927554A (zh) * 2014-05-07 2014-07-16 中国标准化研究院 一种基于拓扑结构的图像稀疏表征面部表情特征提取系统和方法
US11544962B2 (en) * 2014-07-15 2023-01-03 FaceChecks LLC Multi-algorithm-based face recognition system and method with optimal dataset partitioning for a cloud environment
JP6754619B2 (ja) * 2015-06-24 2020-09-16 三星電子株式会社Ｓａｍｓｕｎｇ Ｅｌｅｃｔｒｏｎｉｃｓ Ｃｏ．，Ｌｔｄ． 顔認識方法及び装置
CN105069622B (zh) * 2015-08-03 2018-08-31 福州海景科技开发有限公司 一种面向移动终端的人脸识别支付系统和方法
CN106570822B (zh) * 2016-10-25 2020-10-16 宇龙计算机通信科技(深圳)有限公司 一种人脸贴图方法及装置
CN106686353A (zh) * 2016-12-30 2017-05-17 佛山亚图信息技术有限公司 基于云端的安防控制系统及其方法
CN106998444B (zh) * 2017-02-14 2020-02-18 广东中科人人智能科技有限公司 一种大数据人脸监控系统
CN109583332B (zh) * 2018-11-15 2021-07-27 北京三快在线科技有限公司 人脸识别方法、人脸识别系统、介质及电子设备
CN110222566A (zh) * 2019-04-30 2019-09-10 北京迈格威科技有限公司 一种人脸特征的获取方法、装置、终端及存储介质
CN110675489B (zh) * 2019-09-25 2024-01-23 北京达佳互联信息技术有限公司 一种图像处理方法、装置、电子设备和存储介质
TWI783199B (zh) * 2019-12-25 2022-11-11 亞旭電腦股份有限公司 臉部辨識的處理方法及電子裝置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106022317A (zh) * 2016-06-27 2016-10-12 北京小米移动软件有限公司 人脸识别方法及装置
CN108898068A (zh) * 2018-06-06 2018-11-27 腾讯科技（深圳）有限公司 一种人脸图像的处理方法和装置以及计算机可读存储介质
CN110263673A (zh) * 2019-05-31 2019-09-20 合肥工业大学 面部表情识别方法、装置、计算机设备及存储介质
CN111274916A (zh) * 2020-01-16 2020-06-12 华为技术有限公司 人脸识别方法和人脸识别装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4075324A4

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113688849A (zh) * 2021-08-30 2021-11-23 中国空空导弹研究院 一种用于卷积神经网络的灰度图像序列特征提取方法
CN113688849B (zh) * 2021-08-30 2023-10-24 中国空空导弹研究院 一种用于卷积神经网络的灰度图像序列特征提取方法
CN115424353A (zh) * 2022-09-07 2022-12-02 杭银消费金融股份有限公司 基于ai模型的业务用户特征识别方法及系统
CN115424353B (zh) * 2022-09-07 2023-05-05 杭银消费金融股份有限公司 基于ai模型的业务用户特征识别方法及系统
CN117173758A (zh) * 2022-12-23 2023-12-05 南昌工学院 一种基于多维度特征融合网络的学习注意力状态评估方法

Also Published As

Publication number Publication date
US12307815B2 (en) 2025-05-20
EP4075324A4 (en) 2023-06-14
EP4075324A1 (en) 2022-10-19
US20230080031A1 (en) 2023-03-16
CN111274916A (zh) 2020-06-12
CN111274916B (zh) 2024-02-02
EP4075324B1 (en) 2025-07-30

Similar Documents

Publication Publication Date Title
WO2021143101A1 (zh) 人脸识别方法和人脸识别装置
CN112766158B (zh) 基于多任务级联式人脸遮挡表情识别方法
WO2021218238A1 (zh) 图像处理方法和图像处理装置
US11232286B2 (en) Method and apparatus for generating face rotation image
CN112215180B (zh) 一种活体检测方法及装置
CN112800903B (zh) 一种基于时空图卷积神经网络的动态表情识别方法及系统
CN110728209B (zh) 一种姿态识别方法、装置、电子设备及存储介质
CN112446398B (zh) 图像分类方法以及装置
CN110222717B (zh) 图像处理方法和装置
US20210012198A1 (en) Method for training deep neural network and apparatus
CN106951867B (zh) 基于卷积神经网络的人脸识别方法、装置、系统及设备
CN111368672A (zh) 一种用于遗传病面部识别模型的构建方法及装置
WO2021077984A1 (zh) 对象识别方法、装置、电子设备及可读存储介质
WO2021175050A1 (zh) 三维重建方法和三维重建装置
WO2021227726A1 (zh) 面部检测、图像检测神经网络训练方法、装置和设备
WO2021043168A1 (zh) 行人再识别网络的训练方法、行人再识别方法和装置
WO2022179587A1 (zh) 一种特征提取的方法以及装置
CN117437522B (zh) 一种人脸识别模型训练方法、人脸识别方法及装置
CN110222718A (zh) 图像处理的方法及装置
WO2022246612A1 (zh) 活体检测方法、活体检测模型的训练方法及其装置和系统
WO2022179599A1 (zh) 一种感知网络及数据处理方法
CN114037838A (zh) 神经网络的训练方法、电子设备及计算机程序产品
CN120472530A (zh) 一种换衣场景下的行人重识别方法
CN114120386A (zh) 人脸识别方法、装置、设备及存储介质
CN119049089B (zh) 一种基于近距离单目灰度摄像头的双手拇指关键点实时检测方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20914749

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020914749

Country of ref document: EP

Effective date: 20220714

NENP Non-entry into the national phase

Ref country code: DE

WWG Wipo information: grant in national office

Ref document number: 17758946

Country of ref document: US

WWG Wipo information: grant in national office

Ref document number: 2020914749

Country of ref document: EP