WO2023151237A1 - 人脸位姿估计方法、装置、电子设备及存储介质 - Google Patents

人脸位姿估计方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2023151237A1
WO2023151237A1 PCT/CN2022/107825 CN2022107825W WO2023151237A1 WO 2023151237 A1 WO2023151237 A1 WO 2023151237A1 CN 2022107825 W CN2022107825 W CN 2022107825W WO 2023151237 A1 WO2023151237 A1 WO 2023151237A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature
face
information
pose
feature map
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/107825
Other languages
English (en)
French (fr)
Inventor
杨战波
黄泽元
祁晓婷
蒋召
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Xumi Yuntu Space Technology Co Ltd
Original Assignee
Shenzhen Xumi Yuntu Space Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Xumi Yuntu Space Technology Co Ltd filed Critical Shenzhen Xumi Yuntu Space Technology Co Ltd
Priority to US18/834,647 priority Critical patent/US20250037305A1/en
Priority to JP2024545137A priority patent/JP7770581B2/ja
Priority to EP22925596.3A priority patent/EP4471737B1/en
Priority to KR1020247024587A priority patent/KR20240144139A/ko
Publication of WO2023151237A1 publication Critical patent/WO2023151237A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24133—Distances to prototypes
    • G06F18/24137—Distances to cluster centroïds
    • G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20132—Image cropping
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the present disclosure relates to the field of computer technology, and in particular to a face pose estimation method, device, electronic equipment and storage medium.
  • Face pose estimation is an important research direction in the field of computer vision, and the change of face pose is also a key factor affecting the effect of face recognition. Only by effectively estimating the pose of the face image can the influence of the face pose on the face recognition effect be further weakened. Face pose estimation is also widely used in the field of computer vision, such as living body recognition, human-computer interaction, virtual reality, intelligent monitoring, etc.
  • the traditional face pose estimation method In the current traditional face pose estimation method, some irrelevant facial features are ignored by capturing the pose angle information, so it is impossible to make full use of the face and surrounding information for model optimization, and the traditional face pose estimation based on the general recognition model The way of pose recognition cannot fully obtain the key information of face pose.
  • the traditional face pose estimation method directly uses the detected face frame as input, without considering the inaccuracy of the face frame, which leads to the traditional face pose estimation method when directly regressing or classifying the pose angle. It is difficult to obtain the optimal prediction effect, thereby reducing the accuracy of the face pose estimation results.
  • the embodiments of the present disclosure provide a face pose estimation method, device, electronic equipment, and storage medium to solve the problem that the key information of the face pose cannot be fully obtained in the prior art, and the face pose estimation The accuracy of the results is poor.
  • the first aspect of the embodiments of the present disclosure provides a face pose estimation method, including: acquiring a target image containing face information, and inputting the target image into a pre-built pose estimation model; In the model, the shallow dense connection layer is used to extract the features of the target image, and multiple first feature maps containing shallow feature information are obtained; multiple first feature maps are used as the input of the deep feature multiplexing layer, and the deep feature multiplexing layer is used to obtain multiple first feature maps.
  • the second aspect of the embodiments of the present disclosure provides a face pose estimation device, including: an acquisition module configured to acquire a target image containing face information, and input the target image to a pre-built pose estimation model Middle; the extraction module is configured to perform feature extraction on the target image using a shallow densely connected layer in the pose estimation model to obtain a plurality of first feature maps containing shallow feature information; the fusion module is configured to combine multiple The first feature map is used as the input of the deep feature multiplexing layer, and the deep feature multiplexing layer is used to perform an information fusion operation on multiple first feature maps to obtain a second feature map, so as to integrate deep feature information into the shallow feature information;
  • the prediction module is configured to use the attention layer to extract the face pose information in the second feature map to obtain a third feature map containing the face pose information, and use the classifier to predict the third feature map to obtain The third feature map corresponds to the prediction result of the face pose, and the face pose in the target image is determined according to the prediction result.
  • a third aspect of the embodiments of the present disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and operable on the processor. When the processor executes the program, the steps of the above method are implemented.
  • a fourth aspect of the embodiments of the present disclosure provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above method are implemented.
  • the target image containing face information and input the target image into the pre-built pose estimation model; in the pose estimation model, use the shallow dense connection layer to extract the feature of the target image, and obtain the shallow feature
  • a plurality of first feature maps of information using multiple first feature maps as the input of the deep feature multiplexing layer, using the deep feature multiplexing layer to perform information fusion operations on the multiple first feature maps to obtain the second feature map, so that Integrate deep feature information into the shallow feature information; use the attention layer to extract the face pose information in the second feature map to obtain the third feature map containing face pose information, and use the classifier to classify the third feature
  • the prediction result of the face pose corresponding to the third feature map is obtained, and the face pose in the target image is determined according to the prediction result.
  • the disclosure can fully obtain the key information of the face pose and make the estimation result of the face pose more accurate.
  • FIG. 1 is a schematic diagram of a network structure of a pose estimation model provided by an embodiment of the present disclosure
  • FIG. 2 is a schematic flowchart of a face pose estimation method provided by an embodiment of the present disclosure
  • FIG. 3 is a schematic structural diagram of a face pose estimation device provided by an embodiment of the present disclosure.
  • Fig. 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
  • the present disclosure proposes a new face pose estimation method based on a neural network pose estimation model.
  • the neural network model of the present disclosure mainly includes a backbone network, a Neck module and a Head module.
  • the training set used contains the face frame and the pose annotations of multiple points, including the pose annotations composed of the annotation points of multiple face frames and multiple pose angles. information.
  • the augmented training of the pose estimation model is realized.
  • Use the backbone network composed of the shallow dense connection layer and the deep feature multiplexing layer in the pose estimation model to fuse the shallow features and deep features in the face image, and accurately estimate based on the attention module and the group angle regression module The pose angle of the face in the image.
  • FIG. 1 is a schematic diagram of the network structure of the pose estimation model provided by an embodiment of the present disclosure; as shown in FIG. 1 , the network structure of the pose estimation model Specifically can include:
  • the backbone network includes a shallow dense connection layer (corresponding to C1 ⁇ C5) and a deep feature multiplexing layer (corresponding to P3 ⁇ P5).
  • the shallow dense connection layer is mainly used to extract the forward features in the face image.
  • the dense connection is It means that the output of each module is not only used as the input of the next module, but also as the input of other subsequent modules; the deep feature multiplexing layer is used to integrate the extracted shallow feature information into the deep feature information when performing feature expression.
  • a feature map containing more semantic information ie, pose information
  • the Neck module consists of a SE attention module and a Transformer-based feature transformation module.
  • the SE attention module amplifies important features by weighting the features of different channels, while suppressing relatively unimportant features.
  • Transformer has a strong feature extraction ability through ingenious structural design, and extracts effective features through a multi-head attention mechanism, thereby highlighting important features in the feature map and suppressing interference features.
  • the Head module corresponds to the grouping angle regression module (that is, the classifier).
  • the grouping angle regression module that is, the classifier.
  • Fig. 2 is a schematic flowchart of a method for estimating a face pose provided by an embodiment of the present disclosure.
  • the face pose estimation method in FIG. 2 can be executed by a server.
  • the face pose estimation method may specifically include:
  • the face pose in the target image is determined according to the prediction result.
  • the target image in the embodiment of the present disclosure refers to the face image after face detection is performed on the collected original image, that is, the input of the pose estimation model is the face image after face detection, not the original image.
  • the pose estimation model Before using the pose estimation model to predict the pose angle, it is necessary to build the pose estimation model and perform training, and then use the trained pose estimation model as the model for actual use.
  • the pose estimation model is used for prediction, and three bits are output Attitude angle, that is, pitch angle, yaw angle and roll angle. Due to the differences of different faces and face detection models, the face boundary does not have a completely unified standard, so the pose estimation model obtained by using the detected face images for training has better robustness.
  • the present disclosure obtains the target image containing face information, and inputs the target image into the pre-built pose estimation model; in the pose estimation model, the shallow dense connection layer is used Perform feature extraction on the target image to obtain multiple first feature maps containing shallow feature information; use multiple first feature maps as the input of the deep feature multiplexing layer, and use the deep feature multiplexing layer to separately process multiple first feature maps
  • the image performs information fusion operation to obtain the second feature map, so as to integrate the deep feature information into the shallow feature information; use the attention layer to extract the face pose information in the second feature map, and obtain the face pose information containing
  • the classifier is used to predict the third feature map to obtain a prediction result of the face pose corresponding to the third feature map, and determine the face pose in the target image according to the prediction result.
  • the disclosure can fully obtain the key information of the face pose and make the estimation result of the face pose more accurate.
  • the pose estimation model is constructed in the following manner, including: acquiring an original image containing face information, using a face detection model to detect the original image, and obtaining a face image and a face frame corresponding to the original image, And obtain the face pose information in the original image, use the face image, the position coordinates of the face frame, and the face pose information to generate the first data set; based on the original image and the position coordinates of the face frame, use the preset
  • the cropping method is used to crop the original image to obtain a cropped face image, and use the cropped face image, the position coordinates of the face frame, and the face pose information to generate a second data set; for the first data set and The second data set is combined to obtain a training set, and the training set is used to train the pose estimation model to obtain the trained pose estimation model.
  • the training of the pose estimation model mainly includes two parts, that is, the labeling of the data set and the data augmentation during training.
  • the contents of these two parts will be introduced in detail below in conjunction with specific embodiments, which can specifically include the following:
  • the annotation information in the training set includes 7-point pose annotations, that is, the annotation points of 4 detection frames and 3 pose angles.
  • the above-mentioned detected face image and the annotation information composed of 7-point pose annotations are used as the first data set.
  • the face image in the original image is randomly cropped according to a certain cropping ratio (such as 0.5 times) based on the original image through the random cropping augmentation method, such as the random cropping ratio
  • a face image with a slightly larger (1.0 to 1.2 times) face detection frame thereby generating a face image similar to the face image in the first data set, and combining the newly generated face image and the face image corresponding to the
  • the annotation information composed of 7-point pose annotations is used as the second data set.
  • the scale of the training data set is expanded by means of data augmentation, and when the augmented training set is used to train the pose estimation model, not only the inaccurate labeling frame is reduced Influence, but also expand the amount of data to improve the robustness of the model.
  • the training set contains face images and annotation information
  • the annotation information is used as a label during model training
  • the annotation information includes a plurality of annotation points corresponding to the face frame, and a plurality of pose angles; wherein , the label points of the face frame include the coordinates of any corner point corresponding to the face frame, and the width and height of the face frame, and the pose angle includes pitch angle, yaw angle and roll angle.
  • the shallow densely connected layer is used to extract features of the target image to obtain a plurality of first feature maps containing shallow feature information, including: the shallow densely connected layer contains a plurality of sequentially connected convolution modules , use each convolution module to perform convolution operation on the feature map input to the convolution module in turn, and use the output of each convolution module as the input of the next convolution module, and the input of each convolution module also contains The output of the previous convolution module, the output of the last multiple convolution modules in the shallow densely connected layer is used as the first feature map.
  • the embodiment of the present disclosure proposes a dense A backbone network composed of a connection layer and a deep feature multiplexing layer.
  • the shallow feature information and deep feature information in the face image are extracted through the backbone network, and the shallow feature information is integrated into the deep feature information to obtain a feature map containing more semantic information.
  • the shallow densely connected layer is divided into five convolution modules, C1, C2, C3, C4, and C5, in the pose estimation model, and the densely connected It means that the output of each convolution module is not only used as the input of the next convolution module, but also used as the input of other subsequent convolution modules. This not only enables the deep feature reuse layer to receive more shallow information, but also improves the ability of the feature to express the pose information, and improves the utilization efficiency of the feature.
  • the features extracted by each part can be used by more modules. .
  • the deep feature multiplexing layer is used to perform information fusion operations on multiple first feature maps to obtain the second feature map, so as to incorporate deep feature information into the shallow feature information, including: In the deep feature multiplexing layer Including convolution modules corresponding to the number of the first feature map, using the convolution module of the deep feature multiplexing layer to perform convolution transformation on the first feature map to obtain the second feature map, so that the second feature map containing shallow feature information Integrating deep feature information into the second feature map, performing global average pooling on the second feature map, and obtaining the corresponding second feature map after global average pooling.
  • the deep feature multiplexing layer when using the deep feature multiplexing layer for feature expression, by integrating the shallow feature information into the deep feature information, as shown in Figure 1, use the outputs of the convolution modules C3, C4, and C5 to make three Different levels of feature expression, namely P3, P4, and P5, to incorporate deep feature information into the feature maps output by C3, C4, and C5.
  • the implementation of the deep feature multiplexing layer will be described in detail below in conjunction with Figure 1 in the above-mentioned embodiment, which may specifically include the following:
  • C3, C4, and C5 correspond to the convolution modules in the shallow densely connected layer, and the outputs of C3, C4, and C5 are used as the inputs of P3, P4, and P5, and the convolution modules in the deep feature multiplexing layer are used to pair C3, C4, and
  • the feature map output by C5 performs convolution operation, and outputs a feature map containing semantic information (ie, pose information).
  • the attention layer includes an SE attention module and a feature transformation module, and the attention layer is used to extract the face pose information in the second feature map to obtain a third feature map containing the face pose information , including: using the SE attention module to calculate the weight of the feature channel in the second feature map, and weighting the feature channel according to the channel weight to obtain a weighted second feature map; using the feature transformation module to weight the second feature Feature extraction is performed on the graph to obtain a third feature map containing effective feature information, and the effective feature information includes face pose information.
  • this disclosure provides a Neck module based on SE attention module and Transformer.
  • the SE attention module is used to weight the features of different channels to amplify important features while suppressing relatively unimportant features.
  • the Transformer has a strong feature extraction ability, which can highlight important features and suppress interference features.
  • the SE attention module contains two fully connected layers and a sigmoid layer (ie, a normalization layer), assuming that the number of channels of the second feature map input to the SE attention module is c, then the number of channels is The second feature map of c is processed by two fully connected layers and a sigmoid layer to obtain the channel weight of each feature channel, and then multiply each channel weight with its corresponding feature channel to obtain a weighted feature map.
  • a sigmoid layer ie, a normalization layer
  • the input weighted feature map is processed by Multi-Head Attention and multi-layer fully connected layer MLP to obtain transformed feature maps of c groups.
  • Multi-Head Attention and multi-layer fully connected layer MLP to obtain transformed feature maps of c groups.
  • the classifier is used to predict the third feature map to obtain the prediction result of the face pose corresponding to the third feature map, and determine the face pose in the target image according to the prediction result, including:
  • the face pose corresponds to multiple third feature maps, and each third feature map corresponds to multiple classifiers.
  • Each classifier is used to predict a number of angle values based on the third feature map, and calculate each classification according to a number of angle values.
  • the pose angle predicted by the classifier sum the pose angles predicted by all classifiers, and obtain the pose angle corresponding to each face pose, and use the pose angles corresponding to the three face poses as the target image Estimation results of face pose in .
  • the disclosure adds a grouped angle regression module, also called a classifier module, to the pose estimation model, which is the Head module in FIG. 1 .
  • a classifier module for each pose angle, each set of features can contain 3 classifiers, and each classifier predicts 10 angle values.
  • the results of the nine classifiers are added together to obtain the final pose angle. Perform the above calculations for the three pose angles respectively to obtain the final predicted three pose angles.
  • each third feature map input into the Head module corresponds to three classifiers, each classifier is used to predict 10 different angle values, and according to the prediction results of each classifier, all classifiers (total 9 classifiers), the final prediction result of the Head module is used as the angle value corresponding to the current pose angle.
  • the model will output an angle value, and use the angle values corresponding to all pose angles as the estimation result of the face pose in the target image.
  • the present disclosure adopts the data set format of 7-point pose labeling (4 detection frame label points and 3 pose angles), and based on the labeling scheme, the data augmentation of face images is designed. wide, thereby expanding the amount of data in the training set.
  • the present disclosure designs a backbone network with shallow dense connection and deep feature reuse, which improves the utilization efficiency and expression ability of features.
  • the present disclosure also discloses a Neck module based on SE attention and Transformer, thereby further extracting key information of poses.
  • a grouping angle regression module is added to the model, and 3 pose angles are predicted more accurately through 27 classifiers.
  • Fig. 3 is a schematic structural diagram of a face pose estimation device provided by an embodiment of the present disclosure. As shown in Figure 3, the face pose estimation device includes:
  • the obtaining module 301 is configured to obtain a target image containing face information, and input the target image into a pre-built pose estimation model;
  • the extraction module 302 is configured to perform feature extraction on the target image using a shallow densely connected layer in the pose estimation model to obtain a plurality of first feature maps containing shallow feature information;
  • the fusion module 303 is configured to use the multiple first feature maps as the input of the deep feature multiplexing layer, and use the deep feature multiplexing layer to perform an information fusion operation on the multiple first feature maps to obtain the second feature map, so that the shallow Integrate deep feature information into layer feature information;
  • the prediction module 304 is configured to use the attention layer to extract the face pose information in the second feature map to obtain a third feature map containing the face pose information, and use a classifier to predict the third feature map, A prediction result of the face pose corresponding to the third feature map is obtained, and the face pose in the target image is determined according to the prediction result.
  • the acquisition module 301 in FIG. 3 also adopts the following method to construct the pose estimation model, including: acquiring the original image containing face information, using the face detection model to detect the original image, and obtaining the person corresponding to the original image Face image and face frame, and obtain the face pose information in the original image, use the face image, the position coordinates of the face frame, and the face pose information to generate the first data set; based on the original image and face frame The position coordinates of the original image are cropped using the preset cropping method to obtain the cropped face image, and the second data set is generated by using the cropped face image, the position coordinates of the face frame, and the face pose information ; Combining the first data set and the second data set to obtain a training set, using the training set to train the pose estimation model, and obtaining the trained pose estimation model.
  • the training set contains face images and annotation information
  • the annotation information is used as a label during model training
  • the annotation information includes a plurality of annotation points corresponding to the face frame, and a plurality of pose angles; wherein , the label points of the face frame include the coordinates of any corner point corresponding to the face frame, and the width and height of the face frame, and the pose angle includes pitch angle, yaw angle and roll angle.
  • the shallow densely connected layer contains a plurality of sequentially connected convolution modules
  • the extraction module 302 of FIG. 3 uses each convolution module to sequentially perform a convolution operation on the feature map input to the convolution module, and The output of each convolution module is used as the input of the next convolution module, and the input of each convolution module also includes the output of the previous convolution module, and the output of the last multiple convolution modules in the shallow dense connection layer is used as The first feature map.
  • the deep feature multiplexing layer contains convolution modules corresponding to the number of the first feature maps
  • the fusion module 303 of FIG. 3 uses the convolution modules of the deep feature multiplexing layer to perform
  • the second feature map is obtained by convolution transformation, so as to incorporate deep feature information into the second feature map containing shallow feature information, and perform global average pooling on the second feature map to obtain the corresponding second feature map after global average pooling .
  • the attention layer includes an SE attention module and a feature transformation module.
  • the prediction module 304 of FIG. performing weighting to obtain a weighted second feature map; using a feature transformation module to perform feature extraction on the weighted second feature map to obtain a third feature map containing effective feature information, and the effective feature information includes face pose information.
  • each face pose corresponds to multiple third feature maps
  • each third feature map corresponds to multiple classifiers
  • each classifier is used to predict a number of angle values according to the third feature map, Calculate the pose angle predicted by each classifier according to a number of angle values, sum the pose angles predicted by all classifiers, and obtain the pose angle corresponding to each face pose, and combine the three face poses
  • the pose angle corresponding to the pose is used as the estimation result of the face pose in the target image.
  • FIG. 4 is a schematic structural diagram of an electronic device 4 provided by an embodiment of the present disclosure.
  • the electronic device 4 of this embodiment includes: a processor 401 , a memory 402 , and a computer program 403 stored in the memory 402 and capable of running on the processor 401 .
  • the processor 401 executes the computer program 403
  • the steps in the foregoing method embodiments are implemented.
  • the processor 401 executes the computer program 403 the functions of the modules/units in the foregoing device embodiments are implemented.
  • the computer program 403 can be divided into one or more modules/units, and one or more modules/units are stored in the memory 402 and executed by the processor 401 to complete the present disclosure.
  • One or more modules/units may be a series of computer program instruction segments capable of accomplishing specific functions, and the instruction segments are used to describe the execution process of the computer program 403 in the electronic device 4 .
  • the electronic device 4 can be electronic devices such as desktop computers, notebooks, palmtop computers, and cloud servers.
  • the electronic device 4 may include but not limited to a processor 401 and a memory 402.
  • FIG. 4 is only an example of the electronic device 4, and does not constitute a limitation to the electronic device 4. It may include more or less components than those shown in the illustration, or combine some components, or different components.
  • an electronic device may also include an input and output device, a network access device, a bus, and the like.
  • the processor 401 can be a central processing unit (Central Processing Unit, CPU), and can also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), on-site Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the storage 402 may be an internal storage unit of the electronic device 4 , for example, a hard disk or a memory of the electronic device 4 .
  • the memory 402 can also be an external storage device of the electronic device 4, for example, a plug-in hard disk equipped on the electronic device 4, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash memory card ( Flash Card), etc. Further, the memory 402 may also include both an internal storage unit of the electronic device 4 and an external storage device.
  • the memory 402 is used to store computer programs and other programs and data required by the electronic device.
  • the memory 402 can also be used to temporarily store data that has been output or will be output.
  • the disclosed apparatus/computer equipment and methods can be implemented in other ways.
  • the device/computer device embodiments described above are only illustrative, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, and multiple units or components can be Incorporation may either be integrated into another system, or some features may be omitted, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • a unit described as a separate component may or may not be physically separated, and a component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
  • an integrated module/unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the present disclosure realizes all or part of the processes in the methods of the above embodiments, and can also be completed by instructing related hardware through computer programs.
  • the computer programs can be stored in computer-readable storage media, and the computer programs can be processed. When executed by the controller, the steps in the above-mentioned method embodiments can be realized.
  • a computer program may include computer program code, which may be in source code form, object code form, executable file, or some intermediate form or the like.
  • the computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory (Read-Only Memory, ROM), random access Memory (Random Access Memory, RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in computer readable media may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, computer readable media may not Including electrical carrier signals and telecommunication signals.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Multimedia (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Human Computer Interaction (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Geometry (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Image Analysis (AREA)

Abstract

本公开提供一种人脸位姿估计方法、装置、电子设备及存储介质。方法包括：获取包含人脸信息的目标图像，并将目标图像输入到预先构建的位姿估计模型中；利用浅层密集连接层对目标图像进行特征提取，得到包含浅层特征信息的多个第一特征图；利用深层特征复用层分别对多个第一特征图执行信息融合操作得到第二特征图，以便在浅层特征信息中融入深层特征信息；利用注意力层对第二特征图中的人脸位姿信息进行提取，得到包含人脸位姿信息的第三特征图，利用分类器对第三特征图进行预测，得到第三特征图对应人脸位姿的预测结果，根据预测结果确定目标图像中的人脸位姿。本公开能够充分获取人脸位姿的关键信息，提升人脸位姿估计结果的准确性。

Description

人脸位姿估计方法、装置、电子设备及存储介质 技术领域
本公开涉及计算机技术领域，尤其涉及一种人脸位姿估计方法、装置、电子设备及存储介质。
背景技术
人脸位姿估计是计算机视觉领域的一个重要研究方向，人脸位姿变化也是影响人脸识别效果的一个关键因素。对人脸图像进行有效的位姿估计，才能够进一步减弱人脸位姿对人脸识别效果的影响。人脸位姿估计在计算机视觉领域中也有广泛的应用，比如活体识别、人机交互、虚拟现实、智能监控等。
目前传统的人脸位姿估计方法中，通过捕捉位姿角度信息而忽略一些无关的面部特征，因此无法充分利用人脸及周围信息进行模型优化，并且传统的基于通用识别模型的进行人脸位姿识别的方式，无法充分获取人脸位姿的关键信息。传统的人脸位姿估计方法直接以检测到的人脸框作为输入，没有考虑到人脸框的不准确性，导致传统的人脸位姿估计方法在直接对位姿角度进行回归或分类时难以获得最优的预测效果，从而降低人脸位姿估计结果的准确性。
发明内容
有鉴于此，本公开实施例提供了一种人脸位姿估计方法、装置、电子设备及存储介质，以解决现有技术存在的无法充分获取人脸位姿的关键信息，人脸位姿估计结果的准确性较差的问题。
本公开实施例的第一方面，提供了一种人脸位姿估计方法，包括：获取包含人脸信息的目标图像，并将目标图像输入到预先构建的位姿估计模型中；在位姿估计模型中，利用浅层密集连接层对目标图像进行特征提取，得到包含浅层特征信息的多个第一特征图；将多个第一特征图作为深层特征复用层的输入，利用深层特征复用层分别对多个第一特征图执行信息融合操作得到第二特征图，以便在浅层特征信息中融入深层特征信息；利用注意力层对第二特征图中的人脸位姿信息进行提取，得到包含人脸位姿信息的第三特征图，利用分类器对第三特征图进行预测，得到第三特征图对应人脸位姿的预测结果，根据预测结果确定目标图像中的人脸位姿。
本公开实施例的第二方面，提供了一种人脸位姿估计装置，包括：获取模块，被配置 为获取包含人脸信息的目标图像，并将目标图像输入到预先构建的位姿估计模型中；提取模块，被配置为在位姿估计模型中，利用浅层密集连接层对目标图像进行特征提取，得到包含浅层特征信息的多个第一特征图；融合模块，被配置为将多个第一特征图作为深层特征复用层的输入，利用深层特征复用层分别对多个第一特征图执行信息融合操作得到第二特征图，以便在浅层特征信息中融入深层特征信息；预测模块，被配置为利用注意力层对第二特征图中的人脸位姿信息进行提取，得到包含人脸位姿信息的第三特征图，利用分类器对第三特征图进行预测，得到第三特征图对应人脸位姿的预测结果，根据预测结果确定目标图像中的人脸位姿。
本公开实施例的第三方面，提供了一种电子设备，包括存储器，处理器及存储在存储器上并可在处理器上运行的计算机程序，处理器执行程序时实现上述方法的步骤。
本公开实施例的第四方面，提供了一种计算机可读存储介质，该计算机可读存储介质存储有计算机程序，该计算机程序被处理器执行时实现上述方法的步骤。
本公开实施例采用的上述至少一个技术方案能够达到以下有益效果：
通过获取包含人脸信息的目标图像，并将目标图像输入到预先构建的位姿估计模型中；在位姿估计模型中，利用浅层密集连接层对目标图像进行特征提取，得到包含浅层特征信息的多个第一特征图；将多个第一特征图作为深层特征复用层的输入，利用深层特征复用层分别对多个第一特征图执行信息融合操作得到第二特征图，以便在浅层特征信息中融入深层特征信息；利用注意力层对第二特征图中的人脸位姿信息进行提取，得到包含人脸位姿信息的第三特征图，利用分类器对第三特征图进行预测，得到第三特征图对应人脸位姿的预测结果，根据预测结果确定目标图像中的人脸位姿。本公开能够充分获取人脸位姿的关键信息，使人脸位姿估计结果更加精准。
附图说明
为了更清楚地说明本公开实施例中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1是本公开实施例提供的位姿估计模型的网络结构示意图；
图2是本公开实施例提供的人脸位姿估计方法的流程示意图；
图3是本公开实施例提供的人脸位姿估计装置的结构示意图；
图4是本公开实施例提供的电子设备的结构示意图。
具体实施方式
以下描述中，为了说明而不是为了限定，提出了诸如特定系统结构、技术之类的具体细节，以便透彻理解本公开实施例。然而，本领域的技术人员应当清楚，在没有这些具体细节的其它实施例中也可以实现本公开。在其它情况中，省略对众所周知的系统、装置、电路以及方法的详细说明，以免不必要的细节妨碍本公开的描述。
在人脸位姿估计任务中，由于图像中人脸位置和大小的多样性，不同人脸在各个角度下的差异性，以及复杂的背景环境等因素，对算法精确估计人脸位姿角度带来困难。
在传统的人脸位姿估计算法中，通过直接以检测到的人脸框作为输入，没有考虑到人脸框的不准确性，不能充分利用人脸及周围信息进行模型优化。位姿估计需要捕捉位姿角度信息而忽略一些无关的面部特征，传统基于通用识别模型的骨干网络和neck模块由于缺乏有效设计，不能充分获取位姿关键信息。另外，传统直接对角度进行回归或分类的做法过于直接，难以获得最优效果。导致传统的人脸位姿估计方法在直接对位姿角度进行回归或分类时难以获得最优的预测效果，从而降低人脸位姿估计结果的准确性。
鉴于以上现有技术中的问题，本公开提出一种全新的基于神经网络的位姿估计模型的人脸位姿估计方法，本公开的神经网络模型中主要包含骨干网络、Neck模块和Head模块。在对位姿估计模型进行训练时，所采用的训练集中包含人脸框以及多个点位的位姿标注，包括由多个人脸框的标注点以及多个位姿角度所组成的位姿标注信息。并且通过保留原始图像，并基于原始图像进行数据增广，从而实现对位姿估计模型的增广训练。利用位姿估计模型中的浅层密集连接层及深层特征复用层所组成的骨干网络，对人脸图像中的浅层特征和深层特征进行融合，基于注意力模块以及分组角度回归模块精确估计图片中人脸的位姿角度。
下面结合附图对本公开中位姿估计模型的网络结构进行详细说明，图1是本公开实施例提供的位姿估计模型的网络结构示意图；如图1所示，该位姿估计模型的网络结构具体可以包括：
在骨干网络中包含浅层密集连接层(对应C1～C5)及深层特征复用层(对应P3～P5)，浅层密集连接层主要用于提取人脸图像中的前向特征，密集连接是指每个模块的输出除了作为下一个模块的输入，还作为其后其他模块的输入；深层特征复用层用于在进行特征表达时，将提取到的浅层特征信息中融入深层特征信息，从而得到包含更多语义信息(即位姿信息)的特征图。
Neck模块由SE注意力模块和基于Transformer的特征变换模块组成，SE注意力模块通过给不同通道的特征进行加权来放大重要特征，同时抑制相对不重要的特征。Transformer 通过巧妙的结构设计，具有较强的特征提取能力，通过多头注意力机制提取有效特征，从而突出特征图中的重要特征，并抑制干扰特征。
Head模块对应分组角度回归模块(即分类器)，通过对每种人脸位姿设置多个分类器，每个分类器分别预测多个角度对应的概率值，通过每个角度对应的概率值计算分类器的预测结果，最后将全部分类器的预测结果相加得到最终的估计结果。从而确定人脸图像中全部位姿对应的角度值。
图2是本公开实施例提供的人脸位姿估计方法的流程示意图。图2的人脸位姿估计方法可以由服务器执行。如图2所示，该人脸位姿估计方法具体可以包括：
S201，获取包含人脸信息的目标图像，并将目标图像输入到预先构建的位姿估计模型中；
S202，在位姿估计模型中，利用浅层密集连接层对目标图像进行特征提取，得到包含浅层特征信息的多个第一特征图；
S203，将多个第一特征图作为深层特征复用层的输入，利用深层特征复用层分别对多个第一特征图执行信息融合操作得到第二特征图，以便在浅层特征信息中融入深层特征信息；
S204，利用注意力层对第二特征图中的人脸位姿信息进行提取，得到包含人脸位姿信息的第三特征图，利用分类器对第三特征图进行预测，得到第三特征图对应人脸位姿的预测结果，根据预测结果确定目标图像中的人脸位姿。
具体地，本公开实施例的目标图像是指对采集到的原始图像进行人脸检测后的人脸图像，即位姿估计模型的输入是人脸检测后的人脸图像，而并非原始图像。在利用位姿估计模型进行位姿角度的预测之前，需要先构建位姿估计模型并进行训练，之后将训练后的位姿估计模型作为实际使用时的模型。
进一步地，在本公开实施例的人脸位姿估计方法中，通过将检测后的人脸图像(即目标图像)作为位姿估计模型的输入，利用位姿估计模型进行预测，输出3个位姿角度，即俯仰角、偏航角和翻滚角。由于不同人脸及人脸检测模型的差异性，人脸边界不具有完全统一的标准，因此利用检测后的人脸图像进行训练得到的位姿估计模型具有更好的鲁棒性。
根据本公开实施例提供的技术方案，本公开通过获取包含人脸信息的目标图像，并将目标图像输入到预先构建的位姿估计模型中；在位姿估计模型中，利用浅层密集连接层对目标图像进行特征提取，得到包含浅层特征信息的多个第一特征图；将多个第一特征图作为深层特征复用层的输入，利用深层特征复用层分别对多个第一特征图执行信息融合操作得到第二特征图，以便在浅层特征信息中融入深层特征信息；利用注意力层对第二特征图中的人脸位姿信息进行提取，得到包含人脸位姿信息的第三特征图，利用分类器对第三特征图进行预 测，得到第三特征图对应人脸位姿的预测结果，根据预测结果确定目标图像中的人脸位姿。本公开能够充分获取人脸位姿的关键信息，使人脸位姿估计结果更加精准。
在一些实施例中，采用以下方式构建位姿估计模型，包括：获取包含人脸信息的原始图像，利用人脸检测模型对原始图像进行检测，得到原始图像对应的人脸图像以及人脸框，并获取原始图像中的人脸位姿信息，利用人脸图像、人脸框的位置坐标、以及人脸位姿信息生成第一数据集；基于原始图像以及人脸框的位置坐标，利用预设的裁剪方式对原始图像进行裁剪，得到裁剪后的人脸图像，利用裁剪后的人脸图像、人脸框的位置坐标、以及人脸位姿信息生成第二数据集；对第一数据集以及第二数据集进行组合得到训练集，利用训练集对位姿估计模型进行训练，得到训练后的位姿估计模型。
具体地，位姿估计模型的训练主要包括两部分，即数据集的标注以及训练时的数据增广，下面结合具体实施例分别对这两部分的内容进行详细介绍，具体可以包括以下内容：
在数据集的标注过程中，对于一批待标注数据集，首先使用传统算法获取每张原始图像X中人脸对应的3个位姿角度，分别记为p(俯仰角)、y(偏航角)和r(翻滚角)。然后使用RetinaFace模型进行人脸检测得到人脸框，并确定人脸框对应的左上角坐标(x1,y1)，以及人脸框的宽(w)和高(h)。通过保留原始图像X，并将图像标签记录为Y＝[p,y,r,x1,y1,w,h]。
进一步地，在经过人脸检测后得到的人脸框中，除了可以用上述的左上角坐标以及的宽和高作为检测框(即人脸框)对应的4个标注点之外，还可以将人脸框的矩形对应的四个顶点坐标作为4个标注点。如此一来，训练集中的标注信息就包含了7点位姿标注，即4个检测框的标注点以及3个位姿角度。将上述检测后的人脸图像以及由7点位姿标注组成的标注信息作为第一数据集。
在训练时的数据增广操作中，通过随机裁剪的增广方式，基于原始图像，按照一定的裁剪比例(比如0.5倍)对原始图像中的人脸图像进行随机范围的裁剪，比如随机裁剪比人脸检测框略大一点(1.0～1.2倍)的人脸图像，从而产生与第一数据集中的人脸图像相似的人脸图像，将新产生的人脸图像以及该人脸图像对应的由7点位姿标注组成的标注信息作为第二数据集。
在本公开实施例中，通过数据增广的方式，扩大了训练数据集的规模，在利用增广后的训练集对位姿估计模型进行训练时，不仅减小了标注框不准确带来的影响，还扩充了数据量提高模型的鲁棒性。
在一些实施例中，训练集中包含人脸图像以及标注信息，将标注信息用于作为模型训练时的标签，标注信息中包含人脸框对应的多个标注点，以及多个位姿角度；其中，人脸框 的标注点包括人脸框对应的任一角点坐标、以及人脸框的宽度和高度，位姿角度包括俯仰角、偏航角和翻滚角。
在一些实施例中，利用浅层密集连接层对目标图像进行特征提取，得到包含浅层特征信息的多个第一特征图，包括：浅层密集连接层中包含多个依次连接的卷积模块，利用每个卷积模块依次对输入到卷积模块的特征图执行卷积运算，并且将每个卷积模块的输出作为下一个卷积模块的输入，每个卷积模块的输入中还包含之前卷积模块的输出，将浅层密集连接层中最后多个卷积模块的输出作为第一特征图。
具体地，由于人脸位姿分类需要考虑五官位置和头像的整体角度，因此人脸图像中的部分浅层信息和深层抽象信息都具有重要作用，因此本公开实施例提出一种由浅层密集连接层及深层特征复用层所组成的骨干网络。通过骨干网络提取人脸图像中的浅层特征信息和深层特征信息，并将浅层特征信息融入到深层特征信息中，得到包含更多语义信息的特征图。
进一步地，在浅层密集连接层中进行前向特征的提取时，在位姿估计模型中将浅层密集连接层分为C1、C2、C3、C4和C5共5个卷积模块，密集连接是指每个卷积模块的输出除了作为下一个卷积模块的输入外，还作为后面其他卷积模块的输入。这样不仅使深层特征复用层可以接收到更多的浅层信息，同时提高特征的对位姿信息的表达能力，另外提高了特征的利用效率，每部分提取的特征都可以被更多模块利用。
在一些实施例中，利用深层特征复用层分别对多个第一特征图执行信息融合操作得到第二特征图，以便在浅层特征信息中融入深层特征信息，包括：深层特征复用层中包含与第一特征图的数量相对应的卷积模块，利用深层特征复用层的卷积模块，对第一特征图进行卷积变换得到第二特征图，以便在包含浅层特征信息的第二特征图中融入深层特征信息，对第二特征图进行全局平均池化，得到全局平均池化后对应的第二特征图。
具体地，在利用深层特征复用层进行特征表达时，通过将浅层特征信息融入到深层特征信息中去，如图1所示，使用卷积模块C3、C4、C5的输出分别做3个不同层次的特征表达，即P3、P4和P5，从而在C3、C4、C5输出的特征图中融入深层特征信息。下面结合上述实施例中的图1对深层特征复用层的实现方式进行详细说明，具体可以包括以下内容：
C3、C4和C5对应浅层密集连接层中的卷积模块，将C3、C4和C5的输出作为P3、P4和P5的输入，利用深层特征复用层中的卷积模块对C3、C4和C5输出的特征图进行卷积运算，输出包含语义信息(即位姿信息)的特征图。在对P3、P4和P5进行卷积运算时，采用以下规则：在计算P3时，将C3、C4和C5分别做1x1卷积变换到(28，28，512)的特征图，从而使P3融合C3、C4和C5的信息；在计算P4时，将C4、C5分别做1x1卷积变换到(14，14，512)的特征图，从而融合C4和C5的信息；在计算P5时，将C5做1x1 卷积变换到(7，7，512)的特征图，只使用C5的信息。最后将P3、P4和P5分别做全局平均池化，分别得到长度为128、256和512的特征向量，并输入注意力层中。
在一些实施例中，注意力层包括SE注意力模块和特征变换模块，利用注意力层对第二特征图中的人脸位姿信息进行提取，得到包含人脸位姿信息的第三特征图，包括：利用SE注意力模块对第二特征图中的特征通道进行权重计算，并根据通道权重对特征通道进行加权得到加权后的第二特征图；利用特征变换模块对加权后的第二特征图进行特征提取，得到包含有效特征信息的第三特征图，有效特征信息包含人脸位姿信息。
具体地，为了更好地从骨干网络提取的特征信息中抽取位姿信息，本公开提供了基于SE注意力模块和Transformer的Neck模块。其中，利用SE注意力模块给不同通道的特征进行加权来放大重要特征，同时抑制相对不重要的特征。而Transformer具有较强的特征提取能力，能够突出重要特征，抑制干扰特征。
进一步地，SE注意力模块中包含两个全连接层和一个sigmoid层(即归一化层)，假设输入到SE注意力模块中的第二特征图的通道数为c，那么将通道数为c的第二特征图，经过两个全连接层和一个sigmoid层处理得到每个特征通道的通道权重，然后将每个通道权重与其对应的特征通道相乘，得到加权后的特征图。
进一步地，在Transformer(特征变换模块)中，将输入的加权后的特征图，经过多头注意力(Multi-Head Attention)和多层全连接层MLP的处理，得到变换后的c组特征图。通过Transformer的多头注意力机制能够提取特征图中的有效特征，从而更加突出重要特征。
在一些实施例中，利用分类器对第三特征图进行预测，得到第三特征图对应人脸位姿的预测结果，根据预测结果确定目标图像中的人脸位姿，包括：每一种人脸位姿对应多个第三特征图，每个第三特征图对应多个分类器，每个分类器用于根据第三特征图预测若干数量的角度值，根据若干数量的角度值计算每个分类器预测的位姿角度，将全部分类器预测的位姿角度进行求和，得到每一种人脸位姿对应的位姿角度，将三种人脸位姿对应的位姿角度作为对目标图像中人脸位姿的估计结果。
具体地，为了得到精确的位姿角度，本公开在位姿估计模型中添加了分组角度回归模块，也称为分类器模块，即图1中的Head模块部分。对于每种位姿角度，每组特征可以包含3个分类器，而每个分类器预测10个角度值。例如第一个分类器预测0°到9°的概率值，利用模型预测这10个角度的概率值(p00,p01,…,p09)，则该分类器的预测结果为angle0＝p01*0+p02*1+…+p09*9。最后将9个分类器的结果相加，即可得到最终的位姿角度。分别对3个位姿角度做如上计算，得到最终预测的三个位姿角度。
进一步地，输入到Head模块中的每个第三特征图分别对应三个分类器，每个分类器用 于预测不同的10个角度值，根据每个分类器的预测结果，计算全部分类器(共9个分类器)的最终预测结果，将Head模块的最终预测结果作为当前位姿角度对应的角度值。针对每一个位姿角度，模型均会输出一个角度值，将全部位姿角度对应的角度值作为对目标图像中人脸位姿的估计结果。
根据本公开实施例提供的技术方案，本公开采用7点位姿标注(4个检测框标注点和3个位姿角度)的数据集格式，并基于该标注方案设计了人脸图像的数据增广，从而扩大训练集的数据量。本公开基于对人脸图像位姿特征的分析，设计了浅层密集连接及深层特征复用的骨干网络，提高了特征的利用效率和表达能力。本公开还公开了基于SE注意力和Transformer的Neck模块，从而进一步抽取了位姿的关键信息。本公开在模型中添加分组角度回归模块，通过27个分类器对3个位姿角度进行更加精准的预测。
下述为本公开装置实施例，可以用于执行本公开方法实施例。对于本公开装置实施例中未披露的细节，请参照本公开方法实施例。
图3是本公开实施例提供的人脸位姿估计装置的结构示意图。如图3所示，该人脸位姿估计装置包括：
获取模块301，被配置为获取包含人脸信息的目标图像，并将目标图像输入到预先构建的位姿估计模型中；
提取模块302，被配置为在位姿估计模型中，利用浅层密集连接层对目标图像进行特征提取，得到包含浅层特征信息的多个第一特征图；
融合模块303，被配置为将多个第一特征图作为深层特征复用层的输入，利用深层特征复用层分别对多个第一特征图执行信息融合操作得到第二特征图，以便在浅层特征信息中融入深层特征信息；
预测模块304，被配置为利用注意力层对第二特征图中的人脸位姿信息进行提取，得到包含人脸位姿信息的第三特征图，利用分类器对第三特征图进行预测，得到第三特征图对应人脸位姿的预测结果，根据预测结果确定目标图像中的人脸位姿。
在一些实施例中，图3的获取模块301还采用以下方式构建位姿估计模型，包括：获取包含人脸信息的原始图像，利用人脸检测模型对原始图像进行检测，得到原始图像对应的人脸图像以及人脸框，并获取原始图像中的人脸位姿信息，利用人脸图像、人脸框的位置坐标、以及人脸位姿信息生成第一数据集；基于原始图像以及人脸框的位置坐标，利用预设的裁剪方式对原始图像进行裁剪，得到裁剪后的人脸图像，利用裁剪后的人脸图像、人脸框的位置坐标、以及人脸位姿信息生成第二数据集；对第一数据集以及第二数据集进行组合得到训练集，利用训练集对位姿估计模型进行训练，得到训练后的位姿估计模型。
在一些实施例中，训练集中包含人脸图像以及标注信息，将标注信息用于作为模型训练时的标签，标注信息中包含人脸框对应的多个标注点，以及多个位姿角度；其中，人脸框的标注点包括人脸框对应的任一角点坐标、以及人脸框的宽度和高度，位姿角度包括俯仰角、偏航角和翻滚角。
在一些实施例中，浅层密集连接层中包含多个依次连接的卷积模块，图3的提取模块302利用每个卷积模块依次对输入到卷积模块的特征图执行卷积运算，并且将每个卷积模块的输出作为下一个卷积模块的输入，每个卷积模块的输入中还包含之前卷积模块的输出，将浅层密集连接层中最后多个卷积模块的输出作为第一特征图。
在一些实施例中，深层特征复用层中包含与第一特征图的数量相对应的卷积模块，图3的融合模块303利用深层特征复用层的卷积模块，对第一特征图进行卷积变换得到第二特征图，以便在包含浅层特征信息的第二特征图中融入深层特征信息，对第二特征图进行全局平均池化，得到全局平均池化后对应的第二特征图。
在一些实施例中，注意力层包括SE注意力模块和特征变换模块，图3的预测模块304利用SE注意力模块对第二特征图中的特征通道进行权重计算，并根据通道权重对特征通道进行加权得到加权后的第二特征图；利用特征变换模块对加权后的第二特征图进行特征提取，得到包含有效特征信息的第三特征图，有效特征信息包含人脸位姿信息。
在一些实施例中，每一种人脸位姿对应多个第三特征图，每个第三特征图对应多个分类器，每个分类器用于根据第三特征图预测若干数量的角度值，根据若干数量的角度值计算每个分类器预测的位姿角度，将全部分类器预测的位姿角度进行求和，得到每一种人脸位姿对应的位姿角度，将三种人脸位姿对应的位姿角度作为对目标图像中人脸位姿的估计结果。
应理解，上述实施例中各步骤的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本公开实施例的实施过程构成任何限定。
图4是本公开实施例提供的电子设备4的结构示意图。如图4所示，该实施例的电子设备4包括：处理器401、存储器402以及存储在该存储器402中并且可以在处理器401上运行的计算机程序403。处理器401执行计算机程序403时实现上述各个方法实施例中的步骤。或者，处理器401执行计算机程序403时实现上述各装置实施例中各模块/单元的功能。
示例性地，计算机程序403可以被分割成一个或多个模块/单元，一个或多个模块/单元被存储在存储器402中，并由处理器401执行，以完成本公开。一个或多个模块/单元可以是能够完成特定功能的一系列计算机程序指令段，该指令段用于描述计算机程序403在电子设备4中的执行过程。
电子设备4可以是桌上型计算机、笔记本、掌上电脑及云端服务器等电子设备。电子设 备4可以包括但不仅限于处理器401和存储器402。本领域技术人员可以理解，图4仅仅是电子设备4的示例，并不构成对电子设备4的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如，电子设备还可以包括输入输出设备、网络接入设备、总线等。
处理器401可以是中央处理单元(Central Processing Unit，CPU)，也可以是其它通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其它可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
存储器402可以是电子设备4的内部存储单元，例如，电子设备4的硬盘或内存。存储器402也可以是电子设备4的外部存储设备，例如，电子设备4上配备的插接式硬盘，智能存储卡(Smart Media Card，SMC)，安全数字(Secure Digital，SD)卡，闪存卡(Flash Card)等。进一步地，存储器402还可以既包括电子设备4的内部存储单元也包括外部存储设备。存储器402用于存储计算机程序以及电子设备所需的其它程序和数据。存储器402还可以用于暂时地存储已经输出或者将要输出的数据。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将装置的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中，上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。另外，各功能单元、模块的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。上述系统中单元、模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本公开的范围。
在本公开所提供的实施例中，应该理解到，所揭露的装置/计算机设备和方法，可以通 过其它的方式实现。例如，以上所描述的装置/计算机设备实施例仅仅是示意性的，例如，模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通讯连接可以是通过一些接口，装置或单元的间接耦合或通讯连接，可以是电性，机械或其它的形式。
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
集成的模块/单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读存储介质中。基于这样的理解，本公开实现上述实施例方法中的全部或部分流程，也可以通过计算机程序来指令相关的硬件来完成，计算机程序可以存储在计算机可读存储介质中，该计算机程序在被处理器执行时，可以实现上述各个方法实施例的步骤。计算机程序可以包括计算机程序代码，计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。计算机可读介质可以包括：能够携带计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、电载波信号、电信信号以及软件分发介质等。需要说明的是，计算机可读介质包含的内容可以根据司法管辖区内立法和专利实践的要求进行适当的增减，例如，在某些司法管辖区，根据立法和专利实践，计算机可读介质不包括电载波信号和电信信号。
以上实施例仅用以说明本公开的技术方案，而非对其限制；尽管参照前述实施例对本公开进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本公开各实施例技术方案的精神和范围，均应包含在本公开的保护范围之内。

Claims (10)

  1. 一种人脸位姿估计方法，其特征在于，包括：
    获取包含人脸信息的目标图像，并将所述目标图像输入到预先构建的位姿估计模型中；
    在所述位姿估计模型中，利用浅层密集连接层对所述目标图像进行特征提取，得到包含浅层特征信息的多个第一特征图；
    将多个所述第一特征图作为深层特征复用层的输入，利用所述深层特征复用层分别对多个所述第一特征图执行信息融合操作得到第二特征图，以便在所述浅层特征信息中融入深层特征信息；
    利用注意力层对所述第二特征图中的人脸位姿信息进行提取，得到包含所述人脸位姿信息的第三特征图，利用分类器对所述第三特征图进行预测，得到所述第三特征图对应人脸位姿的预测结果，根据所述预测结果确定所述目标图像中的人脸位姿。
  2. 根据权利要求1所述的方法，其特征在于，采用以下方式构建所述位姿估计模型，包括：
    获取包含人脸信息的原始图像，利用人脸检测模型对所述原始图像进行检测，得到所述原始图像对应的人脸图像以及人脸框，并获取所述原始图像中的人脸位姿信息，利用所述人脸图像、所述人脸框的位置坐标、以及所述人脸位姿信息生成第一数据集；
    基于所述原始图像以及所述人脸框的位置坐标，利用预设的裁剪方式对所述原始图像进行裁剪，得到裁剪后的人脸图像，利用所述裁剪后的人脸图像、所述人脸框的位置坐标、以及所述人脸位姿信息生成第二数据集；
    对所述第一数据集以及所述第二数据集进行组合得到训练集，利用所述训练集对位姿估计模型进行训练，得到训练后的位姿估计模型。
  3. 根据权利要求2所述的方法，其特征在于，所述训练集中包含人脸图像以及标注信息，将所述标注信息用于作为模型训练时的标签，所述标注信息中包含所述人脸框对应的多个标注点，以及多个位姿角度；
    其中，所述人脸框的标注点包括人脸框对应的任一角点坐标、以及所述人脸框的宽度和高度，所述位姿角度包括俯仰角、偏航角和翻滚角。
  4. 根据权利要求1所述的方法，其特征在于，所述利用浅层密集连接层对所述目标图像进行特征提取，得到包含浅层特征信息的多个第一特征图，包括：
    所述浅层密集连接层中包含多个依次连接的卷积模块，利用每个卷积模块依次对输入到所述卷积模块的特征图执行卷积运算，并且将每个所述卷积模块的输出作为下一个卷积模 块的输入，每个所述卷积模块的输入中还包含之前卷积模块的输出，将所述浅层密集连接层中最后多个卷积模块的输出作为所述第一特征图。
  5. 根据权利要求1所述的方法，其特征在于，所述利用所述深层特征复用层分别对多个所述第一特征图执行信息融合操作得到第二特征图，以便在所述浅层特征信息中融入深层特征信息，包括：
    所述深层特征复用层中包含与所述第一特征图的数量相对应的卷积模块，利用所述深层特征复用层的卷积模块，对所述第一特征图进行卷积变换得到第二特征图，以便在包含所述浅层特征信息的第二特征图中融入所述深层特征信息，对所述第二特征图进行全局平均池化，得到所述全局平均池化后对应的第二特征图。
  6. 根据权利要求1所述的方法，其特征在于，所述注意力层包括SE注意力模块和特征变换模块，所述利用注意力层对所述第二特征图中的人脸位姿信息进行提取，得到包含所述人脸位姿信息的第三特征图，包括：
    利用所述SE注意力模块对所述第二特征图中的特征通道进行权重计算，并根据通道权重对所述特征通道进行加权得到加权后的第二特征图；
    利用所述特征变换模块对所述加权后的第二特征图进行特征提取，得到包含有效特征信息的第三特征图，所述有效特征信息包含人脸位姿信息。
  7. 根据权利要求1所述的方法，其特征在于，所述利用分类器对所述第三特征图进行预测，得到所述第三特征图对应人脸位姿的预测结果，根据所述预测结果确定所述目标图像中的人脸位姿，包括：
    每一种人脸位姿对应多个所述第三特征图，每个所述第三特征图对应多个分类器，每个分类器用于根据所述第三特征图预测若干数量的角度值，根据所述若干数量的角度值计算每个所述分类器预测的位姿角度，将全部所述分类器预测的位姿角度进行求和，得到每一种所述人脸位姿对应的位姿角度，将三种所述人脸位姿对应的位姿角度作为对所述目标图像中人脸位姿的估计结果。
  8. 一种人脸位姿估计装置，其特征在于，包括：
    获取模块，被配置为获取包含人脸信息的目标图像，并将所述目标图像输入到预先构建的位姿估计模型中；
    提取模块，被配置为在所述位姿估计模型中，利用浅层密集连接层对所述目标图像进行特征提取，得到包含浅层特征信息的多个第一特征图；
    融合模块，被配置为将多个所述第一特征图作为深层特征复用层的输入，利用所述深层特征复用层分别对多个所述第一特征图执行信息融合操作得到第二特征图，以便在所述浅 层特征信息中融入深层特征信息；
    预测模块，被配置为利用注意力层对所述第二特征图中的人脸位姿信息进行提取，得到包含所述人脸位姿信息的第三特征图，利用分类器对所述第三特征图进行预测，得到所述第三特征图对应人脸位姿的预测结果，根据所述预测结果确定所述目标图像中的人脸位姿。
  9. 一种电子设备，包括存储器，处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时实现如权利要求1所述的方法。
  10. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1所述的方法。
PCT/CN2022/107825 2022-02-11 2022-07-26 人脸位姿估计方法、装置、电子设备及存储介质 Ceased WO2023151237A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US18/834,647 US20250037305A1 (en) 2022-02-11 2022-07-26 Face pose estimation method and apparatus, electronic device, and storage medium
JP2024545137A JP7770581B2 (ja) 2022-02-11 2022-07-26 顔姿勢推定方法、装置、電子機器及び記憶媒体
EP22925596.3A EP4471737B1 (en) 2022-02-11 2022-07-26 Face pose estimation method and apparatus, electronic device, and storage medium
KR1020247024587A KR20240144139A (ko) 2022-02-11 2022-07-26 얼굴 포즈 추정 방법, 장치, 전자 디바이스 및 저장 매체

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210129785.7 2022-02-11
CN202210129785.7A CN114519881B (zh) 2022-02-11 2022-02-11 人脸位姿估计方法、装置、电子设备及存储介质

Publications (1)

Publication Number Publication Date
WO2023151237A1 true WO2023151237A1 (zh) 2023-08-17

Family

ID=81596114

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/107825 Ceased WO2023151237A1 (zh) 2022-02-11 2022-07-26 人脸位姿估计方法、装置、电子设备及存储介质

Country Status (6)

Country Link
US (1) US20250037305A1 (zh)
EP (1) EP4471737B1 (zh)
JP (1) JP7770581B2 (zh)
KR (1) KR20240144139A (zh)
CN (1) CN114519881B (zh)
WO (1) WO2023151237A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117036363A (zh) * 2023-10-10 2023-11-10 国网四川省电力公司信息通信公司 一种基于多特征融合的遮挡绝缘子检测方法
CN117133059A (zh) * 2023-08-18 2023-11-28 北京科技大学 一种基于局部注意力机制的人脸活体检测方法及装置
CN117173476A (zh) * 2023-09-05 2023-12-05 北京交通大学 一种单源域泛化行人再识别方法
CN118506400A (zh) * 2024-05-16 2024-08-16 中国矿业大学 人体姿态估计方法及系统、人体姿态估计模型

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114519881B (zh) * 2022-02-11 2024-11-19 深圳须弥云图空间科技有限公司 人脸位姿估计方法、装置、电子设备及存储介质
CN115661866A (zh) * 2022-11-07 2023-01-31 河海大学 注意力机制与深度学习相结合的红外行人检测方法、系统
CN115984934A (zh) * 2023-01-04 2023-04-18 北京龙智数科科技服务有限公司 人脸位姿估计模型的训练方法、人脸位姿估计方法及装置
CN116129501A (zh) * 2023-02-01 2023-05-16 北京龙智数科科技服务有限公司 人脸位姿估计方法及装置
CN116977417B (zh) * 2023-06-30 2024-10-29 北京百度网讯科技有限公司 位姿估计方法、装置、电子设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108256454A (zh) * 2018-01-08 2018-07-06 浙江大华技术股份有限公司 一种基于cnn模型的训练方法、人脸姿态估测方法及装置
JP2020113000A (ja) * 2019-01-10 2020-07-27 日本電信電話株式会社 物体検出認識装置、方法、及びプログラム
CN112766186A (zh) * 2021-01-22 2021-05-07 北京工业大学 一种基于多任务学习的实时人脸检测及头部姿态估计方法
CN114519881A (zh) * 2022-02-11 2022-05-20 深圳集智数字科技有限公司 人脸位姿估计方法、装置、电子设备及存储介质

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6959109B2 (en) * 2002-06-20 2005-10-25 Identix Incorporated System and method for pose-angle estimation
JP4951498B2 (ja) * 2007-12-27 2012-06-13 日本電信電話株式会社 顔画像認識装置、顔画像認識方法、顔画像認識プログラムおよびそのプログラムを記録した記録媒体
CN110232316A (zh) * 2019-05-05 2019-09-13 杭州电子科技大学 一种基于改进的dsod模型的车辆检测与识别方法
CN110321872B (zh) * 2019-07-11 2021-03-16 京东方科技集团股份有限公司 人脸表情识别方法及装置、计算机设备、可读存储介质
CN110837773A (zh) * 2019-09-27 2020-02-25 深圳市华付信息技术有限公司 基于深度学习的大角度人脸姿态估计方法
CN111339813B (zh) * 2019-09-30 2022-09-27 深圳市商汤科技有限公司 人脸属性识别方法、装置、电子设备和存储介质
CN111932555B (zh) * 2020-07-31 2025-04-01 上海商汤善萃医疗科技有限公司 一种图像处理方法及装置、计算机可读存储介质
CN113066013B (zh) * 2021-05-18 2023-02-10 广东奥普特科技股份有限公司 视觉图像增强的生成方法、系统、装置及存储介质
CN113393457B (zh) * 2021-07-14 2023-02-28 长沙理工大学 一种结合残差密集块与位置注意力的无锚框目标检测方法
CN113688723B (zh) * 2021-08-21 2024-03-19 河南大学 一种基于改进YOLOv5的红外图像行人目标检测方法
CN113901884B (zh) * 2021-09-15 2024-09-24 杭州欣禾圣世科技有限公司 基于特征匹配的人脸姿态估计方法、系统、装置及存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108256454A (zh) * 2018-01-08 2018-07-06 浙江大华技术股份有限公司 一种基于cnn模型的训练方法、人脸姿态估测方法及装置
JP2020113000A (ja) * 2019-01-10 2020-07-27 日本電信電話株式会社 物体検出認識装置、方法、及びプログラム
CN112766186A (zh) * 2021-01-22 2021-05-07 北京工业大学 一种基于多任务学习的实时人脸检测及头部姿态估计方法
CN114519881A (zh) * 2022-02-11 2022-05-20 深圳集智数字科技有限公司 人脸位姿估计方法、装置、电子设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4471737A4 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117133059A (zh) * 2023-08-18 2023-11-28 北京科技大学 一种基于局部注意力机制的人脸活体检测方法及装置
CN117133059B (zh) * 2023-08-18 2024-03-01 北京科技大学 一种基于局部注意力机制的人脸活体检测方法及装置
CN117173476A (zh) * 2023-09-05 2023-12-05 北京交通大学 一种单源域泛化行人再识别方法
CN117173476B (zh) * 2023-09-05 2024-05-24 北京交通大学 一种单源域泛化行人再识别方法
CN117036363A (zh) * 2023-10-10 2023-11-10 国网四川省电力公司信息通信公司 一种基于多特征融合的遮挡绝缘子检测方法
CN117036363B (zh) * 2023-10-10 2024-01-30 国网四川省电力公司信息通信公司 一种基于多特征融合的遮挡绝缘子检测方法
CN118506400A (zh) * 2024-05-16 2024-08-16 中国矿业大学 人体姿态估计方法及系统、人体姿态估计模型

Also Published As

Publication number Publication date
CN114519881B (zh) 2024-11-19
EP4471737A4 (en) 2025-04-30
EP4471737B1 (en) 2026-04-08
EP4471737A1 (en) 2024-12-04
CN114519881A (zh) 2022-05-20
KR20240144139A (ko) 2024-10-02
US20250037305A1 (en) 2025-01-30
JP2025504056A (ja) 2025-02-06
JP7770581B2 (ja) 2025-11-14

Similar Documents

Publication Publication Date Title
WO2023151237A1 (zh) 人脸位姿估计方法、装置、电子设备及存储介质
CN112784869B (zh) 一种基于注意力感知与对抗学习的细粒度图像识别方法
CN110969245B (zh) 医学图像的目标检测模型训练方法和装置
CN110246181B (zh) 基于锚点的姿态估计模型训练方法、姿态估计方法和系统
WO2022001623A1 (zh) 基于人工智能的图像处理方法、装置、设备及存储介质
CN110363817B (zh) 目标位姿估计方法、电子设备和介质
CN108171133B (zh) 一种基于特征协方差矩阵的动态手势识别方法
WO2018108129A1 (zh) 用于识别物体类别的方法及装置、电子设备
CN114549557A (zh) 一种人像分割网络训练方法、装置、设备及介质
CN115881265B (zh) 电子病历智能病案质控方法、系统、设备及存储介质
CN113850136A (zh) 基于yolov5与BCNN的车辆朝向识别方法及系统
CN119941731B (zh) 基于大模型的肺结节分析方法、系统、设备及介质
CN116597471A (zh) 人体跟踪方法、电子设备及存储介质
CN116704511A (zh) 设备清单文字识别方法和装置
CN120726243B (zh) 场景补全方法、装置、电子设备和存储介质
Xia et al. A multilevel fusion network for 3D object detection
CN114419103B (zh) 一种骨架检测跟踪方法、装置及电子设备
CN111723688A (zh) 人体动作识别结果的评价方法、装置和电子设备
WO2023109086A1 (zh) 文字识别方法、装置、设备及存储介质
WO2026001201A1 (zh) 关键点预测模型的训练方法、装置、设备、介质及产品
CN120198449A (zh) 基于空间3d高斯模型的语义特征嵌入开放分割方法
CN116881886A (zh) 身份识别方法、装置、计算机设备和存储介质
CN111967579B (zh) 使用卷积神经网络对图像进行卷积计算的方法和装置
Qian et al. Multi-Scale tiny region gesture recognition towards 3D object manipulation in industrial design
CN112990144B (zh) 一种用于行人重识别的数据增强方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22925596

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024545137

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 18834647

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 2022925596

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2022925596

Country of ref document: EP

Effective date: 20240829

NENP Non-entry into the national phase

Ref country code: DE

WWG Wipo information: grant in national office

Ref document number: 2022925596

Country of ref document: EP