CN108564016A - A kind of AU categorizing systems based on computer vision and method - Google Patents
A kind of AU categorizing systems based on computer vision and method Download PDFInfo
- Publication number
- CN108564016A CN108564016A CN201810297489.1A CN201810297489A CN108564016A CN 108564016 A CN108564016 A CN 108564016A CN 201810297489 A CN201810297489 A CN 201810297489A CN 108564016 A CN108564016 A CN 108564016A
- Authority
- CN
- China
- Prior art keywords
- key point
- face
- module
- classification
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
A kind of AU categorizing systems based on computer vision and method, including：Face key point extraction module, power module, control switch, USB interface, key point characteristic extracting module, AU sort modules, feedback module and training module；The invention uses CLNF model inspection face key points position, Laplacian Eigenmaps Dimension-reduced Strategies are taken to carry out dimensionality reduction to feature, and reduce keypoint quantity to rational range, use the method for more efficient Gabor characteristic, so that the complexity of scheme substantially reduces, calculation amount is reduced, without manually participating in, it can automate and carry out FACS codings, that is AU classifies, the specific method applied is both for face picture, compared to other systems, the efficiency of the classification of this system and accuracy are obtained for prodigious raising.
Description
Technical field
The invention belongs to computer approach control technology field, especially a kind of AU categorizing systems based on computer vision
And method, it is proposed that a kind of new participates in without artificial, the system that automation carries out FACS codings (i.e. AU classification), in the system
In, the specific method applied compares other systems, the efficiency of the classification of this system and accuracy both for face picture
It is obtained for prodigious raising.
Background technology
Depression is a kind of common disease in the whole world, and estimation shares 3.5 hundred million patients, every year because depression is led to committed suicide people
Number estimation is up to 1,000,000 people, and there is also many societies, pessimistic not self-confident and quite a lot of of abandoning for many slight patients with depression
Somatization；Some are studies have shown that the facial expression of depressive patients plays an important role in emotion expression service, if very
The facial expression of good analyzing processing patients with depression, it will be able to more effectively quickly make a definite diagnosis such disease, and obtain best
Therapeutic effect；
Current correlative study uses automatic face Expression analysis technology, to investigate the clinical data of depression, collects suppression
Influence data of the severity of strongly fragrant symptom to facial expression；Facial Action Coding System (FACS) is current facial expression annotation
Gold standard；Facial expression is resolved into the component part of referred to as motor unit (AU) by FACS；Motor unit is based on dissection
Face action corresponds to the contraction of specific facial muscles；For example, AU 12 encodes the contraction of greater zygomatic muscle, AU 6 encodes eye wheel
The contraction of orbiculares；In order to annotate facial expression, AU individually occurs or combines generation with other AU, which realizes 12
AU classifies (i.e. FACS codings), is whether one human face expression picture of mark includes one or more of AU 1-12, if not
Including which AU, which is labeled as -1, including being then labeled as 1；For example one picture comprising AU1 and AU3 is uniquely marked
Note is：1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1；Because carrying out AU classification manually to need profession, accurately judge,
So a large amount of interrogation times of clinician can be expended；
There is also a new invention, the patent No.s for the prior art：CN201710709113.2, title：The feature recognition sides AU
Method, device and storage medium, a kind of AU characteristic recognition methods of invention offer, device and computer readable storage medium are described
Method includes：The extract real-time facial image that is shot to camera using face recognition algorithms utilizes facial averaging model identification t
A face feature point, according to the position extraction local feature of t face feature point, feature vector input respectively it is trained and with
Whether the AU graders of this feature Region Matching are more than predetermined threshold value, judgement according to the probability of each AU features in prediction result
AU classification results；
Although the patent is also using the automatic technology for carrying out AU classification, the specific side that above-mentioned patent modules use
The problem of method classifies to face AU does not have specific aim, and what it is to uses such as critical point detection, feature extractions is all that general classification is asked
Topic method used, classification it is inefficient, accuracy rate also cannot be satisfied the demand of practical application, concretely：
1. the patent identifies that face feature point, facial averaging model know all characteristic points together by facial averaging model
It does not come out, is not concerned with the local feature of characteristic point, accuracy of identification is relatively low.And next our patent application uses
CLNF models, in CLNF models, patch experts are that each face key point training such as eyes, lip, eyebrow is individual
Point distribution collection, then each model inspection to key point be fitted to together according to full face key point overall distribution, accuracy is more
It is high；
2. above-mentioned patent directly extracts the feature of 76 key points, more and intrinsic dimensionality of counting is high, computationally intensive.Ability
60-70 human face characteristic point of domain generally use optimization, not only can guarantee and has extracted enough features, but also be unlikely to calculation amount mistake
Greatly；And the present invention carries out Laplacian Eigenmaps dimensionality reductions after extracting feature, to feature so that the dimension of feature
It is reduced to ten orders of magnitude by thousand orders of magnitude.The efficiency of classification is effectively promoted；
3. the local feature of above-mentioned patent extraction is sift features, and this patent uses Gabor characteristic, with sift spies
Sign is compared, and Gabor characteristic has stronger robustness to the error that face is aligned, and Gabor transformation simple principle is simple, easy
In realization, improves accuracy rate and reduce complexity.
Invention content
In order to solve the above technical problem, the present invention provides a kind of AU categorizing systems based on computer vision and method,
It is designed by the structure and algorithm of optimization, it is proposed that one kind is participated in without artificial, the system that automation carries out AU classification；This is
The input of system is a face picture, has used the method based on object classification in computer vision to solve the above problems, has changed
The accuracy rate of classification has been apt to it, which has that accuracy rate is high, the advantage without manually participating in i.e. codified mass data collection, can be with
Classify for practical AU.
A kind of AU categorizing systems based on computer vision and method, wherein：
A kind of AU categorizing systems based on computer vision, including：Face key point extraction module, power module, control
Switch, USB interface, key point characteristic extracting module, AU sort modules, feedback module and training module；
Further, the input terminal of the face key point extraction module is defeated for receiving AU face pictures to be sorted
Outlet is connect with the input terminal electric signal of the key point characteristic extracting module, one end telecommunications at another end and the USB interface
Number connection；
The face key point extraction module extracts structure using dlib human-face detectors, which uses the Dlib of c++
Face datection algorithm in library, the Face datection algorithm in the libraries Dlib use hog features and cascade classifier, it is very
Classical human face region detection algorithm；
The dlib human-face detectors are used in a pictures, only detect the boundary of human face region, and ignore other
Region；
Illustrate that the data set that the face key point extraction module uses is ck+ data set as a kind of applicating example；
Illustrate as an example, the ck+ data sets include 326 face pictures, per pictures all by professional
Mark has got well the classification results of AU manually, which trains its system using these data；
Further, input traffic of the USB interface as face key point extraction module, the extraction of face key point
Module is used to extract the coordinate position of picture key point；
Further, the power module for the face key point extraction module, key point characteristic extracting module with
And the power supply of AU sort modules；One end of the power module is connect with the other end electric signal of the USB interface, the power supply
The other end of module is electrically connected with control switch；
Further, the key point characteristic extracting module output end and the input terminal electric signal of the AU sort modules connect
It connects；Key point characteristic extracting module receives the coordinate information for the key point that face key point extraction module extracts, in key point
The nearby coordinates extracted region images feature at place, the foundation as subsequent image classification；
Further, after extracting feature, Laplacian Eigenmaps dimension reduction methods are used to feature so that feature
Dimension be reduced to ten orders of magnitude by thousand orders of magnitude, then they are input in AU sort modules；Thus, AU sort modules
Computation amount, the efficiency and effect of classification have obtained effective promotion simultaneously；
Illustrate as an example, the Laplacian Eigenmaps dimension reduction methods are to go to build from the angle of figure
Relationship between data, it requires related point between each other, close as far as possible in the space after dimensionality reduction；
Further, the output end of the AU sort modules is for exporting AU classification results, another end and the training
The input terminal electric signal of module connects, and AU sort modules receive near the face key point that key point characteristic extracting module extracts
The feature in region carries out Mathematical treatment in AU sort modules to these features of picture；CLNF models, extraction Gabor is special
Sign, Laplacian Eigenmaps dimensionality reductions and the classification of SVM bis- combine, and complete the AU classification of face picture；
This system ties CLNF models, extraction Gabor characteristic, Laplacian Eigenmaps dimensionality reductions and the classification of SVM bis-
It closes, completes the AU classification of face picture；
Illustrating as an example, the CLNF models are local nerve field model, are used for face key point extraction module,
CLNF includes two parts, PDM (points distribution models) and patch experts, PDM for capturing full face key point change in shape,
Patch experts are used to capture the local appearance variation of each key point；Specifically, patch experts are 68 people
Face key point training individually point distribution collection, then each model inspection to key point be fitted to together according to PDM；
Illustrate as an example, the SVM algorithm is a kind of two classification model, and solution, which aims at, determines one
The hyperplane of classification, to maximize the interval on feature space.For the feature of the low-dimensional linearly inseparable of input, core skill is added
It is ingeniously implicitly mapped in high-dimensional feature space, is allowed to linear separability；
Further, the output end of the training module is connect with the input terminal electric signal of the feedback module, training mould
Block learns the classification of face AU using machine learning algorithm, and trained result is fed back to feedback module, by feedback module
AU sort modules are passed to, the feedback result of AU sort module application trainings is right according to the feature of face picture to be sorted AU
Picture carries out AU classification, final output AU classification results；
Illustrate as an example, whether the AU classification results are to judge this pictures comprising one in AU 1-12
Or 12 multiple bit digitals；
A kind of AU sorting techniques based on computer vision, including：
Step 1: the foundation of data set：326 face pictures and every figure corresponding have been handled in extraction ck+ data sets
326 AU classification results that dynamic classification is completed；
Step 2: the selection of characteristic point：Face key point extraction module uses CLNF models, and people is extracted from face picture
The position coordinates of 68 key points of face；CLNF models train individually point distribution collection to each face key point, further according to complete
Face overall distribution is fitted to together so that the accuracy rate of detection gets a greater increase；Then key point characteristic extracting module profit
With the position coordinates of key point, the Gabor characteristic of the picture of key point near zone is extracted；
Further, the Gabor characteristic is a kind of feature being used for describing image texture information, to the mistake of face alignment
Difference has very strong robustness, therefore extracts the Gabor characteristic of 68 key point near zones of picture；Gabor characteristic is main
Adding window is carried out to signal in frequency domain by Gabor cores, so as to describe the local frequency information of signal；One Gabor core
The response condition of some frequency neighborhood of image can be got, this response results is a feature of image, with multiple and different frequencies
The Gabor cores of rate go obtain image different frequency neighborhood response condition, can be formed image each frequency band feature,
This feature is used for describing the frequency information of image；In the present invention, we near each key point, with 5 kinds of sizes, 8
40 Gabor cores in direction extract Gabor characteristic, therefore the intrinsic dimensionality extracted per pictures is：5*8*68=2720；
As one kind in step 2 for example, the workflow of the face key point extraction module includes：
1. inputting a face picture, the training stage inputs the picture in ck+ data sets, and it is to be sorted that test phase inputs AU
Face picture；
2. detecting face boundary using the human-face detector of dlib；
3. according to face border cuts picture；
4. to the picture after cutting, with CLNF model inspection face key points；
5. exporting the position coordinates of face key point.
As one kind in step 2 for example, the workflow of the key point characteristic extracting module includes：
1. receiving the coordinate position of face key point from face key point extraction module；
2. extracting Gabor characteristic in face key point near zone；
3. the Gabor characteristic extracted is done normalized；
4. the Gabor characteristic after normalization is written in txt file；
5. the txt file of output storage Gabor characteristic.
Step 3: parameter training：AU sort modules utilize the Gabor characteristic learning classification in step 2, pass through training mould
Algorithm study in block, training and the model parameter for optimizing whole system, and fed back in AU sort modules by feedback module；
One kind as step 3 is for example, the training of the AU sort modules, testing process include：
Training flow：
1. corresponding Gabor characteristic in AU classification results and txt file that input ck+ data sets have marked；
2. choosing first AU, dimensionality reduction parameter is initialized, Laplacian Eigenmaps dimensionality reductions are carried out to feature；
3. calculating the quantity of training set and the positive and negative sample of test set, positive and negative sample is distinguished according to mark, according to quantitative requirement
Extract training set and the corresponding feature of the positive and negative sample of test set；
4. using SVM model training graders, the accuracy rate of test set is calculated；
5. updating dimensionality reduction parameter in variation range, find out so that the maximum dimensionality reduction parameter of classification accuracy；
6. being repeated the above process respectively to other 11 AU.
Testing process：
1. inputting the Gabor characteristic of AU pictures to be sorted；
2. according to the corresponding dimensionality reduction parameter of maximum accuracy rate that training flow obtains, the Gabor characteristic extracted is carried out
Laplacian Eigenmaps dimensionality reductions；
3. calculating classification results with SVM models；
4. 12 AU of pair whole classify；
5. exporting AU classification results, 12 classification results form final classification result.
Because the dimension per pictures is higher (2720 dimension), intrinsic dimensionality is reduced using dimension reduction method, facilitates meter
It calculates.In the libraries sklearn, the SpectralEmbedding functions under Manifold learning (manifold learning) directly invoke
Realize Laplacian Eigenmaps dimension reduction methods.Due to involved in the dimension reduction method to two parameters n_components, n_
Neighbors, n_components are used to specify the dimension of projection subspace, and it is nearest that n_neighbors is used for knn classification kinds
Adjacent quantity (knn classification determines the classification results according to the classification results of several Neighbor Points of a point).The two
Parameter is uncertain, so cycle is added on stream to find optimal parameter value；
In the present invention, we call the svm functions in the libraries sklearn, because to be sorted has 12 AU, and SVM is only
It can be used for two classification；So in the present invention, carrying out two classification to this 12 AU respectively, i.e., individually being instructed to each AU
Practice, to determine whether the picture contains this AU, finally this 12 classification results combine again；
In the training process, sample is divided into training set and test set by us, and training set quantity accounts for 75%, test set quantity
Account for 25%.In order to ensure classifying quality, the number of the positive and negative sample used in training set is identical.The standard tested by comparing test set
True rate is as a result, we choose suitable dimensionality reduction parameter.During the test, we drop feature using optimal dimensionality reduction parameter
Dimension, then classifies to AU with trained SVM classifier.
Step 4: measuring and calculation：After training, AU face picture files to be sorted are input to face key point and are carried
In modulus block, as a result being sequentially inputted in key point characteristic extracting module and AU sort modules step by step；
Step 5: feedback calculates：In AU sort modules, the training result passed back using feedback module, to test pictures
Result carry out AU classification, export AU classification results.
Advantageous effect：
1. the process automation that AU classifies completely, without manually participating in can be completed the classification of all 12 AU；
2. carrying out dimensionality reduction to feature by using Laplacian Eigenmaps Dimension-reduced Strategies, and reduce crucial points
Measure rational range, using the method for more efficient Gabor characteristic so that the complexity of scheme substantially reduces, and calculation amount subtracts
It is few；
3. by the innovation and fusion of a variety of methods, the efficiency and effect of AU classification have obtained effective promotion simultaneously；
4. technology, the Gabor characteristic of application fetches key point near zone of application CLNF model inspection face key points
Technology, using Laplacian Eigenmaps dimension reduction methods, using bis- sorting techniques of svm, by CLNF models, extraction Gabor
Feature, Laplacian Eigenmaps dimensionality reductions and the classification of SVM bis- combine, and complete the AU classification of face picture.
Description of the drawings
Fig. 1 is a kind of overall structure diagram of AU categorizing systems based on computer vision of the present invention
Specific implementation mode
In the following, shown in refer to the attached drawing 1, a kind of AU categorizing systems based on computer vision and method, wherein：
A kind of AU categorizing systems based on computer vision, including：Face key point extraction module 101, power module
102, control switch 103, usb 1 04, key point characteristic extracting module 105, AU sort modules 106, feedback module 107 with
And training module 108；
Further, the input terminal of the face key point extraction module 101 is used to receive AU face pictures to be sorted,
Output end is connect with the input terminal electric signal of the key point characteristic extracting module 105, another end and the usb 1 04
One end electric signal connection；
The face key point extraction module 101 extracts structure using dlib human-face detectors, and the structure is using c++'s
Face datection algorithm in the libraries Dlib, the Face datection algorithm in the libraries Dlib use hog features and cascade classifier, it is
Very classical human face region detection algorithm；
The dlib human-face detectors are used in a pictures, only detect the boundary of human face region, and ignore other
Region；
Illustrate that the data set that the face key point extraction module 101 uses is ck+ data as a kind of applicating example
Collection；
Illustrate as an example, the ck+ data sets include 326 face pictures, per pictures all by professional
Mark has got well the classification results of AU manually, which trains its system using these data；
Further, input traffic of the usb 1 04 as face key point extraction module 101, face are crucial
Point extraction module 101 is used to extract the coordinate position of picture key point；
Further, the power module 102 is for the face key point extraction module 101, key point feature extraction
The power supply of module 105 and AU sort modules 106；The other end of one end of the power module 102 and the usb 1 04
Electric signal connects, and the other end of the power module 102 is electrically connected with the control switch 103；
Further, the input terminal electricity of 105 output end of key point characteristic extracting module and the AU sort modules 106
Signal connects；Key point characteristic extracting module 105 receives the coordinate letter for the key point that face key point extraction module 101 extracts
Breath, in the nearby coordinates extracted region images feature where key point, the foundation as subsequent image classification；
Further, after extracting feature, Laplacian Eigenmaps dimension reduction methods are used to feature so that feature
Dimension be reduced to ten orders of magnitude by thousand orders of magnitude, then they are input in AU sort modules 106；Thus, which AU classifies
The computation amount of module 106, the efficiency and effect of classification have obtained effective promotion simultaneously；
Illustrate as an example, the Laplacian Eigenmaps dimension reduction methods are to go to build from the angle of figure
Relationship between data, it requires related point between each other, close as far as possible in the space after dimensionality reduction；
Further, the output end of the AU sort modules 106 is for exporting AU classification results, another end and the instruction
Practice the input terminal electric signal connection of module 108, AU sort modules 106 receive the people that key point characteristic extracting module 105 extracts
The feature of face key point near zone carries out Mathematical treatment in AU sort modules 106 to these features of picture；
This method ties CLNF models, extraction Gabor characteristic, Laplacian Eigenmaps dimensionality reductions and the classification of SVM bis-
It closes, completes the AU classification of face picture；
Illustrating as an example, the CLNF models are local nerve field model, are used for face key point extraction module,
CLNF includes two parts, PDM (points distribution models) and patch experts, PDM for capturing full face key point change in shape,
Patch experts are used to capture the local appearance variation of each key point；Specifically, patch experts are 68 people
Face key point training individually point distribution collection, then each model inspection to key point be fitted to together according to PDM；
Illustrate as an example, the SVM algorithm is a kind of two classification model, and solution, which aims at, determines one
The hyperplane of classification, to maximize the interval on feature space；For the feature of the low-dimensional linearly inseparable of input, core skill is added
It is ingeniously implicitly mapped in high-dimensional feature space, is allowed to linear separability；
Further, the output end of the training module 108 is connect with the input terminal electric signal of the feedback module 107,
Training module 108 learns the classification of face AU using machine learning algorithm, and trained result is fed back to feedback module, by
Feedback module 107 passes to AU sort modules 106, and the feedback result of 106 application training of AU sort modules is to be sorted according to AU
The feature of face picture carries out AU classification, final output AU classification results to picture；
Illustrate as an example, whether the AU classification results are to judge this pictures comprising one in AU 1-12
Or 12 multiple bit digitals；
A kind of AU sorting techniques based on computer vision, including：
Step 1: the foundation of data set：326 face pictures and every figure corresponding have been handled in extraction ck+ data sets
326 AU classification results that dynamic classification is completed；
Step 2: the selection of characteristic point：Face key point extraction module 101 uses CLNF models, is carried from face picture
Take the position coordinates of 68 key points of face；CLNF models train individually point distribution collection, then root to each face key point
It is fitted to together according to full face overall distribution so that the accuracy rate of detection gets a greater increase；Then key point feature extraction mould
Block 105 extracts the Gabor characteristic of the picture of key point near zone using the position coordinates of key point；
Further, the Gabor characteristic is a kind of feature being used for describing image texture information, to the mistake of face alignment
Difference has very strong robustness, therefore extracts the Gabor characteristic of 68 key point near zones of picture；Gabor characteristic is main
Adding window is carried out to signal in frequency domain by Gabor cores, so as to describe the local frequency information of signal；One Gabor core
The response condition of some frequency neighborhood of image can be got, this response results is a feature of image, with multiple and different frequencies
The Gabor cores of rate go obtain image different frequency neighborhood response condition, can be formed image each frequency band feature,
This feature is used for describing the frequency information of image；In the present invention, we near each key point, with 5 kinds of sizes, 8
40 Gabor cores in direction extract Gabor characteristic, therefore the intrinsic dimensionality extracted per pictures is：5*8*68=2720；
As one kind in step 2 for example, the workflow of the face key point extraction module 101 includes：
1. inputting a face picture, the training stage inputs picture in ck+ data sets, and it is to be sorted that test phase inputs AU
Face picture；
2. detecting face boundary using the human-face detector of dlib；
3. according to face border cuts picture；
4. the picture after pair cutting, with CLNF model inspection face key points；
5. exporting the position coordinates of face key point.
As one kind in step 2 for example, the workflow of the key point characteristic extracting module 105 includes：
1. receiving the coordinate position of face key point from face key point extraction module；
2. extracting Gabor characteristic in face key point near zone；
3. the Gabor characteristic extracted is done normalized；
4. the Gabor characteristic after normalization is written in txt file；
5. the txt file of output storage Gabor characteristic.
Step 3: parameter training：AU sort modules 106 pass through training using the Gabor characteristic learning classification in step 2
Algorithm study in module 108, training and the model parameter for optimizing whole system, and feed back to AU points by feedback module 107
In generic module 106；
One kind as step 3 is for example, the training of the AU sort modules 106, testing process include：
Training flow：
1. corresponding Gabor characteristic in AU classification results and txt file that input ck+ data sets have marked；
2. choosing first AU, dimensionality reduction parameter is initialized, Laplacian Eigenmaps dimensionality reductions are carried out to feature；
3. calculating the quantity of training set and the positive and negative sample of test set, positive and negative sample is distinguished according to mark, according to quantitative requirement
Extract training set and the corresponding feature of the positive and negative sample of test set；
4. using SVM model training graders, the accuracy rate of test set is calculated；
5. updating dimensionality reduction parameter in variation range, find out so that the maximum dimensionality reduction parameter of classification accuracy；
6. being repeated the above process respectively to other 11 AU.
Testing process：
1. inputting the Gabor characteristic of AU pictures to be sorted；
2. according to the corresponding dimensionality reduction parameter of maximum accuracy rate that training flow obtains, the Gabor characteristic extracted is carried out
Laplacian Eigenmaps dimensionality reductions；
3. calculating classification results with SVM models；
4. 12 AU of pair whole classify；
5. exporting AU classification results, 12 classification results form final classification result.
Because the dimension per pictures is higher (2720 dimension), intrinsic dimensionality is reduced using dimension reduction method, facilitates meter
It calculates；In the libraries sklearn, the SpectralEmbedding functions under Manifold learning (manifold learning) directly invoke
Realize Laplacian Eigenmaps dimension reduction methods.Due to involved in the dimension reduction method to two parameters n_components, n_
Neighbors, n_components are used to specify the dimension of projection subspace, and it is nearest that n_neighbors is used for knn classification kinds
Adjacent quantity (knn classification determines the classification results according to the classification results of several Neighbor Points of a point).The two
Parameter is uncertain, so cycle is added on stream to find optimal parameter value；
In the present invention, we call the svm functions in the libraries sklearn, because to be sorted has 12 AU, and SVM is only
It can be used for two classification；So in the present invention, carrying out two classification to this 12 AU respectively, i.e., individually being instructed to each AU
Practice, to determine whether the picture contains this AU, finally this 12 classification results combine again；
In the training process, sample is divided into training set and test set by us, and training set quantity accounts for 75%, test set quantity
Account for 25%.In order to ensure classifying quality, the number of the positive and negative sample used in training set is identical.The standard tested by comparing test set
True rate is as a result, we choose suitable dimensionality reduction parameter.During the test, we drop feature using optimal dimensionality reduction parameter
Dimension, then classifies to AU with trained SVM classifier.
Step 4: measuring and calculation：After training, AU face picture files to be sorted are input to face key point and are carried
In modulus block 101, as a result being sequentially inputted in key point characteristic extracting module 105 and AU sort modules 106 step by step；
Step 5: feedback calculates：In AU sort modules, the training result passed back using feedback module 107, to test chart
The result of piece carries out AU classification, exports AU classification results.
The process automation that the present invention completely classifies AU, without manually participating in can be completed the classification of all 12 AU,
Dimensionality reduction is carried out to feature by using Laplacian Eigenmaps Dimension-reduced Strategies, and reduces keypoint quantity to reasonably
Range uses the method for more efficient Gabor characteristic so that the complexity of scheme substantially reduces, and calculation amount is reduced；By a variety of
The innovation and fusion of method, the efficiency and effect of AU classification have obtained effective promotion simultaneously；Using CLNF model inspection faces
The technology of key point, the technology of the Gabor characteristic of application fetches key point near zone, using Laplacian Eigenmaps
Dimension reduction method, using bis- sorting techniques of svm, by CLNF models, extraction Gabor characteristic, Laplacian Eigenmaps dimensionality reductions and
The classification of SVM bis- combines, and completes the AU classification of face picture.
Disclosed above is only the specific embodiment of the application, and however, this application is not limited to this, any this field
Technical staff can think variation, should all fall in the protection domain of the application.
Claims (10)
1. a kind of AU categorizing systems based on computer vision, which is characterized in that including：Face key point extraction module, power supply
Module, control switch, USB interface, key point characteristic extracting module, AU sort modules, feedback module and training module；
The input terminal of the face key point extraction module is for receiving AU face pictures to be sorted, output end and the key
The input terminal electric signal of point feature extraction module connects, and another end is connect with one end electric signal of the USB interface；The USB
Input traffic of the interface as face key point extraction module, face key point extraction module is for extracting picture key point
Coordinate position；The power module is for the face key point extraction module, key point characteristic extracting module and AU classification
The power supply of module；One end of the power module is connect with the other end electric signal of the USB interface, the power module it is another
One end is electrically connected with control switch；The input terminal of the key point characteristic extracting module output end and the AU sort modules
Electric signal connects；Key point characteristic extracting module receives the coordinate information for the key point that face key point extraction module extracts,
The dimensionality reduction sides Laplacian Eigenmaps are used in the nearby coordinates extracted region images feature where key point, and to feature
Method so that be input to after the dimension reduction of feature to ten orders of magnitude in AU sort modules, the output end of the AU sort modules is used
In output AU classification results, another end is connect with the input terminal electric signal of the training module, the output of the training module
End is connect with the input terminal electric signal of the feedback module.
2. a kind of AU categorizing systems based on computer vision according to claim 1, which is characterized in that described
Laplacian Eigenmaps dimension reduction methods are from the relationship gone with the angle of figure between structure data, it requires mutual
It is related, it is close as far as possible in the space after dimensionality reduction.
3. a kind of AU categorizing systems based on computer vision according to claim 1, which is characterized in that the AU classification
Module receives the feature for the face key point near zone that key point characteristic extracting module extracts, right in AU sort modules
These features of picture carry out Mathematical treatment, by CLNF models, extraction Gabor characteristic, Laplacian Eigenmaps dimensionality reductions
Classify with SVM bis- and combine, completes the AU classification of face picture.
4. a kind of AU categorizing systems based on computer vision according to claim 3, which is characterized in that the CLNF moulds
Type is local nerve field model, is used for face critical point detection, CLNF includes two parts, PDM (points distribution models) and patch
For capturing full face key point change in shape, the part that patch experts are used to capture each key point is outer by experts, PDM
See variation；Individually point distribution collection is trained specifically, patch experts are 68 face key points, then each model
The key point detected is fitted to together according to PDM.
5. a kind of AU categorizing systems based on computer vision according to claim 1, which is characterized in that the trained mould
Block learns the classification of face AU using machine learning algorithm, and trained result is fed back to feedback module, by feedback module
AU sort modules are passed to, the feedback result of AU sort module application trainings is right according to the feature of face picture to be sorted AU
Picture carries out AU classification.
6. a kind of AU categorizing systems based on computer vision according to claim 5, which is characterized in that the AU classification
As a result judge whether this pictures includes 12 bit digitals of one or more of AU 1-12.
7. a kind of AU categorizing systems based on computer vision according to claim 1 or 3, which is characterized in that the people
Face key point extraction module extracts structure using dlib human-face detectors, and the structure is using the Face datection in the libraries Dlib of c++
Algorithm, the Face datection algorithm in the libraries Dlib use hog features and cascade classifier, the face key point extraction module
The data set used is ck+ data set；The ck+ data sets include 326 face pictures, per pictures all by professional's hand
Dynamic mark has got well the classification results of AU.
8. a kind of AU sorting techniques based on computer vision, which is characterized in that including following processing step：
Step 1: the foundation of data set：326 face pictures and every figure corresponding divide manually in extraction ck+ data sets
326 AU classification results that class is completed；
Step 2: the selection of characteristic point：Face key point extraction module uses CLNF models, and face is extracted from face picture
The position coordinates of 68 key points；CLNF models train individually point distribution collection to each face key point, whole further according to full face
Body fitting of distribution is to together so that the accuracy rate of detection gets a greater increase；Then key point characteristic extracting module utilizes pass
The position coordinates of key point extract the Gabor characteristic of the picture of key point near zone；
Step 3: parameter training：AU sort modules utilize the Gabor characteristic learning classification in step 2, by training module
Algorithm study, training simultaneously optimizes the model parameter of whole system, and is fed back in AU sort modules by feedback module；
Step 4: measuring and calculation：After training, AU face picture files to be sorted are input to face key point and extract mould
In block, as a result being sequentially inputted in key point characteristic extracting module and AU sort modules step by step；
Step 5: feedback calculates：In AU sort modules, the training result passed back using feedback module, to the knot of test pictures
Fruit carries out AU classification, exports AU classification results.
9. a kind of AU sorting techniques based on computer vision according to claim 8, which is characterized in that
The workflow of the face key point extraction module includes：A face picture is inputted, the training stage inputs from ck+ numbers
AU face pictures to be sorted are inputted according to picture, test phase is concentrated；Face boundary is detected using the human-face detector of dlib；
According to face border cuts picture；To the picture after cutting, with CLNF model inspection face key points；Export face key point
Position coordinates；
The workflow of the key point characteristic extracting module includes：Face key point is received from face key point extraction module
Coordinate position；Gabor characteristic is extracted in face key point near zone；The Gabor characteristic extracted is done into normalized；Return
Gabor characteristic after one change is written in txt file；The txt file of output storage Gabor characteristic；
The training of the AU sort modules, testing process include：
Training flow：Corresponding Gabor characteristic in AU classification results and txt file that input ck+ data sets have marked；Choose the
One AU initializes dimensionality reduction parameter, and Laplacian Eigenmaps dimensionality reductions are carried out to feature；Calculate training set and test set just
The quantity of negative sample distinguishes positive and negative sample according to mark, extracts training set according to quantitative requirement and the positive and negative sample of test set corresponds to
Feature；With SVM model training graders, the accuracy rate of test set is calculated；Dimensionality reduction parameter is updated in variation range, finding out makes
Obtain the maximum dimensionality reduction parameter of classification accuracy；Other 11 AU are repeated the above process respectively；
Testing process：Input the Gabor characteristic of AU pictures to be sorted；The corresponding drop of maximum accuracy rate obtained according to training flow
Parameter is tieed up, Laplacian Eigenmaps dimensionality reductions are carried out to the Gabor characteristic extracted；Classification results are calculated with SVM models；
Classify to all 12 AU；AU classification results are exported, 12 classification results form final classification result.
10. a kind of AU sorting techniques based on computer vision according to claim 8, which is characterized in that because every
The dimension of picture is higher, so being reduced intrinsic dimensionality using dimension reduction method, in the libraries sklearn, under Manifold learning
SpectralEmbedding functions, which directly invoke, realizes Laplacian Eigenmaps dimension reduction methods；Due in the dimension reduction method
It is related to two parameters n_components, n_neighbors, n_components and is used to specify the dimension of projection subspace,
N_neighbors is used for the quantity of knn classification kind arest neighbors；The two parameters are uncertain, so being added on stream
It recycles to find optimal parameter value；
The svm functions in the libraries sklearn are called, because to be sorted there are 12 AU, and SVM is only used for two classification；So
In the present invention, two classification are carried out to this 12 AU respectively, i.e., each AU are individually trained, whether to determine the picture
Containing this AU, finally this 12 classification results are combined again.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810297489.1A CN108564016A (en) | 2018-04-04 | 2018-04-04 | A kind of AU categorizing systems based on computer vision and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810297489.1A CN108564016A (en) | 2018-04-04 | 2018-04-04 | A kind of AU categorizing systems based on computer vision and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108564016A true CN108564016A (en) | 2018-09-21 |
Family
ID=63534059
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810297489.1A Pending CN108564016A (en) | 2018-04-04 | 2018-04-04 | A kind of AU categorizing systems based on computer vision and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108564016A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110458001A (en) * | 2019-06-28 | 2019-11-15 | 南昌大学 | A Convolutional Neural Network Sight Estimation Method and System Based on Attention Mechanism |
| CN110992455A (en) * | 2019-12-08 | 2020-04-10 | 北京中科深智科技有限公司 | Real-time expression capturing method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103065122A (en) * | 2012-12-21 | 2013-04-24 | 西北工业大学 | Facial expression recognition method based on facial motion unit combination features |
| CN106295678A (en) * | 2016-07-27 | 2017-01-04 | 北京旷视科技有限公司 | Neural metwork training and construction method and device and object detection method and device |
| CN106384083A (en) * | 2016-08-31 | 2017-02-08 | 上海交通大学 | Automatic face expression identification and information recommendation method |
| CN107633207A (en) * | 2017-08-17 | 2018-01-26 | 平安科技（深圳）有限公司 | AU characteristic recognition methods, device and storage medium |
-
2018
- 2018-04-04 CN CN201810297489.1A patent/CN108564016A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103065122A (en) * | 2012-12-21 | 2013-04-24 | 西北工业大学 | Facial expression recognition method based on facial motion unit combination features |
| CN106295678A (en) * | 2016-07-27 | 2017-01-04 | 北京旷视科技有限公司 | Neural metwork training and construction method and device and object detection method and device |
| CN106384083A (en) * | 2016-08-31 | 2017-02-08 | 上海交通大学 | Automatic face expression identification and information recommendation method |
| CN107633207A (en) * | 2017-08-17 | 2018-01-26 | 平安科技（深圳）有限公司 | AU characteristic recognition methods, device and storage medium |
Non-Patent Citations (5)
| Title |
|---|
| BALTRUSAITIS T ET AL: "Constrained Local Neural Fields for Robust Facial Landmark Detection in the Wild", 《IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS》 * |
| PAN S ET AL: "Facial action units for presentation attack detection", 《 INTERNATIONAL CONFERENCE ON EMERGING SECURITY TECHNOLOGIES》 * |
| S.MOHAMMAD MAVADATI ET AL: "Automatic detection of non-posed facial action units", 《IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING》 * |
| 杨宇: "《机械故障诊断的变量预测模式识别方法》", 31 August 2017 * |
| 牛新亚: "基于深度学习的人脸表情识别研究", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110458001A (en) * | 2019-06-28 | 2019-11-15 | 南昌大学 | A Convolutional Neural Network Sight Estimation Method and System Based on Attention Mechanism |
| CN110992455A (en) * | 2019-12-08 | 2020-04-10 | 北京中科深智科技有限公司 | Real-time expression capturing method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Chen et al. | Sckansformer: Fine-grained classification of bone marrow cells via kansformer backbone and hierarchical attention mechanisms | |
| CN109543526B (en) | True and false facial paralysis recognition system based on depth difference characteristics | |
| Foggia et al. | Benchmarking HEp-2 cells classification methods | |
| CN107403142B (en) | A kind of detection method of micro- expression | |
| TWI687159B (en) | Fry counting system and fry counting method | |
| CN108229268A (en) | Expression recognition and convolutional neural network model training method and device and electronic equipment | |
| CN104598888B (en) | A kind of recognition methods of face gender | |
| CN106023151A (en) | Traditional Chinese medicine tongue manifestation object detection method in open environment | |
| CN102194108A (en) | Smiley face expression recognition method based on clustering linear discriminant analysis of feature selection | |
| CN115910338A (en) | A method and device for assessing human health status based on multimodal biometrics | |
| CN119027983B (en) | Method, device and electronic equipment for identifying images of livestock and poultry diseases | |
| CN103258216A (en) | Regional deformation target detection method and system based on online learning | |
| CN117503056A (en) | Multi-scale respiratory event automatic positioning method and device based on dual-modal feature learning | |
| Bilang et al. | Cactaceae detection using MobileNet architecture | |
| Wang et al. | A ROI extraction method for wrist imaging applied in smart bone-age assessment system | |
| CN108564016A (en) | A kind of AU categorizing systems based on computer vision and method | |
| CN119359783B (en) | Posture detection method based on three-dimensional measurement of livestock | |
| Ponraj et al. | Deep learning with histogram of oriented gradients-based computer-aided diagnosis for breast cancer detection and classification | |
| CN107506747A (en) | Face identification system and method based on video data characteristic point | |
| CN117636053A (en) | Tongue surface image analysis method and device | |
| CN117727091A (en) | A human action recognition system and method based on computer vision | |
| Garg et al. | Automated detection and diagnosis of skin-lesion using transfer learning based YOLOv7 approach | |
| CN119672296B (en) | Fish disease detection method, device, computer equipment and storage medium | |
| CN121281120B (en) | Method for constructing obstructive sleep apnea feature recognition model | |
| Ma et al. | Research on Crop Leaf Disease Recognition Algorithm Based on Embedded Device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180921 |