WO2021114990A1 - 人脸畸变校正方法、装置、电子设备及存储介质 - Google Patents

人脸畸变校正方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2021114990A1
WO2021114990A1 PCT/CN2020/127551 CN2020127551W WO2021114990A1 WO 2021114990 A1 WO2021114990 A1 WO 2021114990A1 CN 2020127551 W CN2020127551 W CN 2020127551W WO 2021114990 A1 WO2021114990 A1 WO 2021114990A1
Authority
WO
WIPO (PCT)
Prior art keywords
face
face frame
corrected
pixel
view
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/127551
Other languages
English (en)
French (fr)
Inventor
王运
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority to EP20900298.9A priority Critical patent/EP4071707A4/en
Publication of WO2021114990A1 publication Critical patent/WO2021114990A1/zh
Priority to US17/835,728 priority patent/US12327339B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/80—Geometric correction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • This application relates to the field of image processing technology, and in particular to a method, device, electronic device, and storage medium for correcting facial distortion.
  • Perspective distortion refers to the fact that an object and its surrounding area are completely different from those seen in a standard lens. Due to changes in the relative proportions of the near and far features, they are bent or deformed.
  • the face distortion correction method, device, electronic equipment, and storage medium proposed in this application are used to solve the related technology, in the image collected by a camera with a larger field of view, the perspective distortion of the face at the edge of the image is more serious, which affects the user experience. problem.
  • the face distortion correction method proposed in an embodiment of the present application includes: performing face detection on the acquired image, determining the position of each face frame included in the image; judging each person according to the position of each face frame Whether the face frame is within the range of the preset angle of view; if at least part of the area of the first face frame is not within the range of the preset angle of view, perform distortion correction on the face in the first face frame To generate a corrected image.
  • the face distortion correction device proposed in another embodiment of the present application includes: a first determination module, configured to perform face detection on the acquired image, and determine the position of each face frame included in the image; a judgment module, It is used for judging whether each face frame is within the preset angle of view range according to the position of each face frame; the correction module is used for if at least part of the area of the first face frame is not within the preset angle of view range Within, the distortion correction is performed on the face in the first face frame to generate a corrected image.
  • the electronic device proposed in another embodiment of the present application includes: a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and is characterized in that, when the processor executes the program, the human Facial distortion correction method.
  • the face distortion correction method includes: performing face detection on the acquired image, determining the position of each face frame included in the image; determining whether each face frame is in a preset view according to the position of each face frame Within the field angle range; if at least part of the area of the first face frame is not within the preset field of view range, perform distortion correction on the face in the first face frame to generate a corrected image .
  • the computer-readable storage medium provided by another embodiment of the present application has a computer program stored thereon, and is characterized in that the method for correcting facial distortion is realized when the program is executed by a processor.
  • the face distortion correction method includes: performing face detection on the acquired image, determining the position of each face frame included in the image; determining whether each face frame is in a preset view according to the position of each face frame Within the field angle range; if at least part of the area of the first face frame is not within the preset field of view range, perform distortion correction on the face in the first face frame to generate a corrected image .
  • the face distortion correction method includes: performing face detection on the acquired image, determining the position of each face frame included in the image; determining whether each face frame is in a preset view according to the position of each face frame Within the field angle range; if at least part of the area of the first face frame is not within the preset field of view range, perform distortion correction on the face in the first face frame to generate a corrected image .
  • FIG. 1 is a schematic flowchart of a method for correcting face distortion according to an embodiment of the application
  • FIG. 2 is a schematic flowchart of another method for correcting face distortion provided by an embodiment of the application.
  • FIG. 3 is a schematic flowchart of another method for correcting face distortion according to an embodiment of the application.
  • FIG. 4 is a schematic structural diagram of a face distortion correction device provided by an embodiment of the application.
  • FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the application.
  • the face distortion correction method according to the embodiment of the present application includes:
  • each face frame judge whether each face frame is within the preset angle of view
  • the face distortion correction method before performing distortion correction on the face in the first face frame, the face distortion correction method further includes:
  • determining that the second face frame and the first face frame satisfy a preset condition includes:
  • the method before determining whether each face frame is within the preset field of view range, the method further includes:
  • the preset field of view range is determined according to the attributes of the camera module that collects the image, where the attributes of the camera module include the setting position of the camera module in the terminal and the field of view angle of the camera module.
  • performing distortion correction on the face in the first face frame includes:
  • the current correction operation is a translation operation
  • the current correction operation is a translation operation
  • the pixel to be corrected is translated.
  • the method further includes:
  • the pixel to be corrected is corrected.
  • the method further includes:
  • the correction processing of the pixel to be corrected is ended.
  • the face distortion correction device 40 of the embodiment of the present application includes:
  • the first determining module 41 is configured to perform face detection on the acquired image and determine the position of each face frame included in the image;
  • the judging module 42 is used for judging whether each face frame is within the preset angle of view range according to the position of each face frame.
  • the correction module 43 is configured to perform distortion correction on the face in the first face frame to generate a corrected image if at least a part of the area of the first face frame is not within the preset angle of view.
  • the facial distortion correction device 40 further includes:
  • the projection module is used to perform conformal projection on the first face frame to generate the second face frame;
  • the second determining module is used to determine that the second face frame and the first face frame satisfy a preset condition.
  • the second determining module is further configured to determine that the length of at least one side in the second face frame and the length of the corresponding side in the first face frame satisfy a preset condition.
  • the facial distortion correction device 40 further includes:
  • the third determining module is used to determine the preset field of view range according to the attributes of the camera module that collects the image, where the attributes of the camera module include the location of the camera module in the terminal and the field of view of the camera module angle.
  • the correction module 43 is further configured to perform correction on the pixel to be corrected if the pixel to be corrected is not located in the third face frame.
  • the correction module 43 is further configured to end the correction process of the pixel to be corrected if the current correction operation is not a translation operation.
  • the electronic device 200 in the embodiment of the present application includes a memory 210, a processor 220, that is, a program stored on the memory 210 and running on the processor 220, and the processor 220 can implement the following steps when executing the program: Perform face detection on the image to determine the position of each face frame included in the image; according to the position of each face frame, determine whether each face frame is within the preset field of view range; if the first face frame is at least If a part of the area is not within the range of the preset angle of view, distortion correction is performed on the face in the first face frame to generate a corrected image.
  • the processor 220 may also implement the following steps when executing the program: perform conformal projection on the first face frame to generate a second face frame; determine that the second face frame and The first face frame satisfies the preset condition.
  • the processor 220 may also implement the following steps when executing the program: determining that the length of at least one side in the second face frame and the length of the corresponding side in the first face frame satisfy a preset condition.
  • the processor 220 may also implement the following steps when executing the program: according to the attributes of the camera module that collects the image, The preset field of view range is determined, where the attributes of the camera module include the setting position of the camera module in the terminal, and the field of view angle of the camera module.
  • the processor 220 may also implement the following steps when executing the program: determine whether the pixel to be corrected is located in the third face frame, where the third face frame is located in a preset The face frame within the field of view; if the pixel to be corrected is located in the third face frame, determine whether the current correction operation is a translation operation; if the current correction operation is a translation operation, then the pixel to be corrected Perform panning.
  • the processor 220 may also implement the following steps when executing the program: if the pixel to be corrected is not located in the third face frame. Within the face frame, the pixels to be corrected are corrected.
  • the processor may also implement the following steps when executing the program: if the current correction operation is not a translation operation, the correction processing of the pixel to be corrected is ended.
  • the computer-readable storage medium of the embodiment of the present application stores a computer program, and when the program is executed by a processor, it realizes the face distortion correction method of any of the above-mentioned embodiments.
  • the embodiment of the present application proposes a face distortion correction method for the image collected by a camera with a larger field of view in the related technology, and the perspective distortion of the face at the edge of the image is relatively serious, which affects the user experience.
  • the face distortion correction method determines the position of each face frame included in the image by detecting the face of the acquired image, and determines whether each face frame is located according to the position of each face frame Within the preset angle of view, and when at least part of the first face frame is not within the preset angle of view, perform distortion correction on the face in the first face frame to generate a corrected image .
  • Face correction improves the effect of face distortion correction and improves user experience.
  • FIG. 1 is a schematic flowchart of a method for correcting face distortion provided by an embodiment of the application.
  • the method for correcting face distortion includes the following steps:
  • Step 101 Perform face detection on the acquired image, and determine the position of each face frame included in the image.
  • the face distortion correction method of the embodiment of the present application may be executed by the face distortion correction device of the embodiment of the present application.
  • the facial distortion correction apparatus of the embodiment of the present application can be configured in any electronic device with a camera or an image processing function to perform portrait distortion correction on an image acquired by the electronic device.
  • the electronic devices of the present application may include mobile phones, tablet computers, personal digital assistants, wearable devices, etc., but are not limited thereto.
  • each face frame in the image refers to a frame corresponding to each face included in the image.
  • each pixel corresponding to the human face is located in the human face frame corresponding to the human face.
  • the position of the face frame can be represented by the pixel coordinates corresponding to the four vertices of the face frame in the image.
  • AI face detection algorithms and semantic segmentation of human images may be used to perform face detection on the acquired image to determine the position of each face frame in the image.
  • Step 102 According to the position of each face frame, determine whether each face frame is within a preset field of view range.
  • the preset field of view range refers to the area in the image that is not prone to face deformation, or the image area where the face deformation is small and can be ignored.
  • the severity of the face deformation in the image is related to the distance between the camera module and the subject, the field of view of the camera module and other attributes, it can be based on the camera module’s Properties to determine the preset range of field of view. That is, in a possible implementation form of the embodiment of the present application, before the foregoing step 102, it may further include:
  • the preset field of view range is determined according to the attributes of the camera module that collects the image, where the attributes of the camera module include the setting position of the camera module in the terminal and the field of view angle of the camera module.
  • the location of the camera module in the terminal may include front, rear, etc.
  • the field of view angle of the camera module refers to the field of view of the camera module; the larger the field of view angle of the camera module.
  • the pixels within the 40° field of view will not produce perspective distortion, and the pixels that are slightly larger than the 40° field of view
  • the resulting perspective distortion is also very small and can be ignored.
  • the perspective distortion produced by pixels within a 50° field of view is negligible.
  • the reference standard field of view angle can be determined to be a smaller value, for example, the reference standard field of view angle can be determined It is 40°; if it is determined that the setting position of the camera module in the terminal is “rear”, the reference standard field of view angle can be determined to be a larger value, for example, the reference standard field of view angle can be determined to be 50°.
  • the preset field of view range in the image can be determined according to the field of view of the camera module and the reference standard field of view, that is, the field of view in the image is less than or equal to the reference standard field of view The corner of the image area.
  • the field of view of the camera module is 70°, and the reference standard field of view is determined to be 50°, then it can be determined that the image is located in the [0°,50°] field of view according to the rules of perspective projection
  • the image area within is the preset field of view range.
  • the position of the face frame is represented by the pixel coordinates of the four vertices of the face frame in the image, it can be judged whether the coordinates of the four vertices of the face frame are all located in the preset field of view range.
  • the image area if it is, it can be determined that the face frame is within the preset angle of view range; if at least one of the four vertices in the face frame is not in the image area corresponding to the preset angle of view, it can be determined Part or all of the face frame is not within the preset angle of view.
  • Step 103 If at least a part of the area of the first face frame is not within the preset angle of view, perform distortion correction on the face in the first face frame to generate a corrected image.
  • the first face frame refers to a face frame in which part or all of the area is not within the preset field of view, that is, the face corresponding to the first face frame has serious perspective distortion.
  • the face in the first face frame may have serious perspective distortion, then Perform distortion correction on the face in the first face frame to generate a corrected image.
  • the face in the first face frame can be conformally projected (such as spherical (stereographic) projection, cylindrical fisheye (Pannini) projection, etc.) Correct the distortion of the face.
  • the face in the first face frame can be sparsed into image grids, and then for each face grid corresponding to the face, the conformal projection grid corresponding to each face grid can be calculated, and then the conformal projection grid corresponding to each face grid can be calculated according to each face grid.
  • the conformal projection grid corresponding to the personal face grid corrects the face so that the corrected target face grid is consistent with the conformal projection grid corresponding to the face grid.
  • the error estimate of the face distortion correction can be determined by formula (1), and then according to the relationship between the error estimate and the error threshold, it is determined whether the corrected target face grid is consistent with the conformal projection grid, That is, whether the effect of face distortion correction is ideal.
  • E u human face distortion correction error estimation facial weight of the i W i grid corresponding to a human face weight
  • V i is the i facial mesh face after the correction corresponding to the target face meshes
  • Sk is the similarity transformation matrix
  • u i is the conformal projection grid corresponding to the i-th face grid corresponding to the face
  • t k is the translation vector
  • k is the number of face grids corresponding to the face
  • i is the face corresponding The serial number of the face grid.
  • E u estimation error than or equal to a first threshold value if a human face distortion correction error, it may be determined corrected target face conformal projection coincides with a grid mesh, i.e. on the first face The face in the frame has a better effect of face distortion correction, so that the process of face distortion correction on the face in the first face frame can be ended; if the error estimate of the face distortion correction E u is greater than the first error threshold , It can be determined that the corrected target face grid is inconsistent with the conformal projection grid, that is, the effect of face distortion correction on the face in the first face frame is poor, so that the first face frame can continue to be corrected. human face further facial distortion correction, distortion correction face until error estimate E u is less than or equal to a second error threshold.
  • the specific value of the first error threshold may be preset according to actual needs and specific application scenarios, which is not limited in the embodiment of the present application.
  • the face distortion correction method determines the position of each face frame included in the image by detecting the face of the acquired image, and determines whether each face frame is located according to the position of each face frame Within the preset angle of view, and when at least part of the first face frame is not within the preset angle of view, perform distortion correction on the face in the first face frame to generate a corrected image .
  • Face correction improves the effect of face distortion correction and improves user experience.
  • the face that is not within the preset field of view may not be deformed, so that it can be further based on the degree of deformation of the face in the first face frame, from the first person
  • the face frame filters the face frames that need to be corrected for face distortion, so as to further improve the effect of face distortion correction.
  • FIG. 2 is a schematic flowchart of another method for correcting face distortion according to an embodiment of the application.
  • the method for correcting face distortion includes the following steps:
  • Step 201 Perform face detection on the acquired image, and determine the position of each face frame included in the image.
  • step 202 according to the position of each face frame, it is determined whether each face frame is within a preset field of view range.
  • Step 203 If at least a part of the area of the first face frame is not within the preset angle of view, perform conformal projection on the first face frame to generate a second face frame.
  • the face in the first face frame has a high possibility of perspective distortion. But it does not mean that all faces in the first face frame will have obvious perspective distortion, especially the first face frame whose part of the area is not within the preset field of view range, it is very likely that no perspective distortion will occur. Or the degree of perspective distortion is so small that it is difficult for the human eye to detect.
  • the first face frame can be further filtered according to the degree of perspective deformation of the face in the first face frame, and only the face in the first face frame whose degree of perspective deformation meets the conditions can be subjected to distortion correction, so that further Protect the undistorted face, and further improve the effect of face distortion correction.
  • the conformal projection of the face in the first face frame can be used as a reference to determine the first person The degree of perspective distortion of the face in the face frame. Therefore, after each first face frame included in the image is determined, the face in each first face frame may be conformally projected first to generate a second face frame corresponding to each first face frame.
  • Step 204 Determine that a preset condition is satisfied between the second face frame and the first face frame.
  • the preset conditions that need to be met by the first face frame that needs to be corrected for face distortion may be determined, so that the second face frame corresponding to the first face frame can be determined according to the second face frame corresponding to the first face frame.
  • the severity of the perspective deformation of the face in the first face frame may be determined according to the difference between the size of the first face frame and the size of the second face frame. That is, in a possible implementation form of the embodiment of the present application, the foregoing step 204 may include:
  • the preset condition may be that the absolute value of the difference between the length of at least one side in the second face frame and the length of the corresponding side in the first face frame is greater than the difference threshold; or, at least The ratio of the length of one side to the length of the corresponding side in the first face frame is within a preset range, etc., which is not limited in the embodiment of the present application.
  • the length of at least one side in the second face frame and the length of the corresponding side in the first face frame meet the preset condition, it can be determined that the face in the first face frame is generated The degree of perspective distortion or perspective distortion is more obvious, so that the distortion correction of the face in the first face frame can be performed; if the length of each side in the second face frame is the same as that of the corresponding side in the first face frame If the length does not meet the preset condition, it can be determined that the face in the first face frame has not undergone perspective distortion, or the degree of perspective distortion is not obvious, so that there is no need to perform distortion correction on the face in the first face frame.
  • the preset condition is "the length difference is greater than 10 pixels", the length of one side in the second face frame is 100 pixels, and the length of the corresponding side in the first face frame is 120 pixels, so that the second face can be determined
  • the length of a side in the face frame and the length of the corresponding side in the first face frame meet the preset condition, so that the face in the first face frame can be corrected for distortion;
  • the preset condition is "length The ratio is less than 0.9 or greater than 1.1”
  • the length of one side in the second face frame is 115 pixels, and the length of the corresponding side in the first face frame is 100 pixels, so that the length of the side in the second face frame can be determined
  • the ratio of the length of the corresponding side to the first face frame is 1.15, which satisfies the preset condition, that is, the distortion correction can be performed on the face in the first face frame.
  • Step 205 Perform distortion correction on the face in the first face frame to generate a corrected image.
  • the face distortion correction method determines the position of each face frame included in the image by detecting the face of the acquired image, and determines whether each face frame is located according to the position of each face frame Within the preset angle of view, then when at least part of the first face frame is not within the preset angle of view, the first face frame is conformally projected to generate a second face frame, and then When it is determined that the preset condition is satisfied between the second face frame and the first face frame, distortion correction is performed on the face in the first face frame to generate a corrected image. Therefore, by further filtering the first face frame according to the perspective deformation degree of the face in the first face frame that is completely within the preset field of view The face without perspective distortion is protected, which further improves the effect of face distortion correction and improves the user experience.
  • FIG. 3 is a schematic flowchart of yet another method for correcting face distortion according to an embodiment of the application.
  • the face distortion correction method includes the following steps:
  • Step 301 Perform face detection on the acquired image, and determine the position of each face frame included in the image.
  • Step 302 according to the position of each face frame, determine whether each face frame is within a preset field of view range.
  • Step 303 If at least a part of the area of the first face frame is not within the preset angle of view range, perform distortion correction on the face in the first face frame.
  • Step 304 Determine whether the pixel to be corrected is located in the third face frame, where the third face frame is the face frame located within the preset field of view, if yes, go to step 305; otherwise, go to step 308.
  • the corrected image when performing distortion correction on the face in the first face frame, in order to ensure a smooth transition between the face distortion correction area and the non-correction area, the corrected image can be more natural, and the first face can be corrected.
  • the pixels around the frame are also corrected.
  • the pixels around the first face frame may fall into the third face frame within the preset field of view. Therefore, in a possible implementation of the present application, for each pixel to be corrected, It is possible to first determine whether the pixel to be corrected is located in the third face frame, so as to protect the pixel to be corrected located in the third face frame to improve the quality of the corrected image.
  • step 305 it is determined whether the current correction operation is a translation operation, if so, step 306 is executed, otherwise, step 307 is executed.
  • the correction operation may be a translation operation, a rotation operation, a twist operation, an interpolation operation, etc., which are not limited in the embodiment of the present application.
  • a translation operation on the pixels in the third face frame may be allowed. Therefore, when it is determined that the pixel to be corrected is located in the third face frame, it can be further determined whether the current correction operation is a translation operation, so as to determine whether the current correction operation can be used to correct the pixel to be corrected.
  • Step 306 Shift the pixel to be corrected.
  • the current correction operation is a translation operation
  • it can be based on the parameters (direction of translation, length of translation, etc.) included in the current correction operation. )
  • the pixel to be corrected is translated to complete the correction of the pixel to be corrected.
  • Step 307 End the correction processing of the pixel to be corrected.
  • the correction processing of the pixel to be corrected can be ended, that is, the pixel to be corrected is not Perform correction to protect the face in the third face frame.
  • step 308 the pixel to be corrected is corrected.
  • the pixel to be corrected can be corrected directly according to the current to-be-corrected operation.
  • the error estimation of the face distortion correction can be determined according to formula (2).
  • E face distortion correction error estimate for the introduction of the protected item of human face, right face grid person i W i corresponding to a human face weight, V i for the i-th face corresponding to facial mesh is corrected
  • Sk is the similarity transformation matrix
  • u i is the conformal projection grid corresponding to the i-th face grid corresponding to the face
  • t k is the translation vector
  • p i is the i-th face corresponding to the face Grid
  • k is the number of face grids corresponding to the face
  • i is the serial number of the face grid corresponding to the face.
  • the effect of the face distortion correction on the face in the first face frame can be determined Better, and better protection for faces that do not require face distortion correction, so that the process of face distortion correction on the face in the first face frame can be ended; if the face distortion of the face protection item is introduced
  • the corrected error estimate E is greater than the second error threshold, it can be determined that the effect of face distortion correction on the face in the first face frame is poor, or the protection effect of the face that does not require face distortion correction is better. Therefore, it is possible to continue to perform further face distortion correction on the face in the first face frame until the error estimate E of the face distortion correction introduced into the face protection item is less than or equal to the second error threshold.
  • the specific value of the second error threshold can be preset according to actual needs and specific application scenarios, which is not limited in the embodiment of the present application.
  • the face distortion correction method determines the position of each face frame included in the image by performing face detection on the acquired image, and determines that at least part of the area of the first face frame is not in the preset
  • the distortion correction is performed on the face in the first face frame, and when the distortion correction is performed, it is determined whether the pixel to be corrected is located in the third face frame, and then only the face in the third face Perform a translation operation on the pixels to be corrected in the frame, and perform corresponding correction operations on the pixels to be corrected that are not located in the third face frame. Therefore, by only allowing the translation operation of the pixels to be corrected in the third face frame, the protection effect of the face within the preset angle of view is further improved, and the image quality after correction is improved. Improved user experience.
  • this application also proposes a face distortion correction device.
  • FIG. 4 is a schematic structural diagram of a face distortion correction device provided by an embodiment of the application.
  • the facial distortion correction device 40 includes:
  • the first determining module 41 is configured to perform face detection on the acquired image and determine the position of each face frame included in the image;
  • the judging module 42 is used for judging whether each face frame is within the preset angle of view range according to the position of each face frame.
  • the correction module 43 is configured to perform distortion correction on the face in the first face frame to generate a corrected image if at least a part of the area of the first face frame is not within the preset angle of view.
  • the facial distortion correction apparatus provided in the embodiments of the present application can be configured in any electronic device to execute the aforementioned facial distortion correction method.
  • the face distortion correction device determines the position of each face frame included in the image by detecting the face of the acquired image, and determines whether each face frame is located according to the position of each face frame Within the preset angle of view, and when at least part of the first face frame is not within the preset angle of view, perform distortion correction on the face in the first face frame to generate a corrected image .
  • Face correction improves the effect of face distortion correction and improves user experience.
  • the aforementioned facial distortion correction device 40 further includes:
  • the projection module is used to perform conformal projection on the first face frame to generate the second face frame;
  • the second determining module is used to determine that the second face frame and the first face frame satisfy a preset condition.
  • the above-mentioned second determining module is specifically used for:
  • the aforementioned facial distortion correction device 40 further includes:
  • the third determining module is used to determine the preset field of view range according to the attributes of the camera module that collects the image, where the attributes of the camera module include the location of the camera module in the terminal and the field of view of the camera module angle.
  • the above-mentioned correction module 43 is specifically used for:
  • the current correction operation is a translation operation
  • the current correction operation is a translation operation
  • the pixel to be corrected is translated.
  • correction module 43 is also used for:
  • the pixel to be corrected is corrected.
  • correction module 43 is also used for:
  • the correction processing of the pixel to be corrected is ended.
  • the face distortion correction device determines the position of each face frame included in the image by detecting the face of the acquired image, and determines that at least part of the area of the first face frame is not in the preset
  • the distortion correction is performed on the face in the first face frame, and when the distortion correction is performed, it is determined whether the pixel to be corrected is located in the third face frame, and then only the face in the third face Perform a translation operation on the pixels to be corrected in the frame, and perform corresponding correction operations on the pixels to be corrected that are not located in the third face frame. Therefore, by only allowing the translation operation of the pixels to be corrected in the third face frame, the protection effect of the face within the preset angle of view is further improved, and the image quality after correction is improved. Improved user experience.
  • this application also proposes an electronic device.
  • Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
  • the above-mentioned electronic device 200 includes:
  • the memory 210 and the processor 220 are connected to the bus 230 of different components (including the memory 210 and the processor 220).
  • the memory 210 stores a computer program.
  • the processor 220 executes the program, the face distortion described in the embodiment of the present application is realized Correction method.
  • the bus 230 represents one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any bus structure among multiple bus structures.
  • these architectures include, but are not limited to, industry standard architecture (ISA) bus, microchannel architecture (MAC) bus, enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and peripheral component interconnection ( PCI) bus.
  • ISA industry standard architecture
  • MAC microchannel architecture
  • VESA Video Electronics Standards Association
  • PCI peripheral component interconnection
  • the electronic device 200 typically includes a variety of electronic device-readable media. These media may be any available media that can be accessed by the electronic device 200, including volatile and non-volatile media, removable and non-removable media.
  • the memory 210 may also include a computer system readable medium in the form of volatile memory, such as random access memory (RAM) 240 and/or cache memory 250.
  • the electronic device 200 may further include other removable/non-removable, volatile/nonvolatile computer system storage media.
  • the storage system 260 may be used to read and write non-removable, non-volatile magnetic media (not shown in FIG. 5, usually referred to as a "hard drive").
  • a disk drive for reading and writing to a removable non-volatile disk such as a "floppy disk”
  • a removable non-volatile optical disk such as CD-ROM, DVD-ROM
  • other optical media read and write optical disc drives.
  • each drive may be connected to the bus 230 through one or more data medium interfaces.
  • the memory 210 may include at least one program product, and the program product has a set (for example, at least one) program modules, and these program modules are configured to perform the functions of the embodiments of the present application.
  • a program/utility tool 280 having a set of (at least one) program module 270 may be stored in, for example, the memory 210.
  • Such program module 270 includes, but is not limited to, an operating system, one or more application programs, and other programs Modules and program data, each of these examples or some combination may include the realization of a network environment.
  • the program module 270 generally executes the functions and/or methods in the embodiments described in this application.
  • the electronic device 200 may also communicate with one or more external devices 290 (such as a keyboard, pointing device, display 291, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 200, and/or communicate with Any device (such as a network card, modem, etc.) that enables the electronic device 200 to communicate with one or more other computing devices. This communication can be performed through an input/output (I/O) interface 292.
  • the electronic device 200 may also communicate with one or more networks (for example, a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) through the network adapter 293.
  • networks for example, a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet
  • the network adapter 293 communicates with other modules of the electronic device 200 through the bus 230.
  • other hardware and/or software modules can be used in conjunction with the electronic device 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives And data backup storage system, etc.
  • the processor 220 executes various functional applications and data processing by running programs stored in the memory 210.
  • the electronic device provided by the embodiment of the present application can execute the face distortion correction method as described above, by performing face detection on the acquired image, the position of each face frame included in the image is determined, and the position of each face frame is determined according to each face frame.
  • Face correction improves the effect of face distortion correction and improves user experience.
  • this application also proposes a computer-readable storage medium.
  • the computer-readable storage medium has a computer program stored thereon, and when the program is executed by a processor, the method for correcting face distortion described in the embodiment of the present application is realized.
  • another embodiment of the present application provides a computer program, which when executed by a processor, implements the facial distortion correction method described in the embodiments of the present application.
  • this embodiment may adopt any combination of one or more computer-readable media.
  • the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination of any of the above.
  • computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), Erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or flash memory Erasable programmable read-only memory
  • CD-ROM compact disk read-only memory
  • the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device.
  • the computer-readable signal medium may include a data signal propagated in baseband or as a part of a carrier wave, and computer-readable program code is carried therein. This propagated data signal can take many forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium.
  • the computer-readable medium may send, propagate, or transmit the program for use by or in combination with the instruction execution system, apparatus, or device .
  • the program code contained on the computer-readable medium can be transmitted by any suitable medium, including, but not limited to, wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
  • the computer program code used to perform the operations of this application can be written in one or more programming languages or a combination thereof.
  • the programming languages include object-oriented programming languages—such as Java, Smalltalk, C++, and also conventional Procedural programming language-such as "C" language or similar programming language.
  • the program code can be executed entirely on the user electronic device, partly executed on the user electronic device, executed as an independent software package, partly executed on the user electronic device and partly executed on the remote electronic device, or completely executed on the remote electronic device or Execute on the server.
  • the remote electronic equipment can be connected to the user electronic equipment through any kind of network-including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external electronic device (for example, using Internet services). Provider to connect via the Internet).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Studio Devices (AREA)
  • Image Processing (AREA)
  • Geometry (AREA)

Abstract

一种人脸畸变校正方法、人脸畸变校正装置（40）、电子设备（200）及存储介质，人脸畸变校正方法包括对获取的图像进行人脸检测，确定所述图像中包括的各人脸框的位置；根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内；若第一人脸框的至少部分区域未在所述预设视场角范围内，则对所述第一人脸框中的人脸进行畸变校正，以生成校正后的图像。

Description

人脸畸变校正方法、装置、电子设备及存储介质
优先权信息
本申请请求2019年12月9日向中国国家知识产权局提交的、专利申请号为201911252854.8的专利申请的优先权和权益，并且通过参照将其全文并入此处。
技术领域
本申请涉及图像处理技术领域，尤其涉及一种人脸畸变校正方法、装置、电子设备及存储介质。
背景技术
用户在使用带有摄像头的电子设备进行拍照时，靠近图像边缘的人脸会产生变形，这是由于摄像头成像过程中的透视投影导致的透视变形。透视变形指的是一个物体及其周围区域与标准镜头中看到的相比完全不同，由于远近特征的相对比例变化，发生了弯曲或变形。
发明内容
本申请提出的人脸畸变校正方法、装置、电子设备及存储介质，用于解决相关技术中，视场角较大的摄像头采集的图像，图像边缘人脸透视变形较为严重，影响了用户体验的问题。
本申请一方面实施例提出的人脸畸变校正方法，包括：对获取的图像进行人脸检测，确定所述图像中包括的各人脸框的位置；根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内；若第一人脸框的至少部分区域未在所述预设视场角范围内，则对所述第一人脸框中的人脸进行畸变校正，以生成校正后的图像。
本申请另一方面实施例提出的人脸畸变校正装置，包括：第一确定模块，用于对获取的图像进行人脸检测，确定所述图像中包括的各人脸框的位置；判断模块，用于根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内；校正模块，用于若第一人脸框的至少部分区域未在所述预设视场角范围内，则对所述第一人脸框中的人脸进行畸变校正，以生成校正后的图像。
本申请再一方面实施例提出的电子设备，其包括：存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其特征在于，所述处理器执行所述程序时实现人脸畸变校正方法。所述人脸畸变校正方法包括：对获取的图像进行人脸检测，确定所述图像中包括的各人脸框的位置；根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内；若第一人脸框的至少部分区域未在所述预设视场角范围内，则对所述第一人脸框中的人脸进行畸变校正，以生成校正后的图像。
本申请再一方面实施例提出的计算机可读存储介质，其上存储有计算机程序，其特征在于，所述程序被处理器执行时实现人脸畸变校正方法。所述人脸畸变校正方法包括：对获取的图像进行人脸检测，确定所述图像中包括的各人脸框的位置；根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内；若第一人脸框的至少部分区域未在所述预设视场角范围内，则对所述第一人脸框中的人脸进行畸变校正，以生成校正后的图像。
本申请又一方面实施例提出的计算机程序，该程序被处理器执行时，以实现人脸畸变校正方法。所述人脸畸变校正方法包括：对获取的图像进行人脸检测，确定所述图像中包括的各人脸框的位置；根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内；若第一人脸框的至少部分区域未在所述预设视场角范围内，则对所述第一人脸框中的人脸进行畸变校正，以生成校正后的图像。
本申请附加的方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本申请的实践了解到。
附图说明
本申请上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解，其中：
图1为本申请实施例所提供的一种人脸畸变校正方法的流程示意图；
图2为本申请实施例所提供的另一种人脸畸变校正方法的流程示意图；
图3为本申请实施例所提供的再一种人脸畸变校正方法的流程示意图；
图4为本申请实施例提供的一种人脸畸变校正装置的结构示意图；
图5为本申请实施例提供的电子设备的结构示意图。
具体实施方式
下面详细描述本申请的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的要素。下面通过参考附图描述的实施例是示例性的，旨在用于解释本申请，而不能理解为对本申请的限制。
请参阅图1，本申请实施方式的人脸畸变校正方法，包括：
对获取的图像进行人脸检测，确定图像中包括的各人脸框的位置；
根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内；
若第一人脸框的至少部分区域未在预设视场角范围内，则对第一人脸框中的人脸进行畸变校正，以生成校正后的图像。
请参阅图2，在某些实施方式中，在对第一人脸框中的人脸进行畸变校正前，人脸畸变校正方法还包括：
对第一人脸框进行保形投影，以生成第二人脸框；
确定第二人脸框与第一人脸框间满足预设条件。
在某些实施方式中，确定第二人脸框与第一人脸框间满足预设条件，包括：
确定第二人脸框中至少一条边的长度与第一人脸框中对应边的长度满足预设条件。
在某些实施方式中，判断每个人脸框是否在预设视场角范围内之前，还包括：
根据采集图像的摄像模组的属性，确定预设的视场角范围，其中，摄像模组的属性包括摄像模组在终端中的设置位置、摄像模组的视场角。
请参阅图3，在某些实施方式中，对第一人脸框中的人脸进行畸变校正，包括：
判断待校正的像素点是否位于第三人脸框内，其中，第三人脸框为位于预设视场角范围内的人脸框；
若待校正的像素点位于第三人脸框内，则判断当前的校正操作是否为平移操作；
若当前的校正操作是平移操作，则对待校正的像素点进行平移。
请参阅图3，在某些实施方式中，判断待校正的像素点是否位于第三人脸框内之后，还包括：
若待校正的像素点未位于第三人脸框内，则对待校正的像素点进行校正。
请参阅图3，在某些实施方式中，判断当前的校正操作是否为平移操作之后，还包括：
若当前的校正操作非平移操作，则结束对待校正的像素点的校正处理。
请参阅图4，本申请实施方式的人脸畸变校正装置40包括：
第一确定模块41，用于对获取的图像进行人脸检测，确定图像中包括的各人脸框的位置；
判断模块42，用于根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内。
校正模块43，用于若第一人脸框的至少部分区域未在预设视场角范围内，则对第一人脸框中的人脸进行畸变校正，以生成校正后的图像。
请参阅图4，在某些实施方式中，人脸畸变校正装置40，还包括：
投影模块，用于对第一人脸框进行保形投影，以生成第二人脸框；
第二确定模块，用于确定第二人脸框与第一人脸框间满足预设条件。
在某些实施方式中，第二确定模块还用于确定第二人脸框中至少一条边的长度与第一人脸框中对应边的长度满足预设条件。
请参阅图4，在某些实施方式中，人脸畸变校正装置40，还包括：
第三确定模块，用于根据采集图像的摄像模组的属性，确定预设的视场角范围，其中，摄像模组的属性包括摄像模组在终端中的设置位置、摄像模组的视场角。
请参阅图4，在某些实施方式中，校正模块43还用于若待校正的像素点未位于第三人脸框内，则对待校正的像素点进行校正。
在某些实施方式中，校正模块43还用于若当前的校正操作非平移操作，则结束对待校正的像素点的校正处理。
请参阅图5，本申请实施方式电子设备200包括存储器210、处理器220、即存储在存储器210上并可在处理器220上运行的程序，处理器220执行程序时可实现如下步骤：对获取的图像进行人脸检测，确定图像中包括的各人脸框的位置；根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内；若第一人脸框的至少部分区域未在预设视场角范围内，则对第一人脸框中的人脸进行畸变校正，以生成校正后的图像。
请参阅图5，在某些实施方式中，处理器220执行程序时还可实现如下步骤：对第一人脸框进行保形投影，以生成第二人脸框；确定第二人脸框与第一人脸框间满足预设条件。
请参阅图5，在某些实施方式中，处理器220执行程序时还可实现如下步骤：确定第二人脸框中至少一条边的长度与第一人脸框中对应边的长度满足预设条件。
请参阅图5，在某些实施方式中，判断每个人脸框是否在预设视场角范围内之前，处理器220执行程序时还可实现如下步骤：根据采集图像的摄像模组的属性，确定预设的视场角范围，其中，摄像模组的属性包括摄像模组在终端中的设置位置、摄像模组的视场角。
请参阅图5，在某些实施方式中，处理器220执行程序时还可实现如下步骤：判断待校正的像素点是否位于第三人脸框内，其中，第三人脸框为位于预设视场角范围内的人脸框；若待校正的像素点位于第三人脸框内，则判断当前的校正操作是否为平移操作；若当前的校正操作是平移操作，则对待校正的像素点进行平移。
请参阅图5，在某些实施方式中，判断待校正的像素点是否位于第三人脸框内之后，处理器220执行程序时还可实现如下步骤：若待校正的像素点未位于第三人脸框内，则对待校正的像素点进行校正。
请参阅图5，在某些实施方式中，处理器执行程序时还可实现如下步骤：若当前的校正操作非平移操作，则结束对待校正的像素点的校正处理。
本申请实施方式的计算机可读存储介质存储有计算机程序，程序被处理器执行时实现上述任一实 施方式的人脸畸变校正方法。
本申请实施例针对相关技术中，视场角较大的摄像头采集的图像，图像边缘人脸透视变形较为严重，影响了用户体验的问题，提出一种人脸畸变校正方法。
本申请实施例提供的人脸畸变校正方法，通过对获取的图像进行人脸检测，确定图像中包括的各人脸框的位置，并根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内，进而在第一人脸框的至少部分区域未在预设视场角范围内时，对第一人脸框中的人脸进行畸变校正，以生成校正后的图像。由此，通过对未处于预设视场角范围内的人脸进行校正，对处于预设视场角范围内的人脸进行保护，从而实现了保护未畸变人脸质量的同时，对畸变人脸进行校正，提高了人脸畸变校正的效果，改善了用户体验。
下面参考附图对本申请提供的人脸畸变校正方法、装置、电子设备、存储介质及计算机程序进行详细描述。
图1为本申请实施例所提供的一种人脸畸变校正方法的流程示意图。
如图1所示，该人脸畸变校正方法，包括以下步骤：
步骤101，对获取的图像进行人脸检测，确定图像中包括的各人脸框的位置。
需要说的是，本申请实施例的人脸畸变校正方法，可以由本申请实施例的人脸畸变校正装置执行。本申请实施例的人脸畸变校正装置可以配置在任意具有摄像头或者具有图像处理功能的电子设备中，以对电子设备获取的图像进行人像畸变校正。其中，本申请的电子设备可以包括手机、平板电脑、个人数字助理、穿戴式设备等，但不仅限于此。
其中，图像中的各人脸框，是指图像中包括的各人脸对应的框。其中，人脸对应的每个像素均位于该人脸对应的人脸框内。
需要说明的是，人脸框的位置，可以采用人脸框的四个顶点在图像中对应的像素坐标进行表示。
在本申请实施例中，可以采用人工智能(Artificial Intelligence，简称AI)人脸检测算法，以及人像语义分割，对获取的图像进行人脸检测，以确定图像中各人脸框的位置。
步骤102，根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内。
其中，预设视场角范围，是指图像中不易产生人脸变形的区域，或者人脸变形较小可以忽略不计的图像区域。
作为一种可能的实现方式，由于图像中的人脸产生变形的严重程度与拍摄时摄像模组与被摄物体的距离、摄像模组的视场角等属性有关，从而可以根据摄像模组的属性，确定预设的视场角范围。即在本申请实施例一种可能的实现形式中，上述步骤102之前，还可以包括：
根据采集图像的摄像模组的属性，确定预设的视场角范围，其中，摄像模组的属性包括摄像模组在终端中的设置位置、摄像模组的视场角。
其中，摄像模组在终端中的设置位置，可以包括前置、后置等。摄像模组的视场角，是指摄像模组的视野范围；摄像模组的视场角越大。
需要说明的是，摄像模组与被摄物体的距离越小，摄像模组的视场角越大，则图像边缘的人脸透视变形越严重，从而可以根据摄像模组在终端中的设置位置确定参考标准视场角，进而根据参考标准视场角(和摄像模组的视场角，确定预设的视场角范围。
可选的，由于标准镜头视场角为40°，因此对于采集的图像来说，位于40°视场角范围内的像素不会产生透视变形，以及略大于40°视场角范围内的像素产生的透视变形也很小，可以忽略不计。比如，50°视场角范围内的像素产生的透视变形是可以忽略不计的。从而，由于与采用后置摄像模组相 比，在采用前置摄像模组进行拍摄时，摄像模组与被摄物体间的距离通常较小，导致图像边缘的透视变形较严重(即产生透视变形的图像范围较大)，因此若确定摄像模组在终端中的设置位置为“前置”，则可以将参考标准视场角确定为较小的值，比如可以将参考标准视场角确定为40°；若确定摄像模组在终端中的设置位置为“后置”，则可以将参考标准视场角确定为较大的值，比如可以将参考标准视场角确定为50°。
在确定出参考标准视场角之后，可以根据摄像模组的视场角与参考标准视场角，确定图像中的预设的视场角范围，即图像中视场角小于或等于参考标准视场角的图像区域。
举例来说，若摄像模组的视场角为70°，确定的参考标准视场角为50°，则可以根据透视投影的规则，确定图像中位于[0°,50°]视场角范围内的图像区域，即预设的视场角范围。
在本申请实施例中，确定出每个人脸框的位置及预设视场角范围之后，即可以确定各人脸框的位置是否在预设视场角范围内。
可选的，若人脸框的位置是采用人脸框四个顶点在图像中的像素坐标表示的，则可以判断人脸框的四个顶点的坐标是否都位于预设视场角范围对应的图像区域，若是，则可以确定该人脸框在预设视场角范围；若人脸框中的四个顶点中存在至少一个顶点不在预设视场角范围对应的图像区域，则可确定该人脸框的部分区域或全部区域不在预设视场角范围内。
步骤103，若第一人脸框的至少部分区域未在预设视场角范围内，则对第一人脸框中的人脸进行畸变校正，以生成校正后的图像。
其中，第一人脸框，是指部分区域或全部区域未在预设视场角范围内的人脸框，即第一人脸框对应的人脸产生的透视变形较严重。
在本申请实施例中，若第一人脸框的部分区域或全部区域均未在预设视场角范围内，即第一人脸框中的人脸可能产生较严重的透视变形，则可以对第一人脸框中的人脸进行畸变校正，以生成校正后的图像。
作为一种可能的实现方式，可以通过对第一人脸框中的人脸进行保形投影(如球形(stereographic)投影、圆柱鱼眼(Pannini)投影等)，以对第一人脸框中的人脸进行畸变校正。
具体的，可以首先将第一人脸框中的人脸稀疏为图像网格，之后对人脸对应的每个人脸网格，计算每个人脸网格对应的保形投影网格，进而根据每个人脸网格对应的保形投影网格对人脸进行校正，以使得校正后的目标人脸网格与该人脸网格对应的保形投影网格一致。在本申请实施例中，可以通过公式(1)确定人脸畸变校正的误差估计，进而根据误差估计与误差阈值的关系，确定校正后的目标人脸网格是否与保形投影网格一致，即人脸畸变校正的效果是否理想。
Figure PCTCN2020127551-appb-000001
其中，E u为人脸畸变校正的误差估计，w i为人脸对应的第i个人脸网格的权重，v i为对人脸对应的第i个人脸网格进行校正后的目标人脸网格，S k为相似变换矩阵，u i为人脸对应的第i个人脸网格对应的保形投影网格，t k为平移向量，k为人脸对应的人脸网格的数量，i为人脸对应的人脸网格的序号。
在本申请实施例中，若人脸畸变校正的误差估计E u小于或等于第一误差阈值，则可以确定校正后的目标人脸网格与保形投影网格一致，即对第一人脸框中的人脸进行人脸畸变校正的效果较好，从而可以结束对第一人脸框中的人脸进行人脸畸变校正过程；若人脸畸变校正的误差估计E u大于第一误差阈值，则可以确定校正后的目标人脸网格与保形投影网格不一致，即对第一人脸框中的人脸进行人脸畸变校正的效果较差，从而可以继续对第一人脸框中的人脸进行进一步人脸畸变校正，直至人脸畸变 校正的误差估计E u小于或等于第二误差阈值。
需要说的的是，实际使用时，第一误差阈值的具体取值可以根据实际需要及具体的应用场景预设，本申请实施例对此不做限定。
相关技术中，对于视场角较大的摄像头(如广角摄像头、手机自拍摄像头等)，图像边缘人脸透视变形更为严重，影响了用户体验。
本申请实施例提供的人脸畸变校正方法，通过对获取的图像进行人脸检测，确定图像中包括的各人脸框的位置，并根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内，进而在第一人脸框的至少部分区域未在预设视场角范围内时，对第一人脸框中的人脸进行畸变校正，以生成校正后的图像。由此，通过对未处于预设视场角范围内的人脸进行校正，对处于预设视场角范围内的人脸进行保护，从而实现了保护未畸变人脸质量的同时，对畸变人脸进行校正，提高了人脸畸变校正的效果，改善了用户体验。
在本申请一种可能的实现形式中，未处于预设视场角范围内的人脸也可能未产生变形，从而可以进一步根据第一人脸框中的人脸的变形程度，从第一人脸框中筛选需要进行人脸畸变校正的人脸框，以进一步提高人脸畸变校正的效果。
下面结合图2，对本申请实施例提供的人脸畸变校正方法进行进一步说明。
图2为本申请实施例所提供的另一种人脸畸变校正方法的流程示意图。
如图2所示，该人脸畸变校正方法，包括以下步骤：
步骤201，对获取的图像进行人脸检测，确定图像中包括的各人脸框的位置。
步骤202，根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内。
上述步骤201-202的具体实现过程及原理，可以参照上述实施例的详细描述，此处不再赘述。
步骤203，若第一人脸框的至少部分区域未在预设视场角范围内，则对第一人脸框进行保形投影，以生成第二人脸框。
在本申请实施例中，若第一人脸框的部分区域或全部区域为在预设视场角范围内，则可以说明第一人脸框中的人脸产生透视变形的可能性很大，但是并不代表所有第一人脸框中的人脸都会产生明显的透视变形，尤其是部分区域未在预设的视场角范围内的第一人脸框，很有可能没有产生透视变形，或者透视变形程度很小，人眼不易察觉。从而可以根据第一人脸框中的人脸的透视变形程度对第一人脸框进行进一步筛选，仅对透视变形程度符合条件的第一人脸框中的人脸进行畸变校正，从而可以进一步保护未产生变形的人脸，进一步提高人脸畸变校正的效果。
作为一种可能的实现方式，由于保形投影对图像中人脸的透视变形有很好的校正效果，从而可以将第一人脸框中的人脸的保形投影作为参考，确定第一人脸框中的人脸的透视变形严重程度。从而可以在确定出图像中包括的各第一人脸框之后，首先对各第一人脸框中的人脸进行保形投影，以生成各第一人脸框对应的第二人脸框。
步骤204，确定第二人脸框与第一人脸框间满足预设条件。
在本申请实施例中，可以需要进行人脸畸变校正的第一人脸框需要满足的预设条件，从而可以根据第一人脸框对应的第二人脸框，确定出与其对应的第二人脸框之间满足预设条件的第一人脸框，从而完成对第一人脸框的筛选过程。
可选的，可以根据第一人脸框尺寸与第二人脸框的尺寸之间的差异，确定第一人脸框中的人脸的透视变形的严重程度。即在本申请实施例一种可能的实现形式中，上述步骤204，可以包括：
确定第二人脸框中至少一条边的长度与第一人脸框中对应边的长度满足所述预设条件。
其中，预设条件，可以是第二人脸框中至少一条边的长度与第一人脸框中对应边的长度的差值的绝对值大于差值阈值；或者，第二人脸框中至少一条边的长度与第一人脸框中对应边的长度的比值处于预设范围，等等，本申请实施例对此不做限定。
作为一种可能的实现方式，若第二人脸框中存在至少一条边的长度与第一人脸框中对应边的长度满足预设条件，则可以确定第一人脸框中的人脸产生了透视变形或透视变形的程度较为明显，从而可以对第一人脸框中的人脸进行畸变校正；若第二人脸框中的每条边的长度与第一人脸框中对应边的长度均不满足预设条件，则可以确定第一人脸框中的人脸未发生透视变形，或者透视变形的程度不明显，从而无需对第一人脸框中的人脸进行畸变校正。
举例来说，预设条件为“长度差值大于10像素”，第二人脸框中一条边的长度为100像素，第一人脸框中对应边的长度为120像素，从而可以确定第二人脸框中存在一条边的长度与第一人脸框中对应边的长度满足预设条件，从而可以对第一人脸框中的人脸进行畸变校正；又如，预设条件为“长度比值小于0.9或者大于1.1”，第二人脸框中一条边的长度为115像素，第一人脸框中对应边的长度为100像素，从而可以确定第二人脸框中该条边的长度与第一人脸框中对应边的长度比值为1.15，满足预设条件，即可以对第一人脸框中的人脸进行畸变校正。
步骤205，对第一人脸框中的人脸进行畸变校正，以生成校正后的图像。
上述步骤205的具体实现过程及原理，可以参照上述实施例的详细描述，此处不再赘述。
本申请实施例提供的人脸畸变校正方法，通过对获取的图像进行人脸检测，确定图像中包括的各人脸框的位置，并根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内，之后在第一人脸框的至少部分区域未在预设视场角范围内时，对第一人脸框进行保形投影，以生成第二人脸框，进而在确定第二人脸框与第一人脸框间满足预设条件时，对第一人脸框中的人脸进行畸变校正，以生成校正后的图像。由此，通过根据为完全处于预设视场角范围内第一人脸框中人脸的透视变形程度，对第一人脸框进行进一步筛选，从而可以对未处于预设视场范围内且未产生透视变形的人脸进行保护，进一步提高了人脸畸变校正的效果，改善了用户体验。
在本申请一种可能的实现形式中，在对未完全处于预设视场角范围内的人脸进行畸变校正的同时，还可以对完全处于预设视场角内的人脸进行保护处理，以提高人脸畸变校正后图像的整体质量。
下面结合图3，对本申请实施例提供的人脸畸变校正方法进行进一步说明。
图3为本申请实施例所提供的再一种人脸畸变校正方法的流程示意图。
如图3所示，该人脸畸变校正方法，包括以下步骤：
步骤301，对获取的图像进行人脸检测，确定图像中包括的各人脸框的位置。
步骤302，根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内。
步骤303，若第一人脸框的至少部分区域未在预设视场角范围内，则对第一人脸框中的人脸进行畸变校正。
上述步骤301-303的具体实现过程及原理，可以参照上述实施例的详细描述，此处不再赘述。
步骤304，判断待校正的像素点是否位于第三人脸框内，其中，第三人脸框为位于预设视场角范围内的人脸框，若是，则执行步骤305；否则，执行步骤308。
在本申请实施例中，对第一人脸框中的人脸进行畸变校正时，为保证人脸畸变校正区域与非校正区域的平滑过渡，使得校正后的图像更加自然，可以对第一人脸框周围的像素点也进行校正。而第一人脸框周围的像素点可能落进位于预设视场角范围内的第三人脸框内，从而在本申请一种可能的实现方式中，对于每个待校正的像素点，可以首先判断该待校正的像素点是否位于第三人脸框内，以对位 于第三人脸框内的待校正的像素点进行保护处理，提升校正后的图像质量。
步骤305，判断当前的校正操作是否为平移操作，若是，则执行步骤306，否则，执行步骤307。
其中，校正操作，可以是平移操作、旋转操作、扭曲操作、插值操作等，本申请实施例对此不做限定。
作为一种可能的实现方式，在对第三人脸框中的人脸进行保护时，可以允许对第三人脸框中的像素点进行平移操作。因此，在确定待校正的像素点位于第三人脸框中时，可以进一步判断当前的校正操作是否为平移操作，以确定是否可以采用当前的校正操作对待校正的像素点进行校正。
步骤306，对待校正的像素点进行平移。
在本身实施例中，若确定待校正的像素点位于第三人脸框中，且当前的校正操作为平移操作，则可以根据当前的校正操作中包括的参数(平移的方向、平移的长度等)，对待校正的像素点进行平移，以完成对待校正的像素点的校正。
步骤307，结束对待校正的像素点的校正处理。
在本申请实施例中，若确定待校正的像素点位于第三人脸框中，且当前的校正操作不是平移操作，则可以结束对待校正的像素点的校正处理，即不对待校正的像素点进行校正，从而对第三人脸框中的人脸进行保护。
步骤308，对待校正的像素点进行校正。
在本申请实施例中，若确定待校正的像素点没有位于第三人脸框中，则可以直接根据当前的待校正操作对待校正的像素点进行校正处理。在对第一人脸框中的人脸进行畸变校正时，对于所有待校正的像素点均需要重复进行步骤304-308的校正过程，直至所有待校正的像素点均处理完毕，进而生成校正后的图像。
作为一种可能的实现方式，还可以在人脸畸变校正的误差估计中引入人脸保护项，进而根据人脸畸变校正的误差估计衡量对第一人脸框中的人脸进行畸变校正的效果，以及对不需要进行人脸校正的人脸的保护效果。引入人脸保护项的人脸畸变校正的误差估计可以根据公式(2)确定。
Figure PCTCN2020127551-appb-000002
其中，E为引入人脸保护项的人脸畸变校正的误差估计，w i为人脸对应的第i个人脸网格的权重，v i为对人脸对应的第i个人脸网格进行校正后的目标人脸网格，S k为相似变换矩阵，u i为人脸对应的第i个人脸网格对应的保形投影网格，t k为平移向量，p i为人脸对应的第i个人脸网格，k为人脸对应的人脸网格的数量，i为人脸对应的人脸网格的序号。
在本申请实施例中，若引入人脸保护项的人脸畸变校正的误差估计E小于或等于第二误差阈值，则可以确定对第一人脸框中的人脸进行人脸畸变校正的效果较好，以及对不需要进行人脸畸变校正的人脸保护效果较好，从而可以结束对第一人脸框中的人脸进行人脸畸变校正过程；若引入人脸保护项的人脸畸变校正的误差估计E大于第二误差阈值，则可以确定对第一人脸框中的人脸进行人脸畸变校正的效果较差，或对不需要进行人脸畸变校正的人脸的保护效果较差，从而可以继续对第一人脸框中的人脸进行进一步人脸畸变校正，直至引入人脸保护项的人脸畸变校正的误差估计E小于或等于第二误差阈值。
需要说的的是，实际使用时，第二误差阈值的具体取值可以根据实际需要及具体的应用场景预设，本申请实施例对此不做限定。
本申请实施例提供的人脸畸变校正方法，通过对获取的图像进行人脸检测，确定图像中包括的各人脸框的位置，并在确定第一人脸框的至少部分区域未在预设视场角范围内时，对第一人脸框中的人脸进行畸变校正，以及在进行畸变校正时判断待校正的像素点是否位于第三人脸框内，进而仅对位于第三人脸框中的待校正的像素点进行平移操作，对没有位于第三人脸框中的待校正像素点进行相应的校正操作。由此，通过仅允许对位于第三人脸框内的待校正像素点进行平移操作，进一步提高了对处于预设视场角范围内的人脸的保护效果，提高了校正后的图像质量，改善了用户体验。
为了实现上述实施例，本申请还提出一种人脸畸变校正装置。
图4为本申请实施例提供的一种人脸畸变校正装置的结构示意图。
如图4所示，该人脸畸变校正装置40，包括：
第一确定模块41，用于对获取的图像进行人脸检测，确定图像中包括的各人脸框的位置；
判断模块42，用于根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内。
校正模块43，用于若第一人脸框的至少部分区域未在预设视场角范围内，则对第一人脸框中的人脸进行畸变校正，以生成校正后的图像。
在实际使用时，本申请实施例提供的人脸畸变校正装置，可以被配置在任意电子设备中，以执行前述人脸畸变校正方法。
本申请实施例提供的人脸畸变校正装置，通过对获取的图像进行人脸检测，确定图像中包括的各人脸框的位置，并根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内，进而在第一人脸框的至少部分区域未在预设视场角范围内时，对第一人脸框中的人脸进行畸变校正，以生成校正后的图像。由此，通过对未处于预设视场角范围内的人脸进行校正，对处于预设视场角范围内的人脸进行保护，从而实现了保护未畸变人脸质量的同时，对畸变人脸进行校正，提高了人脸畸变校正的效果，改善了用户体验。
在本申请一种可能的实现形式中，上述人脸畸变校正装置40，还包括：
投影模块，用于对第一人脸框进行保形投影，以生成第二人脸框；
第二确定模块，用于确定第二人脸框与第一人脸框间满足预设条件。
进一步的，在本申请另一种可能的实现形式中，上述第二确定模块，具体用于：
确定第二人脸框中至少一条边的长度与第一人脸框中对应边的长度满足预设条件。
进一步的，在本申请再一种可能的实现形式中，上述人脸畸变校正装置40，还包括：
第三确定模块，用于根据采集图像的摄像模组的属性，确定预设的视场角范围，其中，摄像模组的属性包括摄像模组在终端中的设置位置、摄像模组的视场角。
在本申请一种可能的实现形式中，上述校正模块43，具体用于：
判断待校正的像素点是否位于第三人脸框内，其中，第三人脸框为位于预设视场角范围内的人脸框；
若待校正的像素点位于第三人脸框内，则判断当前的校正操作是否为平移操作；
若当前的校正操作是平移操作，则对待校正的像素点进行平移。
进一步的，在本申请另一种可能的实现形式中，上述校正模块43，还用于：
若待校正的像素点未位于第三人脸框内，则对待校正的像素点进行校正。
进一步的，在本申请再一种可能的实现形式中，上述校正模块43，还用于：
若当前的校正操作非平移操作，则结束对待校正的像素点的校正处理。
需要说明的是，前述对图1、图2、图3所示的人脸畸变校正方法实施例的解释说明也适用于该实 施例的人脸畸变校正装置40，此处不再赘述。
本申请实施例提供的人脸畸变校正装置，通过对获取的图像进行人脸检测，确定图像中包括的各人脸框的位置，并在确定第一人脸框的至少部分区域未在预设视场角范围内时，对第一人脸框中的人脸进行畸变校正，以及在进行畸变校正时判断待校正的像素点是否位于第三人脸框内，进而仅对位于第三人脸框中的待校正的像素点进行平移操作，对没有位于第三人脸框中的待校正像素点进行相应的校正操作。由此，通过仅允许对位于第三人脸框内的待校正像素点进行平移操作，进一步提高了对处于预设视场角范围内的人脸的保护效果，提高了校正后的图像质量，改善了用户体验。
为了实现上述实施例，本申请还提出一种电子设备。
图5为本发明一个实施例的电子设备的结构示意图。
如图5所示，上述电子设备200包括：
存储器210及处理器220，连接不同组件(包括存储器210和处理器220)的总线230，存储器210存储有计算机程序，当处理器220执行所述程序时实现本申请实施例所述的人脸畸变校正方法。
总线230表示几类总线结构中的一种或多种，包括存储器总线或者存储器控制器，外围总线，图形加速端口，处理器或者使用多种总线结构中的任意总线结构的局域总线。举例来说，这些体系结构包括但不限于工业标准体系结构(ISA)总线，微通道体系结构(MAC)总线，增强型ISA总线、视频电子标准协会(VESA)局域总线以及外围组件互连(PCI)总线。
电子设备200典型地包括多种电子设备可读介质。这些介质可以是任何能够被电子设备200访问的可用介质，包括易失性和非易失性介质，可移动的和不可移动的介质。
存储器210还可以包括易失性存储器形式的计算机系统可读介质，例如随机存取存储器(RAM)240和/或高速缓存存储器250。电子设备200可以进一步包括其它可移动/不可移动的、易失性/非易失性计算机系统存储介质。仅作为举例，存储系统260可以用于读写不可移动的、非易失性磁介质(图5未显示，通常称为“硬盘驱动器”)。尽管图5中未示出，可以提供用于对可移动非易失性磁盘(例如“软盘”)读写的磁盘驱动器，以及对可移动非易失性光盘(例如CD-ROM,DVD-ROM或者其它光介质)读写的光盘驱动器。在这些情况下，每个驱动器可以通过一个或者多个数据介质接口与总线230相连。存储器210可以包括至少一个程序产品，该程序产品具有一组(例如至少一个)程序模块，这些程序模块被配置以执行本申请各实施例的功能。
具有一组(至少一个)程序模块270的程序/实用工具280，可以存储在例如存储器210中，这样的程序模块270包括——但不限于——操作系统、一个或者多个应用程序、其它程序模块以及程序数据，这些示例中的每一个或某种组合中可能包括网络环境的实现。程序模块270通常执行本申请所描述的实施例中的功能和/或方法。
电子设备200也可以与一个或多个外部设备290(例如键盘、指向设备、显示器291等)通信，还可与一个或者多个使得用户能与该电子设备200交互的设备通信，和/或与使得该电子设备200能与一个或多个其它计算设备进行通信的任何设备(例如网卡，调制解调器等等)通信。这种通信可以通过输入/输出(I/O)接口292进行。并且，电子设备200还可以通过网络适配器293与一个或者多个网络(例如局域网(LAN)，广域网(WAN)和/或公共网络，例如因特网)通信。如图所示，网络适配器293通过总线230与电子设备200的其它模块通信。应当明白，尽管图中未示出，可以结合电子设备200使用其它硬件和/或软件模块，包括但不限于：微代码、设备驱动器、冗余处理单元、外部磁盘驱动阵列、RAID系统、磁带驱动器以及数据备份存储系统等。
处理器220通过运行存储在存储器210中的程序，从而执行各种功能应用以及数据处理。
需要说明的是，本实施例的电子设备的实施过程和技术原理参见前述对本申请实施例的人脸畸变校正方法的解释说明，此处不再赘述。
本申请实施例提供的电子设备，可以执行如前所述的人脸畸变校正方法，通过对获取的图像进行人脸检测，确定图像中包括的各人脸框的位置，并根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内，进而在第一人脸框的至少部分区域未在预设视场角范围内时，对第一人脸框中的人脸进行畸变校正，以生成校正后的图像。由此，通过对未处于预设视场角范围内的人脸进行校正，对处于预设视场角范围内的人脸进行保护，从而实现了保护未畸变人脸质量的同时，对畸变人脸进行校正，提高了人脸畸变校正的效果，改善了用户体验。
为了实现上述实施例，本申请还提出一种计算机可读存储介质。
其中，该计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时，以实现本申请实施例所述的人脸畸变校正方法。
为了实现上述实施例，本申请再一方面实施例提供一种计算机程序，该程序被处理器执行时，以实现本申请实施例所述的人脸畸变校正方法。
一种可选实现形式中，本实施例可以采用一个或多个计算机可读的介质的任意组合。计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本文件中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。
计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括——但不限于——电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。
计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括——但不限于——无线、电线、光缆、RF等等，或者上述的任意合适的组合。
可以以一种或多种程序设计语言或其组合来编写用于执行本申请操作的计算机程序代码，所述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户电子设备上执行、部分地在用户电子设备上执行、作为一个独立的软件包执行、部分在用户电子设备上部分在远程电子设备上执行、或者完全在远程电子设备或服务器上执行。在涉及远程电子设备的情形中，远程电子设备可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到用户电子设备，或者，可以连接到外部电子设备(例如利用因特网服务提供商来通过因特网连接)。
本领域技术人员在考虑说明书及实践这里申请的发明后，将容易想到本申请的其它实施方案。本申请旨在涵盖本申请的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本申请的一般性原理并包括本申请未发明的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本申请的真正范围和精神由权利要求指出。
应当理解的是，本申请并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本申请的范围仅由所附的权利要求来限制。

Claims (22)

  1. 一种人脸畸变校正方法，其特征在于，包括：
    对获取的图像进行人脸检测，确定所述图像中包括的各人脸框的位置；
    根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内；
    若第一人脸框的至少部分区域未在所述预设视场角范围内，则对所述第一人脸框中的人脸进行畸变校正，以生成校正后的图像。
  2. 如权利要求1所述的方法，其特征在于，所述对所述第一人脸框中的人脸进行畸变校正之前，还包括：
    对所述第一人脸框进行保形投影，以生成第二人脸框；
    确定所述第二人脸框与所述第一人脸框间满足预设条件。
  3. 如权利要求2所述的方法，其特征在于，所述确定所述第二人脸框与所述第一人脸框间满足预设条件，包括：
    确定所述第二人脸框中至少一条边的长度与所述第一人脸框中对应边的长度满足所述预设条件。
  4. 如权利要求1所述的方法，其特征在于，所述判断每个人脸框是否在预设视场角范围内之前，还包括：
    根据采集所述图像的摄像模组的属性，确定所述预设的视场角范围，其中，所述摄像模组的属性包括所述摄像模组在终端中的设置位置、所述摄像模组的视场角。
  5. 如权利要求1-4任一所述的方法，其特征在于，所述对所述第一人脸框中的人脸进行畸变校正，包括：
    判断待校正的像素点是否位于第三人脸框内，其中，第三人脸框为位于所述预设视场角范围内的人脸框；
    若所述待校正的像素点位于第三人脸框内，则判断当前的校正操作是否为平移操作；
    若所述当前的校正操作是平移操作，则对所述待校正的像素点进行平移。
  6. 如权利要求5所述的方法，其特征在于，所述判断所述待校正的像素点是否位于第三人脸框内之后，还包括：
    若所述待校正的像素点未位于第三人脸框内，则对所述待校正的像素点进行校正。
  7. 如权利要求6所述的方法，其特征在于，所述判断当前的校正操作是否为平移操作之后，还包括：
    若所述当前的校正操作非平移操作，则结束对所述待校正的像素点的校正处理。
  8. 一种人脸畸变校正装置，其特征在于，包括：
    第一确定模块，用于对获取的图像进行人脸检测，确定所述图像中包括的各人脸框的位置；
    判断模块，用于根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内；
    校正模块，用于若第一人脸框的至少部分区域未在所述预设视场角范围内，则对所述第一人脸框中的人脸进行畸变校正，以生成校正后的图像。
  9. 根据权利要求8所述的人脸畸变校正装置，其特征在于，还包括：
    投影模块，用于对第一人脸框进行保形投影，以生成第二人脸框；
    第二确定模块，用于确定第二人脸框与第一人脸框间满足预设条件。
  10. 根据权利要求9所述的人脸畸变校正装置，其特征在于，还包括：
    所述第二确定模块还用于确定所述第二人脸框中至少一条边的长度与所述第一人脸框中对应边的 长度满足所述预设条件。
  11. 根据权利要求8所述的人脸畸变校正装置，其特征在于，还包括：
    第三确定模块，用于根据采集所述图像的摄像模组的属性，确定所述预设的视场角范围，其中，所述摄像模组的属性包括所述摄像模组在终端中的设置位置、所述摄像模组的视场角。
  12. 根据权利要求8-11任一所述的人脸畸变校正装置，其特征在于，所述校正模块还用于判断待校正的像素点是否位于第三人脸框内，其中，第三人脸框为位于所述预设视场角范围内的人脸框；若所述待校正的像素点位于第三人脸框内，则判断当前的校正操作是否为平移操作；若所述当前的校正操作是平移操作，则对所述待校正的像素点进行平移。
  13. 根据权利要求12所述的人脸畸变校正装置，其特征在于，所述校正模块还用于若所述待校正的像素点未位于第三人脸框内，则对所述待校正的像素点进行校正。
  14. 根据权利要求13所述的人脸畸变校正装置，其特征在于，所述校正模块还用于若所述当前的校正操作非平移操作，则结束对所述待校正的像素点的校正处理。
  15. 一种电子设备，其特征在于，包括：存储器、处理器及存储在存储器上并可在处理器上运行的程序，其特征在于，所述处理器执行所述程序时可实现如下步骤：对获取的图像进行人脸检测，确定所述图像中包括的各人脸框的位置；根据每个人脸框的位置，判断每个人脸框是否在预设视场角范围内；若第一人脸框的至少部分区域未在所述预设视场角范围内，则对所述第一人脸框中的人脸进行畸变校正，以生成校正后的图像。
  16. 根据权利要求15所述的电子设备，其特征在于，所述处理器执行所述程序时还可实现如下步骤：对所述第一人脸框进行保形投影，以生成第二人脸框；确定所述第二人脸框与所述第一人脸框间满足预设条件。
  17. 根据权利要求16所述的电子设备，其特征在于，所述处理器执行所述程序时还可实现如下步骤：确定所述第二人脸框中至少一条边的长度与所述第一人脸框中对应边的长度满足所述预设条件。
  18. 根据权利要求16所述的电子设备，其特征在于，所述判断每个人脸框是否在预设视场角范围内之前，所述处理器执行所述程序时还可实现如下步骤：根据采集所述图像的摄像模组的属性，确定所述预设的视场角范围，其中，所述摄像模组的属性包括所述摄像模组在终端中的设置位置、所述摄像模组的视场角。
  19. 根据权利要求15-18任一所述的电子设备，其特征在于，所述处理器执行所述程序时还可实现如下步骤：判断待校正的像素点是否位于第三人脸框内，其中，第三人脸框为位于所述预设视场角范围内的人脸框；若所述待校正的像素点位于第三人脸框内，则判断当前的校正操作是否为平移操作；若所述当前的校正操作是平移操作，则对所述待校正的像素点进行平移。
  20. 根据权利要求19所述的电子设备，其特征在于，所述判断所述待校正的像素点是否位于第三人脸框内之后，所述处理器执行所述程序时还可实现如下步骤：若所述待校正的像素点未位于第三人脸框内，则对所述待校正的像素点进行校正。
  21. 根据权利要求20所述的电子设备，其特征在于，所述处理器执行所述程序时还可实现如下步骤：若所述当前的校正操作非平移操作，则结束对所述待校正的像素点的校正处理。
  22. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述程序被处理器执行时实现如权利要求1-7中任一所述的人脸畸变校正方法。
PCT/CN2020/127551 2019-12-09 2020-11-09 人脸畸变校正方法、装置、电子设备及存储介质 Ceased WO2021114990A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP20900298.9A EP4071707A4 (en) 2019-12-09 2020-11-09 METHOD AND DEVICE FOR CORRECTING FACIAL DISTORTION, ELECTRONIC DEVICE AND STORAGE MEDIA
US17/835,728 US12327339B2 (en) 2019-12-09 2022-06-08 Method and apparatus for correcting face distortion, electronic device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911252854.8 2019-12-09
CN201911252854.8A CN111105367B (zh) 2019-12-09 2019-12-09 人脸畸变校正方法、装置、电子设备及存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/835,728 Continuation US12327339B2 (en) 2019-12-09 2022-06-08 Method and apparatus for correcting face distortion, electronic device, and storage medium

Publications (1)

Publication Number Publication Date
WO2021114990A1 true WO2021114990A1 (zh) 2021-06-17

Family

ID=70423124

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/127551 Ceased WO2021114990A1 (zh) 2019-12-09 2020-11-09 人脸畸变校正方法、装置、电子设备及存储介质

Country Status (4)

Country Link
US (1) US12327339B2 (zh)
EP (1) EP4071707A4 (zh)
CN (1) CN111105367B (zh)
WO (1) WO2021114990A1 (zh)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111105367B (zh) 2019-12-09 2023-07-18 Oppo广东移动通信有限公司 人脸畸变校正方法、装置、电子设备及存储介质
CN113724140B (zh) * 2020-05-25 2024-08-13 华为技术有限公司 图像处理方法、电子设备、介质和系统
CN113747002B (zh) * 2020-05-29 2023-04-07 青岛海信移动通信技术股份有限公司 终端及图像拍摄方法
CN113741681B (zh) * 2020-05-29 2024-04-26 华为技术有限公司 一种图像校正方法与电子设备
CN112381740B (zh) * 2020-11-24 2024-02-06 维沃移动通信有限公司 图像处理方法、装置和电子设备
CN115205131B (zh) * 2021-04-14 2025-07-15 Oppo广东移动通信有限公司 畸变图像校正方法及装置、计算机可读介质和电子设备
CN113870148A (zh) * 2021-10-19 2021-12-31 Oppo广东移动通信有限公司 人脸畸变校正方法、装置、电子设备、芯片及存储介质
CN114119405A (zh) * 2021-11-23 2022-03-01 Oppo广东移动通信有限公司 图像处理方法及装置、计算机可读存储介质和电子设备
CN114119406A (zh) * 2021-11-23 2022-03-01 Oppo广东移动通信有限公司 图像处理方法及装置、计算机可读存储介质和电子设备
CN114187206B (zh) * 2021-12-15 2025-07-15 Oppo广东移动通信有限公司 图像畸变校正方法、装置、电子设备、芯片及存储介质
CN114612324B (zh) * 2022-03-09 2024-06-07 Oppo广东移动通信有限公司 用于畸变校正的图像处理方法、装置、介质与电子设备
WO2024001342A1 (zh) * 2022-06-30 2024-01-04 华为技术有限公司 成像畸变矫正方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104994281A (zh) * 2015-06-30 2015-10-21 广东欧珀移动通信有限公司 一种人脸畸变校正的方法及终端
CN107451965A (zh) * 2017-07-24 2017-12-08 深圳市智美达科技股份有限公司 畸变人脸图像校正方法、装置、计算机设备和存储介质
KR20190060228A (ko) * 2017-11-24 2019-06-03 한국전자통신연구원 3차원 컬러 메쉬 복원 방법 및 장치
CN109948397A (zh) * 2017-12-20 2019-06-28 Tcl集团股份有限公司 一种人脸图像校正方法、系统及终端设备
CN111105367A (zh) * 2019-12-09 2020-05-05 Oppo广东移动通信有限公司 人脸畸变校正方法、装置、电子设备及存储介质

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS596996B2 (ja) * 1978-08-11 1984-02-15 清水建設株式会社 シ−ルド掘進工法および装置
JP3596996B2 (ja) * 1996-11-15 2004-12-02 沖電気工業株式会社 生体の特徴を用いた個人認識装置およびこの個人認識装置を用いた自動取引システム
CA2359269A1 (en) * 2001-10-17 2003-04-17 Biodentity Systems Corporation Face imaging system for recordal and automated identity confirmation
JP5011504B2 (ja) * 2005-07-22 2012-08-29 カシオ計算機株式会社 画像合成装置、画像合成方法及びプログラム
JP4714174B2 (ja) * 2007-03-27 2011-06-29 富士フイルム株式会社 撮像装置
JP2013219544A (ja) * 2012-04-09 2013-10-24 Ricoh Co Ltd 画像処理装置、画像処理方法、及び画像処理プログラム
JP2014179937A (ja) * 2013-03-15 2014-09-25 Olympus Imaging Corp 撮像装置、撮像方法および撮像プログラム
JP5971216B2 (ja) * 2013-09-20 2016-08-17 カシオ計算機株式会社 画像処理装置、画像処理方法及びプログラム
CN105405104B (zh) 2015-09-24 2018-05-29 华为技术有限公司 一种人脸图像校正的方法及装置
CN105554403B (zh) * 2016-02-29 2018-12-04 广东欧珀移动通信有限公司 控制方法、控制装置及电子装置
CN106797456B (zh) * 2016-12-30 2019-02-01 深圳前海达闼云端智能科技有限公司 投影图像校正方法、校正装置及机器人
US10572982B2 (en) * 2017-10-04 2020-02-25 Intel Corporation Method and system of image distortion correction for images captured by using a wide-angle lens
CN108932698B (zh) * 2017-11-17 2021-07-23 北京猎户星空科技有限公司 图像畸变的校正方法、装置、电子设备和存储介质
CN110232667B (zh) * 2019-06-17 2021-06-04 厦门美图之家科技有限公司 图像畸变修正方法、装置、电子设备及可读存储介质
US11132800B2 (en) * 2019-07-31 2021-09-28 Google Llc Real time perspective correction on faces

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104994281A (zh) * 2015-06-30 2015-10-21 广东欧珀移动通信有限公司 一种人脸畸变校正的方法及终端
CN107451965A (zh) * 2017-07-24 2017-12-08 深圳市智美达科技股份有限公司 畸变人脸图像校正方法、装置、计算机设备和存储介质
KR20190060228A (ko) * 2017-11-24 2019-06-03 한국전자통신연구원 3차원 컬러 메쉬 복원 방법 및 장치
CN109948397A (zh) * 2017-12-20 2019-06-28 Tcl集团股份有限公司 一种人脸图像校正方法、系统及终端设备
CN111105367A (zh) * 2019-12-09 2020-05-05 Oppo广东移动通信有限公司 人脸畸变校正方法、装置、电子设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4071707A4 *

Also Published As

Publication number Publication date
US20220301121A1 (en) 2022-09-22
CN111105367B (zh) 2023-07-18
CN111105367A (zh) 2020-05-05
EP4071707A1 (en) 2022-10-12
US12327339B2 (en) 2025-06-10
EP4071707A4 (en) 2023-05-24

Similar Documents

Publication Publication Date Title
CN111105367B (zh) 人脸畸变校正方法、装置、电子设备及存储介质
CN107113381B (zh) 时空局部变形及接缝查找的容差视频拼接方法、装置及计算机可读介质
WO2018214365A1 (zh) 图像校正方法、装置、设备、系统及摄像设备和显示设备
CN112686824A (zh) 图像校正方法、装置、电子设备和计算机可读介质
CN106993133B (zh) 一种畸变校正方法及终端
JP2010118040A (ja) 魚眼補正と透視歪み削減の画像処理方法及び画像処理装置
CN104994281A (zh) 一种人脸畸变校正的方法及终端
CN110784651A (zh) 一种防抖方法及电子设备
CN113837970B (zh) 图像数据的脱敏方法和装置
WO2021189733A1 (zh) 图像处理方法及装置、电子设备、存储介质
WO2022179045A1 (zh) 图像处理方法、装置、存储介质、程序及程序产品
WO2021114991A1 (zh) 人脸图像中直线处理方法、装置、终端设备和存储介质
WO2018058476A1 (zh) 一种图像校正方法及装置
CN115174878A (zh) 投影画面校正方法、装置和存储介质
JP2025510536A (ja) 画像処理方法及び装置、並びにデバイス
CN111429353A (zh) 图像拼接及全景图拼接方法和装置、存储介质、电子设备
CN111314606A (zh) 拍照方法、装置、电子设备及存储介质
CN113870148A (zh) 人脸畸变校正方法、装置、电子设备、芯片及存储介质
WO2021179923A1 (zh) 用户面部图像展示方法、展示装置及对应的存储介质
CN109379536B (zh) 画面生成方法、装置、终端及对应的存储介质
CN111353945A (zh) 鱼眼图像校正方法、装置及存储介质
CN111161148B (zh) 一种全景图像生成方法、装置、设备和存储介质
CN114463213A (zh) 视频处理方法、视频处理装置、终端及存储介质
CN115690191A (zh) 光心确定方法、装置、电子设备及介质
CN113628190A (zh) 一种深度图去噪方法、装置、电子设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20900298

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020900298

Country of ref document: EP

Effective date: 20220706