WO2023162131A1 - Dispositif de conversion d'image, procédé de conversion d'image, et programme de conversion d'image - Google Patents

Dispositif de conversion d'image, procédé de conversion d'image, et programme de conversion d'image Download PDF

Info

Publication number
WO2023162131A1
WO2023162131A1 PCT/JP2022/007869 JP2022007869W WO2023162131A1 WO 2023162131 A1 WO2023162131 A1 WO 2023162131A1 JP 2022007869 W JP2022007869 W JP 2022007869W WO 2023162131 A1 WO2023162131 A1 WO 2023162131A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature point
image
feature
point
points
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/007869
Other languages
English (en)
Japanese (ja)
Inventor
雄貴 蔵内
真奈 笹川
直紀 萩山
文香 佐野
隆二 山本
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NTT Inc
Original Assignee
Nippon Telegraph and Telephone Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nippon Telegraph and Telephone Corp filed Critical Nippon Telegraph and Telephone Corp
Priority to PCT/JP2022/007869 priority Critical patent/WO2023162131A1/fr
Priority to JP2024502364A priority patent/JP7704288B2/ja
Publication of WO2023162131A1 publication Critical patent/WO2023162131A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • Embodiments of the present invention relate to an image conversion device, an image conversion method, and an image conversion program.
  • Non-Patent Document 1 discloses the possibility of manipulating emotional experience through real-time facial expression deformation feedback.
  • a subject's face is tracked in real time and natural facial expression deformation processing is performed.
  • the Rigid MLS (Moving Least Squares) method is used as an image transformation method to transform facial expressions in facial images.
  • the Rigid MLS method is a method of distorting an image by moving each control point using feature points in the image recognized from the image as control points.
  • the face image is an image obtained by photographing the face of the subject, an image obtained by extracting the face of a computer-generated avatar, or the like.
  • the eyebrows recognized only on the upper side are image-transformed by this method so as to be moved upward, the obtained face image will have thicker eyebrows, and only a face image with an unnatural expression will be obtained.
  • the face image with an unnatural expression will be similarly obtained when the image is deformed.
  • the present invention seeks to provide an image conversion technique that enables conversion into an image with a natural expression even when only one side of the facial parts is recognized.
  • an image conversion device includes a control point generation section and an expression conversion section.
  • the control point generation unit generates second feature points that are unrecognized feature points on the other side based on first feature points that are feature points on one side of facial parts recognized from an image of a person's face. Add the first and second feature points as control points.
  • the facial expression transforming unit transforms the control points by a deformation amount according to the transformed facial expression to be transformed, thereby obtaining a transformed image in which the facial expression of the person is transformed.
  • the feature points on one side of the face part are added to convert the image. It is possible to provide an image conversion technique that enables conversion into a face image of expression.
  • FIG. 1 is a block diagram showing an example of the configuration of an image conversion device according to one embodiment of the invention.
  • FIG. 2 is a diagram showing an example of the hardware configuration of the image conversion device.
  • FIG. 3 is a diagram showing an example of facial feature points.
  • FIG. 4 is a diagram showing an example of a storage form of feature points.
  • FIG. 5 is a diagram showing an example of a storage form of the amount of change.
  • FIG. 6 is a flow chart showing an example of an image conversion processing operation by the image conversion device.
  • FIG. 7 is a schematic diagram for explaining the relationship between the feature points of the eyebrows and the feature points above the eyes.
  • FIG. 8 is a schematic diagram for explaining a method of adding feature points on the lower side of eyebrows.
  • FIG. 9 is a schematic diagram for explaining a method of adding characteristic points of eyelids.
  • FIG. 1 is a block diagram showing an example of the configuration of an image conversion device 1 according to one embodiment of the invention.
  • the image conversion device 1 has an image acquisition section 11 , a feature point recognition section 12 , a control point generation section 13 , a converted facial expression input section 14 , a change amount storage section 15 , a facial expression conversion section 16 and an image output section 17 .
  • the image acquisition unit 11 acquires a face image from a web camera, avatar, or the like.
  • the image acquisition unit 11 outputs the acquired face image to the feature point recognition unit 12 and the facial expression conversion unit 16 .
  • the feature point recognition unit 12 receives the face image acquired by the image acquisition unit 11, and recognizes feature points from the face image. A method of recognizing feature points in the feature point recognition unit 12 will be described later.
  • the feature point recognition unit 12 outputs the recognized feature points to the control point generation unit 13 .
  • the control point generation unit 13 receives as input the first feature points that are the feature points recognized by the feature point recognition unit 12, and generates second feature points that are unrecognized feature points based on the input first feature points. Add the feature points of . For example, the control point generation unit 13 calculates the distance between the eyebrow feature point, which is the first feature point, and the eye feature point, and adds half the distance obtained from each eyebrow feature point downward to the second feature point. Add the feature points of . A method for adding the second feature point will be described in detail later.
  • the control point generator 13 outputs the first and second feature points to the facial expression converter 16 as control points. Based on which of the first feature points the second feature points are added and the number of second feature points to be added are predetermined. Therefore, the number of control points is also predetermined.
  • the converted facial expression input unit 14 acquires a converted facial expression, such as a smiling face, which is specified and input by the user from a user interface such as a keyboard.
  • the converted facial expression input unit 14 outputs the acquired converted facial expression to the facial expression conversion unit 16 .
  • the change amount storage unit 15 stores in advance the change amount for each control point for each facial expression to be converted.
  • the amount of change is information indicating how much the control point should be moved.
  • the amount of change can be obtained in advance by, for example, adjusting a specific facial image so that the user applies expression deformation processing to an expressionless face so as to obtain a natural expression.
  • the facial expression transforming unit 16 receives the face image acquired by the image acquiring unit 11, the control points output by the control point generating unit 13, and the transformed facial expression acquired by the transformed facial expression input unit 14. Moreover, the facial expression transforming unit 16 reads from the variation storage unit 15 the amount of change in the facial expression to be converted indicated by the transformed facial expression input from the transformed facial expression input unit 14 .
  • the change amount storage unit 15 obtains a face image in which the expression of the face image is converted by moving each control point in the input face image based on the read movement amount of the control point.
  • the facial expression conversion section 16 outputs the converted face image to the image output section 17 .
  • the image output unit 17 receives the face image after conversion from the facial expression conversion unit 16, and outputs the input face image.
  • output includes, for example, storing in a storage medium, displaying on a display, transmitting to another device via a communication network, and the like.
  • FIG. 2 is a diagram showing an example of the hardware configuration of the image conversion device 1. As shown in FIG.
  • the image conversion device 1 is composed of a computer such as a personal computer, a smart phone, a server computer, etc., for example.
  • the image conversion device 1 has a hardware processor 100 such as a CPU (Central Processing Unit), as shown in FIG. By using a multi-core and multi-threaded CPU, it is possible to execute a plurality of information processes at the same time. Also, the processor 100 may include multiple CPUs.
  • a program memory 200, a data memory 300, a communication interface 400, and an input/output interface (input/output IF in FIG. 2) 500 are connected to the processor 100 via a bus 600. connected.
  • the communication interface 400 can include, for example, one or more wired or wireless communication modules.
  • the communication interface 400 can communicate with other computers, web cameras, etc. connected via a cable or a network such as a LAN (Local Area Network) or the Internet.
  • LAN Local Area Network
  • the input unit 700 and a display unit 800 are connected to the input/output interface 500 .
  • the input unit 700 includes input devices such as a keyboard, a pointing device such as a mouse, a sensor device such as a camera, and the like.
  • the display unit 800 is a display device such as a liquid crystal display, a CRT (Cathode Ray Tube) display, or the like.
  • the input unit 700 and the display unit 800 can also use what is called a tablet-type input/display device.
  • This type of input/display device is configured by arranging an input detection sheet adopting an electrostatic method or a pressure method on a display screen of a display device using liquid crystal or organic EL (Electro Luminescence), for example.
  • the input/output interface 500 inputs to the processor 100 the operation information input through the input unit 700 and causes the display unit 800 to display display information generated by the processor 100 .
  • the input unit 700 and the display unit 800 may not be connected to the input/output interface 500.
  • the input unit 700 and the display unit 800 are provided with a communication unit for connecting to the communication interface 400 directly or via a network, so that information can be exchanged with the processor 100 .
  • the input/output interface 500 may have a read/write function for a recording medium such as a semiconductor memory such as a flash memory, or may be connected to a reader/writer having a read/write function for such a recording medium. It may have functions. Furthermore, the input/output interface 500 may have a connection function with other devices.
  • the program memory 200 is a combination of a non-volatile memory that can be written and read at any time and a non-volatile memory that can only be read at any time as a non-temporary tangible computer-readable storage medium.
  • Non-volatile memories that can be written and read at any time are, for example, HDDs (Hard Disk Drives), SSDs (Solid State Drives), and the like.
  • Non-volatile memory that can only be read at any time is, for example, ROM.
  • the program memory 200 stores a program necessary for the processor 100 to execute various control processes according to one embodiment, such as an image conversion program.
  • the processing function units in each unit of the image acquisition unit 11, the feature point recognition unit 12, the control point generation unit 13, the converted expression input unit 14, the change amount storage unit 15, the expression conversion unit 16, and the image output unit 17 are: Both can be realized by causing the processor 100 to read and execute the image conversion program stored in the program memory 200 .
  • Some or all of these processing functions may be implemented in various other forms, including integrated circuits such as Application Specific Integrated Circuits (ASICs) or field-programmable gate arrays (FPGAs). May be.
  • ASICs Application Specific Integrated Circuits
  • FPGAs field-programmable gate arrays
  • the data memory 300 is used as a tangible computer-readable storage medium, for example, by combining the above nonvolatile memory and a volatile memory such as RAM (Random Access Memory).
  • This data memory 300 is used to store various data acquired and created in the process of performing various processes. That is, in the data memory 300, an area for storing various data is appropriately secured in the process of performing various processes.
  • the data memory 300 includes, for example, an acquired image storage unit 301, a feature point storage unit 302, a control point storage unit 303, a converted facial expression designation storage unit 304, a change amount storage unit 305, a converted image storage unit 306, and so on. and a temporary storage unit 307 can be provided.
  • the acquired image storage unit 301 is used to store face images acquired when the processor 100 operates as the image acquisition unit 11 described above.
  • the feature point storage unit 302 is used to store feature points acquired when the processor 100 operates as the feature point recognition unit 12 described above.
  • FIG. 3 is a diagram showing an example of facial feature points.
  • the asterisks in FIG. 3 are feature points recognized by the processor 100, and the numbers attached to each feature point are unique feature point IDs for identifying each feature point.
  • the number of feature point IDs and the portion of the face for each feature point ID are determined by the feature point recognition method employed. For example, the feature point with the feature point ID "18" is predetermined as the left edge of the left eyebrow.
  • FIG. 4 is a diagram showing an example of a storage form of feature points in the feature point storage unit 302.
  • the feature point storage unit 302 stores x-coordinates and y-coordinates in the face image in a table format in association with feature point IDs. Coordinate values are in pixels. Therefore, in the example of FIG. 3, the feature point storage unit 302 stores the xy coordinates of feature points with feature point IDs "1" to "68".
  • the control point storage unit 303 is used to store control points generated when the processor 100 operates as the control point generation unit 13 described above.
  • the storage form of the control points in the control point storage unit 303 is the same as the storage form of the feature points in the feature point storage unit 302 shown in FIG. 4, for example. That is, the control point storage unit 303 can store the x-coordinate and y-coordinate in the face image in association with the control point ID in a table format.
  • the control point storage unit 303 stores the feature point IDs "1" to "68" assigned to the feature points shown in FIG. ” to “68” are associated with each other and stored.
  • the processor 100 also stores the xy coordinates of the second feature points, which are the added feature points, in association with the control point IDs "69" and so on.
  • the converted facial expression specification storage unit 304 is used to store the converted facial expression specified by the user, which is acquired when the processor 100 operates as the above-described converted facial expression input unit 14 .
  • the change amount storage unit 305 corresponds to the change amount storage unit 15 described above.
  • FIG. 5 is a diagram showing an example of the storage form of the amount of change in the amount of change storage unit 305.
  • the change amount storage unit 305 can have a table format that stores the change amount of the x-coordinate and the change amount of the y-coordinate in association with the control point ID for each converted facial expression.
  • the delta value is in pixels.
  • the amount of change is represented by the direction and amount of movement of the control point. For example, a movement amount of "+1" represents a movement of 1 pixel in the positive direction.
  • the converted image storage unit 306 is used to store face images converted when the processor 100 operates as the facial expression conversion unit 16 described above.
  • a temporary storage unit 307 stores the acquired image storage unit 301, the feature point storage unit 302, the control point storage unit 303, the converted facial expression designation storage unit 304, the change amount storage unit 305, and the converted image storage unit 304, which are generated during the operation of the processor 100. It is used to store various intermediate data that are not stored in unit 306 .
  • FIG. 6 is a flowchart showing an example of the image conversion processing operation by the image conversion device 1.
  • the processor 100 of the image conversion device 1 reads and executes the image conversion program stored in the program memory 200, thereby starting the operation of the image conversion device 1 shown in this flow chart. Execution of the image conversion program by the processor 100 is started when an instruction to perform image conversion is given from the input unit 700 via the input/output interface 500 or via the communication interface 400 .
  • the processor 100 operates as the converted facial expression input unit 14 and waits for the user's specification input of a converted facial expression, such as a smile, which is the facial expression to be converted (step S1). For example, the processor 100 determines whether or not the input signal from the input unit 700 via the input/output interface 500 or the communication interface 400 includes a specified input of a converted facial expression. If there is an input specifying a converted facial expression, the processor 100 proceeds to the process of step S2.
  • the processor 100 stores the designated converted facial expression in the converted facial expression designation storage section 304 of the data memory 300 (step S2).
  • the processor 100 operates as the image acquisition unit 11 and acquires a face image (step S3).
  • the processor 100 acquires through the input/output interface 500 an image of the subject's face captured by the camera of the input unit 700 .
  • the processor 100 acquires a face image captured by a web camera connected to a network or an avatar face generated by another computer via the communication interface 400 .
  • the processor 100 stores the acquired face image in the acquired image storage section 301 of the data memory 300 .
  • the processor 100 operates as the feature point recognition unit 12 and recognizes the first feature points from the face image stored in the acquired image storage unit 301 (step S4).
  • the processor 100 uses the face_landmark_detection function of dlib (see, for example, http://dlib.net/face_landmark_detection.py.html) to recognize feature points in the face image.
  • the processor 100 extracts the gradient direction distribution of luminance called HOG (Histogram of Oriented Gradients) features from the input face image.
  • HOG Heistogram of Oriented Gradients
  • a model trained based on data in which HOG features and positions of facial feature points are associated is generally provided. Therefore, the processor 100 inputs the extracted HOG features into this learning model to obtain the positions of the feature points of the face.
  • the processor 100 stores the acquired position of the first feature point in the feature point storage unit 302 of the data memory 300 .
  • the processor 100 operates as the control point generation unit 13 and generates control points (step S5). Specifically, the processor 100 stores the recognized first feature point in the control point storage unit 303 of the data memory 300 as a control point. Furthermore, the processor 100 adds a second feature point, which is the feature point on the other side, to the facial part for which the feature points on only one side have been recognized. The processor 100 then stores the added second feature points in the control point storage unit 303 as additional control points.
  • eyebrows, eyelids, and contours are examples of facial parts for which feature points are recognized on only one side. Since only the upper feature points of the eyebrows are recognized, the processor 100 adds the lower feature points of the eyebrows. Since only the upper feature point of the eye, which is the lower feature point of the double eyelid, is recognized, the processor 100 adds the upper feature point. Since the contour without shadows is recognized as a feature point, the processor 100 adds the feature points of the shadowed part.
  • the processor 100 adds eyebrow feature points as follows.
  • FIG. 7 is a schematic diagram for explaining the relationship between the feature points of the eyebrows and the feature points above the eyes.
  • the processor 100 draws a vertical line from each feature point of the eyebrows (feature point IDs “18” to “22”), and draws a vertical line from the eyebrow feature points (feature point IDs “37” to “40”). , the feature point with the closest distance from the perpendicular is obtained.
  • the processor 100 detects the feature point with the feature point ID “37” if the feature point has the feature point ID “18”, the feature point with the feature point ID “37” if the feature point has the feature point ID “19”, the feature point If it is a feature point with ID "20”, it is a feature point with feature point ID "38", if it is a feature point with feature point ID "21”, it is a feature point with feature point ID "39”, and if it is a feature point with feature point ID "22” For example, a feature point with a feature point ID of "40" is obtained, . At this time, d18, d19, d20, d21, d22, .
  • FIG. 8 is a schematic diagram for explaining a method of adding the lower feature point, which is the second eyebrow feature point.
  • the processor 100 calculates the average distance da of the distances d18-d27 between the above feature points.
  • the average distance da may be obtained without distinguishing between the right eye and the left eye, or the average distance da may be obtained separately since there is generally a slight difference between the left and right eyes.
  • the processor 100 sets 1/2 of the average distance da thus calculated, that is, da/2, as the feature point addition distance d, and adds the second feature point below each of the first feature points of the eyebrows by the feature point addition distance d. do. That is, the processor 100 adds the second feature point with the feature point ID of "69” below the first feature point with the feature point ID of "18” by the feature point addition distance d. Similarly, the processor 100 places the second feature point with the feature point ID "70” below the first feature point with the feature point ID "19", the second feature point with the feature point ID "20” below the first feature point with the feature point ID "20” , the second feature point with the feature point ID "78” below the first feature point with the feature point ID "27".
  • the processor 100 adds feature points of the eyelids, for example, as follows.
  • FIG. 9 is a schematic diagram for explaining a method of adding the second feature point of the eyelid.
  • the recognized unilateral feature point of the eyelid is the upper feature point of the eye. Therefore, when adding the second eyelid feature points, the average distance da used when adding the second eyebrow feature points can be used.
  • the processor 100 sets 1/4 of the left and right separate average distances da, ie, da/4, as the feature point additional distance d.
  • the processor 100 adds a second feature point above each feature point on the eye (the first feature points with feature point IDs "37" to "40") by a feature point addition distance d.
  • the processor 100 adds the second feature point with the feature point ID "79" above the first feature point with the feature point ID "37” by the feature point addition distance d. Similarly, the processor 100 causes the second feature point with the feature point ID "80” above the first feature point with the feature point ID "38", and the first feature point with the feature point ID "39”. , the second feature point with the feature point ID "86” above the first feature point with the feature point ID "46".
  • the processor 100 adds the second feature point of the contour shadow, for example, as follows. For example, the processor 100 adds a second feature point at a predetermined direction and distance for each contour feature point (first feature points with feature point IDs "1" to "17").
  • the processor 100 operates as the facial expression transforming unit 16 to transform the facial image stored in the acquired image storage unit 301 (step S6). That is, the processor 100 stores the control points stored in the control point storage unit 303 and the amount of change corresponding to the converted facial expression stored in the converted facial expression designation storage unit 304, which is stored in the amount of change storage unit 305. , to transform the face image.
  • processor 100 utilizes an implementation of MLS (see, eg, https://github.com/Jarvis73/Moving-Least-Squares), or the like. Specifically, the processor 100 moves each control point by the change amount corresponding to the transformed facial expression stored in the transformed facial expression designation storage unit 304 .
  • the x-y coordinates before conversion are (23, 45) for the control point with the control point ID "1" (see FIG. 4). is "+1" and the y-coordinate is "+2" (see FIG. 5), so that the pixel of the control point is moved to (24, 47).
  • x, y are the coordinates of the neighboring control point
  • x', y' are the coordinates obtained by adding the amount of change to the coordinates of the control point
  • a, b, c, d are the parameters
  • t x , t y are the translations.
  • the processor 100 calculates the least square mean of the coordinates x, y of the control points and the coordinates x′, y′ obtained by adding the amount of change, and sets the parameters a, b, c, d, t x , t y is determined by global optimization. Then, the coordinates of the object point to be transformed are set to x and y, and the coordinates after transformation are determined using the determined parameters.
  • the processor 100 uses the parameters a, b, c, d, t x , and t y obtained in this way to obtain the coordinates after transformation by the above affine transformation from the added control points.
  • the processor 100 stores the face image thus converted in the converted image storage section 306 of the data memory 300 as a converted image.
  • the processor 100 operates as the image output unit 17 and outputs the converted image stored in the converted image storage unit 306 (step S7).
  • the processor 100 causes the display unit 800 to display a facial image via the input/output interface 500 .
  • the processor 100 transmits over the network via the communication interface 400 and displays it on a display device connected to the network, or displays it on the display unit of another computer connected to the network.
  • the processor 100 determines whether or not to end the operation as the image conversion device 1 shown in this flow chart (step S8). For example, the processor 100 checks whether or not the user has instructed to end the image conversion from the input unit 700 via the input/output interface 500 or via the communication interface 400 . Here, when ending the operation, the processor 100 ends the operation shown in this flowchart.
  • the processor 100 operates as the converted facial expression input unit 14 and determines whether or not the user has entered a change designation input for the converted facial expression (step S9). If there is no change specification input for the converted facial expression, the processor 100 proceeds to the process of step S3. Also, when there is an input specifying a change in the converted facial expression, the processor 100 proceeds to the process of step S2.
  • the image conversion device 1 includes the control point generation section 13 and the facial expression conversion section 16 .
  • the control point generation unit 13 generates second feature points, which are unrecognized feature points on the other side, based on first feature points, which are feature points on one side of facial parts recognized from the image of a person's face. and let the first and second feature points be control points.
  • a facial expression conversion unit 16 obtains a transformed image in which the human facial expression is transformed by transforming the control points by a deformation amount according to the transformed facial expression to be transformed. Therefore, the image conversion apparatus 1 according to one embodiment converts an image by adding feature points on one side of the face part based on the feature points on the other side, so even if only one side of the face part is recognized. It is also possible to provide an image conversion technique that enables conversion into a face image with a natural expression.
  • facial parts include at least eyebrows or eyelids.
  • the lower eyebrow feature points and/or the upper eyelid feature points are added to create a natural image. It is possible to convert the facial expression into a face image.
  • control point generation unit 13 calculates the feature point addition distance d based on the distance between the feature points above the eyebrows and the feature points above the eyes, which are the first feature points.
  • control point generation unit 13 calculates the feature point addition distance d based on the distance between the feature point above the eyebrow and the feature point below the eyelid, which is the first feature point, and calculates the first feature point.
  • the amount of change storage unit 15 stores in advance the amount of change representing the amount of deformation for each control point for each converted facial expression to be converted, and the converted facial expression to be converted is input.
  • the control points corresponding to the second feature points are also used to create a facial image with a natural expression. It is possible to convert.
  • a fixed value such as 1/2 or 1/4 is used for the average distance da, but the user may specify an arbitrary value.
  • the user may be allowed to select which part of the face to add the second feature point, which is the feature point on the other side.
  • the method described in the embodiment can be executed by a computer (computer) as a program (software means), such as a magnetic disk (floppy (registered trademark) disk, hard disk, etc.), an optical disk (CD-ROM, DVD, MO, etc.), semiconductor memory (ROM, RAM, flash memory, etc.), or the like, or can be transmitted and distributed via a communication medium.
  • the programs stored on the medium also include a setting program for configuring software means (including not only execution programs but also tables and data structures) to be executed by the computer.
  • a computer that realizes this apparatus reads a program recorded on a recording medium, and optionally constructs software means by a setting program. The operation is controlled by this software means to execute the above-described processes.
  • the term "recording medium” as used herein is not limited to those for distribution, and includes storage media such as magnetic disks, semiconductor memories, etc. provided in computers or devices connected via a network.
  • the present invention is not limited to the above embodiments, and can be modified in various ways without departing from the gist of the invention at the implementation stage.
  • each embodiment may be implemented in combination as much as possible, in which case the combined effect can be obtained.
  • the above-described embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)

Abstract

Un dispositif de conversion d'image selon un mode de réalisation comprend une unité de génération de point de contrôle et une unité de conversion d'expression. Sur la base d'un premier point caractéristique qui se trouve sur un côté d'une partie de visage reconnue à partir d'une image du visage d'une personne, l'unité de génération de point de contrôle ajoute un second point caractéristique qui se trouve sur l'autre côté qui n'est pas reconnu, et règle les premier et second points caractéristiques en tant que points de contrôle. L'unité de conversion d'expression transforme les points de contrôle avec une quantité de transformation qui correspond à une expression de conversion à convertir, permettant ainsi d'obtenir une image convertie dans laquelle l'expression du visage de la personne est convertie.
PCT/JP2022/007869 2022-02-25 2022-02-25 Dispositif de conversion d'image, procédé de conversion d'image, et programme de conversion d'image Ceased WO2023162131A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/JP2022/007869 WO2023162131A1 (fr) 2022-02-25 2022-02-25 Dispositif de conversion d'image, procédé de conversion d'image, et programme de conversion d'image
JP2024502364A JP7704288B2 (ja) 2022-02-25 2022-02-25 画像変換装置、画像変換方法及び画像変換プログラム

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/007869 WO2023162131A1 (fr) 2022-02-25 2022-02-25 Dispositif de conversion d'image, procédé de conversion d'image, et programme de conversion d'image

Publications (1)

Publication Number Publication Date
WO2023162131A1 true WO2023162131A1 (fr) 2023-08-31

Family

ID=87765086

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/007869 Ceased WO2023162131A1 (fr) 2022-02-25 2022-02-25 Dispositif de conversion d'image, procédé de conversion d'image, et programme de conversion d'image

Country Status (2)

Country Link
JP (1) JP7704288B2 (fr)
WO (1) WO2023162131A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004164649A (ja) * 2002-11-14 2004-06-10 Eastman Kodak Co 刺激に応じてポートレート画像を変更するシステム及び方法
JP2004265406A (ja) * 2003-02-28 2004-09-24 Eastman Kodak Co バッチモードで処理されるポートレート画像を向上する方法及びシステム
JP2005215763A (ja) * 2004-01-27 2005-08-11 Konica Minolta Photo Imaging Inc 画像処理方法、画像処理装置及びプログラム

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007026088A (ja) * 2005-07-15 2007-02-01 Oki Electric Ind Co Ltd モデル生成装置
JP5028338B2 (ja) * 2008-06-06 2012-09-19 日立オムロンターミナルソリューションズ株式会社 顔向き検出ユニット、および顔向き検出方法
US10621771B2 (en) * 2017-03-21 2020-04-14 The Procter & Gamble Company Methods for age appearance simulation

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004164649A (ja) * 2002-11-14 2004-06-10 Eastman Kodak Co 刺激に応じてポートレート画像を変更するシステム及び方法
JP2004265406A (ja) * 2003-02-28 2004-09-24 Eastman Kodak Co バッチモードで処理されるポートレート画像を向上する方法及びシステム
JP2005215763A (ja) * 2004-01-27 2005-08-11 Konica Minolta Photo Imaging Inc 画像処理方法、画像処理装置及びプログラム

Also Published As

Publication number Publication date
JP7704288B2 (ja) 2025-07-08
JPWO2023162131A1 (fr) 2023-08-31

Similar Documents

Publication Publication Date Title
US10832039B2 (en) Facial expression detection method, device and system, facial expression driving method, device and system, and storage medium
JP4799105B2 (ja) 情報処理装置及びその制御方法、コンピュータプログラム、記憶媒体
JP5505409B2 (ja) 特徴点生成システム、特徴点生成方法および特徴点生成プログラム
CN110598638A (zh) 模型训练方法、人脸性别预测方法、设备及存储介质
JP4799104B2 (ja) 情報処理装置及びその制御方法、コンピュータプログラム、記憶媒体
JP7064257B2 (ja) 画像深度確定方法及び生き物認識方法、回路、装置、記憶媒体
JP2021517330A (ja) 画像中の物体を識別するための方法、及び当該方法を実施するためのモバイル装置
JP6207210B2 (ja) 情報処理装置およびその方法
CN109241810B (zh) 虚拟角色图像的构建方法及装置、存储介质
CN105184249A (zh) 用于人脸图像处理的方法和装置
JP7149124B2 (ja) 画像オブジェクト抽出装置及びプログラム
CN103562964A (zh) 图像处理装置、信息生成装置、图像处理方法、信息生成方法、控制程序及记录介质
CN118230394A (zh) 一种基于人脸3d关键点的面部表情驱动方法及装置
CN113129362A (zh) 一种三维坐标数据的获取方法及装置
JP6694638B2 (ja) プログラム、情報記憶媒体及び認識装置
JP2022537196A (ja) 表面認識
JP7700951B2 (ja) 画像変換装置、方法およびプログラム
CN112580395A (zh) 基于深度信息的3d人脸活体识别方法、系统、设备及介质
JP7704288B2 (ja) 画像変換装置、画像変換方法及び画像変換プログラム
JP2017122993A (ja) 画像処理装置、画像処理方法及びプログラム
US20250118103A1 (en) Joint image normalization and landmark detection
CN117746502B (zh) 图像标注方法、动作识别方法、装置和电子设备
JP7623994B2 (ja) 画像処理装置、画像処理方法、及びプログラム
Rollend et al. Face detection and object recognition for a retinal prosthesis
CN111199204A (zh) 一种基于OpenGL人脸图像处理方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22928658

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024502364

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 18832152

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22928658

Country of ref document: EP

Kind code of ref document: A1