WO2024196128A1 - Procédé et dispositif pour optimiser un modèle d'analyse de peau à l'aide d'une image de reproduction de caractéristique de réflexion de la lumière reconstruite par approche optique - Google Patents

Procédé et dispositif pour optimiser un modèle d'analyse de peau à l'aide d'une image de reproduction de caractéristique de réflexion de la lumière reconstruite par approche optique Download PDF

Info

Publication number
WO2024196128A1
WO2024196128A1 PCT/KR2024/003455 KR2024003455W WO2024196128A1 WO 2024196128 A1 WO2024196128 A1 WO 2024196128A1 KR 2024003455 W KR2024003455 W KR 2024003455W WO 2024196128 A1 WO2024196128 A1 WO 2024196128A1
Authority
WO
WIPO (PCT)
Prior art keywords
skin
image
light
feature map
analysis model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/KR2024/003455
Other languages
English (en)
Korean (ko)
Inventor
정근호
김세민
이종하
유상욱
최용준
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Lululab Inc
Original Assignee
Lululab Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Lululab Inc filed Critical Lululab Inc
Publication of WO2024196128A1 publication Critical patent/WO2024196128A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/145—Measuring characteristics of blood in vivo, e.g. gas concentration or pH-value ; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid or cerebral tissue
    • A61B5/1455—Measuring characteristics of blood in vivo, e.g. gas concentration or pH-value ; Measuring characteristics of body fluids or tissues, e.g. interstitial fluid or cerebral tissue using optical sensors, e.g. spectral photometrical oximeters
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems

Definitions

  • the present invention relates to a deep learning-based skin analysis technology, and more specifically, to a method and device for optimizing a skin analysis model using an image reconstructed through an optical approach to reproduce light reflection characteristics.
  • a method for analyzing complex skin conditions such as melanin and hemoglobin, a method is used in which skin images are taken in a state where ambient light is blocked to facilitate accurate measurement of skin conditions in clinical settings, and the taken skin images are analyzed using professional analysis tools.
  • This analysis method requires expensive special filming equipment and analysis tools dependent on the conditions of the special filming equipment to analyze skin images based on the interaction between light and skin tissue and provide reliable results on skin components. Therefore, it is difficult to obtain reliable analysis results when this analysis method is applied in a camera and light source environment that is generally widespread.
  • deep learning-based skin analysis models have the advantage of not being restricted by relatively expensive special filming equipment, but they have the same limitations as existing analysis methods in that they must precisely film and analyze skin analysis results in advance as a learning data set.
  • Non-patent Document 1 Prahl, Scott. “Optical absorption of hemoglobin. 1999.” URL: https:/omlc.org/spectra/hemoglobin/ ( uncleart am 04. 09. 2018) (2014).
  • Non-patent Document 2 Jacques, Steven. “Optical absorption of melanin.” Oregon Medical Laser Center, URI: https:/omlc.org/spectra/melanin/extcoeff.html (1988).
  • Non-patent Document 3 Krishnaswamy, Aravind, and Gladimir VG Baranoski. “A biophysically-based spectral model of light interaction with human skin.” (2004).
  • Non-patent Document 4 Preece, Stephen J., and Ela Claridge. “Spectral filter optimization for the recovery of parameters which describe human skin.” (2004).
  • Non-patent Document 5 Dawson, J. B., et al. “A theoretical and experimental study of light absorption and scattering by in vivo skin.”, (1980).
  • Non-patent Document 6 Non-patent Document 6 N. Ohta, "The Basis of Color Reproduction Engineering” (Japanese), Corona-sha Co of Japan, published (1997)
  • the purpose of the present invention to solve the above problems is to provide a method and device for optimizing a skin analysis model using an image of light reflection characteristics reconstructed through an optical approach.
  • One aspect of the present invention for achieving the above object provides a computing device for performing a method for optimizing a skin analysis model using an image of light reflection characteristics reconstructed through an optical approach.
  • the computing device includes at least one processor; and instructions for directing the at least one processor to perform at least one operation.
  • the at least one operation includes: a step of capturing a user's face area to obtain an initial image; a step of preprocessing the initial image to generate an input image; a step of inputting the generated input image to a skin analysis model and obtaining a skin feature map as an output of the skin analysis model; a step of generating a light reflection characteristic reproduction image from the obtained skin feature map; and a step of calculating a loss function using the generated light reflection characteristic reproduction image and the input image, and performing supervised learning of the skin analysis model such that an output value of the calculated loss function is minimized.
  • the above skin feature map includes a hemoglobin feature map predicting the hemoglobin distribution constituting the user's facial skin; a melanin feature map predicting the melanin distribution constituting the user's facial skin; a shadow feature map predicting the shadow distribution appearing on the user's facial skin; and a reflection feature map predicting the surface morphological characteristic distribution due to reflected light from the user's facial surface.
  • the step of generating the input image includes: a step of generating a skin detection mask from the initial image; a step of dividing the initial image into preset sizes to obtain a plurality of first patches, and a step of dividing the skin detection mask into preset sizes to obtain a plurality of second patches; a step of generating a plurality of third patches in which the first patches and the second patches correspond 1:1 with patches at the same positions in the corresponding images, and the pixel values of the corresponding positions are multiplied by each of the two corresponding patches to generate the pixel values of the corresponding positions; and a step of generating the input image by arranging the generated plurality of third patches at the same positions as the initial image and connecting them to each other.
  • the step of generating the above-described light reflection characteristic reproduction image includes: a step of generating skin light reflection spectrum data using the melanin feature map and the hemoglobin feature map; a step of generating a camera color space image by reflecting light source spectrum information and camera characteristic information corresponding to a light source used when photographing the face region to the skin light reflection spectrum data; and a step of generating the light reflection characteristic reproduction image by performing color correction based on the camera color space image.
  • the above skin light reflectance spectrum data includes light reflectance, which represents the ratio of light reflected by the entire facial skin according to the wavelength of light, based on the light absorbance and light scattering of human facial skin calculated in the visible light wavelength range.
  • the above light reflectance is determined by determining the light absorbency, which indicates the degree of light absorption according to the wavelength of light in the human skin layer, and the light scattering, which indicates the degree of light scattering according to the wavelength of light in the skin layer, and using the determined light absorbency and light scattering.
  • Another aspect of the present invention for achieving the above object provides a method for optimizing a skin analysis model using an image of light reflection characteristics reconstructed through an optical approach.
  • the method comprises the steps of: capturing a user's face area to obtain an initial image; preprocessing the initial image to generate an input image; inputting the generated input image into a skin analysis model and obtaining a skin feature map as an output of the skin analysis model; generating a light reflection characteristic reproduction image from the obtained skin feature map; and calculating a loss function using the generated light reflection characteristic reproduction image and the input image, and supervising learning the skin analysis model so that an output value of the calculated loss function is minimized.
  • the skin light reflection characteristics are reproduced to generate an image reproducing the light reflection characteristics, and the generated image reproducing the light reflection characteristics is utilized for supervised learning of the skin analysis model. Therefore, there is an advantage in that model optimization is possible without securing learning data using an expensive skin diagnosis device after the initial model configuration of the skin analysis model.
  • model optimization is performed by reflecting optical characteristics according to the light source and skin, there is an advantage in that the performance of the skin analysis model can be improved even without expensive professional photography equipment.
  • Figure 1 is a conceptual diagram illustrating the basic configuration of a kiosk device according to one embodiment.
  • FIG. 2 is a drawing for explaining the operation of a kiosk device for skin analysis according to one embodiment.
  • FIG. 3 is a diagram exemplarily showing skin analysis results displayed by a kiosk device according to one embodiment.
  • FIG. 4 is a conceptual diagram illustrating a process of performing a method for optimizing a skin analysis model using an image of light reflection characteristics reconstructed through an optical approach according to one embodiment.
  • FIG. 5 is a conceptual diagram illustrating a preprocessing process for generating an input image from an initial image according to one embodiment.
  • FIG. 6 is a drawing for explaining a concept for generating an image reproducing light reflection characteristics according to one embodiment.
  • Figure 7 is a representative flowchart for explaining a process for generating an image reproducing light reflection characteristics according to one embodiment.
  • FIG. 8 is a diagram comparing an image obtained as an output of an optimized skin analysis model using a method for optimizing a skin analysis model by using an image of light reflection characteristic reproduction reconstructed through an optical approach according to one embodiment with an image captured using professional diagnostic equipment.
  • FIG. 9 is a diagram exemplarily showing the hardware configuration of a device that performs a method for optimizing a skin analysis model using an image of light reflection characteristics reconstructed through an optical approach according to one embodiment.
  • first, second, A, B, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are only used to distinguish one component from another.
  • the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.
  • the term and/or includes any combination of a plurality of related described items or any item among a plurality of related described items.
  • Figure 1 is a conceptual diagram illustrating the basic configuration of a kiosk device according to one embodiment.
  • a kiosk device 1000
  • various skin analysis methods and operations related thereto described throughout the specification of the present invention may be performed by a kiosk device (1000), but are not limited thereto, and it should be interpreted that they may be implemented by various types of computing devices capable of computational processing, including a processor and memory, in addition to the kiosk device (1000).
  • the computing device may be a communication-capable desktop computer, a laptop computer, a notebook, a smart phone, a tablet PC, a mobile phone, a smart watch, smart glasses, an e-book reader, a portable multimedia player (PMP), a portable game console, a navigation device, a digital camera, a digital multimedia broadcasting (DMB) player, a digital audio recorder, a digital audio player, a digital video recorder, a digital video player, a PDA (Personal Digital Assistant), etc.
  • PMP portable multimedia player
  • DMB digital multimedia broadcasting
  • a kiosk device (1000) may include a shooting module (300) for shooting a facial image of a user (2000), a display module (200) for analyzing the facial image shot by the shooting module (300) and displaying a skin analysis result corresponding to the facial image to the user (2000), a body frame (100) on which the display module (200) and the shooting module (300) are mounted, a bottom support (500) for supporting a load of the body frame (100) from the ground, and a connecting leg (400) for physically connecting the bottom support (500) and the body frame (100) to each other to transmit the load of the body frame (100) to the bottom support (500).
  • a shooting module (300) for shooting a facial image of a user (2000)
  • a display module (200) for analyzing the facial image shot by the shooting module (300) and displaying a skin analysis result corresponding to the facial image to the user (2000)
  • a body frame (100) on which the display module (200) and the shooting module (300) are mounted a
  • the shooting module (300) is positioned at the top center of the body frame (100) and can capture an image of the user's (2000) face.
  • the display module (200) can display a shooting guide for skin analysis to the user (2000) and display a facial image of the user (2000) captured in real time by the shooting module (300) according to the shooting guide. That is, the user (200) can adjust the shooting position and align the shooting guide while checking the image of his/her face displayed in real time on the display module (200).
  • the display module (200) can display skin analysis results for a facial image to a user (2000).
  • the display module (200) may be a flat display device, but may also be a flexible display device.
  • a flat display device it may be a liquid crystal display (LCD) or an electroluminescence display (ELD).
  • the electroluminescence display device may be divided into an inorganic light emitting display device and an organic light emitting display device depending on the material of the light emitting layer.
  • An organic light emitting display device of an active matrix type may include an organic light emitting diode (OLED) that emits light by itself.
  • the display module (200) may include a display panel composed of a plurality of pixels, a display panel driver for writing pixel values to pixels of the display panel, a power supply unit for supplying power to the pixels and the display panel driver, etc.
  • the display module (200) may further include a touch screen for receiving input from a user (2000).
  • the touch screen may be arranged on top of the display panel and overlapped with the display panel.
  • the touch screen may be a capacitive or pressure-sensitive touch screen.
  • the kiosk device (1000) may have an input interface device for receiving user input separately coupled to the body frame (100).
  • the input interface device may include at least one mechanical button.
  • the connecting leg portion (400) may be composed of a pair of bar-shaped legs arranged in parallel with each other. At this time, the connecting leg portion (400) may further include a leg receiving portion (410) including two bar-shaped holes formed perpendicularly to the ground and penetrating inwardly to receive the bar-shaped legs.
  • the leg receiving portion (410) can support adjustment of the length of the connecting leg portion (400) by receiving at least a portion of a pair of bar-shaped legs and fixing the pair of bar-shaped legs at the received height position.
  • a pair of bar-shaped legs are not physically connected to the bottom support (500), but are fixed at a specific height by the leg receiving portion (410), and the leg receiving portion (410) can be physically connected to the bottom support (500).
  • the leg receiving portion (410) may further include a leg fixing portion (not shown) coupled to penetrate one side (e.g., one side of the back).
  • the leg fixing portion may penetrate one side of the leg receiving portion (410) and press the pair of bar-shaped legs accommodated in the leg receiving portion (410) in a direction perpendicular to the ground, thereby fixing the pair of bar-shaped legs to a specific height of the leg receiving portion (410).
  • FIG. 2 is a drawing for explaining the operation of a kiosk device for skin analysis according to one embodiment.
  • the kiosk device (1000) may further include a control unit (hereinafter, the control unit is interpreted as including at least one of a processor (described below) or a central processing unit (CPU), a graphic processing unit (GPU) included in the kiosk device (1000), and the control unit may be interpreted as being replaced with the aforementioned processor, etc.) that controls a shooting module to acquire a facial image, analyzes the acquired facial image to generate skin analysis result data, and transmits the generated skin analysis result data to a display module.
  • a control unit hereinafter, the control unit is interpreted as including at least one of a processor (described below) or a central processing unit (CPU), a graphic processing unit (GPU) included in the kiosk device (1000), and the control unit may be interpreted as being replaced with the aforementioned processor, etc.
  • control unit of the kiosk device (1000) when the control unit of the kiosk device (1000) receives an input for initiating skin analysis from a user (2000) through the display module (200), it can enter the photographing preparation stage (PG1) for skin analysis.
  • the control unit of the kiosk device (1000) can control the display module (200) to display a boundary area (FACEGD) and a shooting guide message to the user (2000).
  • FACEGD boundary area
  • the boundary area (FACEGD) may be formed as an oval with a vertical length longer than a horizontal width, and a central area corresponding to a preset size of the boundary area (FACEGD) may be displayed within the boundary area (FACEGD) in an image captured in real time by the shooting module (300).
  • the shooting guide message (Please properly align your face and/or temporarily remove your hat and glasses) may be displayed overlapping the video being shot in real time in the boundary area (FACEGD) or may be displayed at a location adjacent to the boundary area (FACEGD) (e.g., at the top of the boundary area).
  • the control unit of the kiosk device (1000) can identify the face of the user (2000) in the image displayed in the boundary area (FACEGD) and optionally enter the auxiliary shooting preparation stage (PG2) based on the identification result.
  • the control unit of the kiosk device (1000) may enter the auxiliary shooting preparation stage (PG2) and display an additional guide message through the display module (200).
  • the control unit of the kiosk device (1000) can optionally control parameters (e.g., aperture value, gamma, color tone, focal ratio, shutter speed, focus position, etc.) of the camera included in the shooting module (300) in the auxiliary shooting preparation stage (PG2).
  • parameters e.g., aperture value, gamma, color tone, focal ratio, shutter speed, focus position, etc.
  • the control unit of the kiosk device (1000) can enter the general shooting stage (PG3) after the shooting preparation stage (PG1) or the auxiliary shooting preparation stage (PG2).
  • the control unit of the kiosk device (1000) controls the shooting module (300) to the first mode and can acquire the first face image of the user (2000) identified in the boundary area (FACEGD).
  • the first mode may be referred to as a general light shooting mode, and in the first mode, the control unit of the kiosk device (1000) may control the 1-1 LED (Light Emitting Diode) group included in the shooting module (300) to emit light.
  • the 1-1 LED Light Emitting Diode
  • the control unit of the kiosk device (1000) controls the shooting module (300) to the second mode and can acquire a second face image of the user (2000) identified in the boundary area (FACEGD).
  • the second mode may be referred to as a polarization shooting mode
  • the control unit of the kiosk device (1000) may control the first-second LED groups included in the shooting module (300) to emit light.
  • each of the 1-1 LED group and the 1-2 LED group may emit light of the same color in the visible light range, but may be arranged in parallel with each other in the vertical direction within the photographing module (300).
  • each of the 1-1 LED group and the 1-2 LED group may be composed of LEDs that emit white color having a color temperature of 4000k (Kelvin) and a color rendering index (CRI) of 80.
  • the control unit of the kiosk device (1000) can enter the UV (ultra violet) shooting stage (PG4).
  • the control unit of the kiosk device (1000) may display a warning message through the display module (300).
  • the warning message may be a message to close the eyes to prevent eye damage due to ultraviolet rays.
  • the warning message may be displayed not only through the display module (300), but may also be output as a voice through a speaker.
  • the kiosk device (1000) may further include a speaker (not shown).
  • the control unit of the kiosk device (1000) can identify the face of the user (2000) from the image displayed in the boundary area (FACEGD) and identify the eye-closing state of the user (2000) from the identified face.
  • the control unit of the kiosk device (1000) controls the shooting module (300) to the third mode and can acquire a third face image of the identified user (2000) in the boundary area (FACEGD).
  • the third mode may be referred to as a UV shooting mode, and in the third mode, the control unit of the kiosk device (1000) may control the second LED group included in the shooting module (300) to emit light.
  • the second group of LEDs may consist of LEDs that emit light in the ultraviolet range with a wavelength of 365 nm.
  • the control unit of the kiosk device (1000) can analyze the skin of the user (2000) using the first to third facial images and generate skin analysis result data.
  • control unit of the kiosk device (1000) can perform skin analysis based on AI (Artificial Intelligence) using the first to third facial images.
  • AI Artificial Intelligence
  • control unit of the kiosk device (1000) can determine the pores and wrinkles of the user using the first facial image.
  • the control unit of the kiosk device (1000) can determine at least one of redness, pigmentation, and skin trouble of the user using the second facial image.
  • the control unit of the kiosk device (10000) can determine the user's sebum by using a third facial image.
  • FIG. 3 is a diagram exemplarily showing skin analysis results displayed by a kiosk device according to one embodiment.
  • control unit of the kiosk device (1000) can determine at least one of wrinkles, pigmentation, redness, pores, sebum, dark circles, and skin trouble of the user (2000) as skin analysis result data.
  • control unit of the kiosk device (1000) can determine an evaluation score for each item corresponding to at least one of wrinkles, pigmentation, redness, pores, sebum, dark circles, and skin trouble, and display the determined evaluation score to the user through the display module (300).
  • the control unit of the kiosk device (1000) can determine at least one of the user's (2000) skin age and skin composite score based on the skin analysis result data.
  • control unit of the kiosk device (1000) can compare the item-by-item evaluation scores according to the skin analysis result data with the person's age and item-by-item evaluation scores (or the average value of the evaluation scores corresponding to the age and the item) collected and stored in advance, select the age having the most similar item-by-item evaluation scores, and determine the selected age as the skin age.
  • control unit of the kiosk device (1000) may add up the evaluation scores for each item according to the skin analysis result data (or may add up the evaluation scores for each item by applying a weight to them) and determine a comprehensive skin score based on the added evaluation scores. At this time, the control unit of the kiosk device (1000) may also determine a comprehensive skin score by applying a correction value according to skin age to the added evaluation scores.
  • a user face area corresponding to the item may be displayed through the display module (300).
  • the item-by-item evaluation scores may be set to the same score distribution range (e.g., between 0 and 10 points) for each item, but are not limited thereto.
  • control unit of the kiosk device (1000) may select a cosmetic suitable for the user from among candidate cosmetics included in a cosmetic list stored in the internal storage based on an evaluation score for each item corresponding to at least one of wrinkles, pigmentation, redness, pores, sebum, dark circles, and skin trouble, and display the selected cosmetic as a recommended cosmetic to the user through the display module (300).
  • control unit of the kiosk device (1000) may select items having evaluation scores that are lower than (or below) a preset threshold score from among the evaluation scores for each item, select at least one cosmetic ingredient (or raw material) corresponding to each of the selected items, select a cosmetic containing the selected cosmetic ingredient (or raw material) from among the candidate cosmetics, and determine the selected cosmetic as a recommended cosmetic.
  • the control unit of the kiosk device (1000) can receive an initial image of a user, generate an input image from the received initial image, input the generated input image into a skin analysis model, and generate skin analysis result data based on the output of the input skin analysis model.
  • the control unit of the kiosk device (1000) may correspond to a processor of various computing devices described throughout the specification, or may be a central processing unit (CPU), and may be interpreted as a replacement for the corresponding hardware unit.
  • CPU central processing unit
  • FIG. 4 is a conceptual diagram illustrating a process of performing a method for optimizing a skin analysis model using an image of light reflection characteristics reconstructed through an optical approach according to one embodiment.
  • a method for optimizing a skin analysis model using an image of a light reflection characteristic reproduction reconstructed through an optical approach may include the steps of: receiving an initial image; generating an input image by preprocessing the received initial image; inputting the generated input image into a skin analysis model, and obtaining a skin feature map as an output of the skin analysis model; generating an image of a light reflection characteristic reproduction from the acquired skin feature map; and calculating a loss function using the generated image of the light reflection characteristic reproduction and the input image, and supervising learning the skin analysis model so that an output value of the calculated loss function is minimized.
  • the method for optimizing a skin analysis model using an image of light reflection characteristics reconstructed through an optical approach can be performed by the control unit of the kiosk device (1000) described above or by various computing devices described throughout the specification.
  • the initial image is a facial image in which the user's face area is captured.
  • the image may be captured by capturing the user's face area using a shooting module (300) built into the kiosk device (1000), or may be captured by inputting an image captured using an external shooting device.
  • the initial image may be a video of the user's eyes, nostrils, mouth, eyebrows, hair, and external background other than the face, captured together with the facial skin.
  • the input image can be generated by removing the remaining areas (e.g., eyes, nostrils, mouth, eyebrows, hair, external background other than the face, etc.) from the initial image except for the user's facial skin.
  • the remaining areas e.g., eyes, nostrils, mouth, eyebrows, hair, external background other than the face, etc.
  • the skin analysis model can receive an input image with all areas except facial skin removed, and output a skin feature map corresponding to the input image.
  • a skin analysis model is an artificial neural network based on a convolutional neural network (CNN), and the artificial neural network based on a convolutional neural network (CNN) may include a convolutional layer that receives image frames of a preset size as input images and outputs a feature map, an activation layer that determines whether to activate the output using an activation function for the extracted feature map, a pooling layer that performs sampling on the output according to the activation layer, a fully connected layer that performs classification according to class, and an output layer that finally outputs the output according to the fully connected layer.
  • CNN convolutional neural network
  • CNN convolutional neural network
  • a convolutional layer may be a layer that extracts features of input data by convolving an input image and a filter input to the layer.
  • the filter is a function that detects a characteristic part of the input image, and may be a function that is generally expressed as a matrix and determined by continuously learning by learning data.
  • the output image output after the convolution operation by the convolutional layer may be referred to as a feature map.
  • the interval value at which the convolution is performed may be referred to as a stride, and feature maps of different sizes may be extracted depending on the stride value.
  • the padding process may be a process of maintaining the size of the input image and the size of the feature map the same by adding a preset value (for example, 0 or 1) to the periphery of the generated feature map.
  • the convolutional layer according to one embodiment of the present invention may use a structure in which 3 ⁇ 3 convolutional layers are sequentially and repeatedly connected, but is not limited thereto.
  • the activation layer is a layer that determines whether to activate or not by changing the feature map into a nonlinear value according to the activation function.
  • the activation function that can be used is the sigmoid function, ReLU function, softmax function, LeakyReLU function, etc.
  • the softmax function can be a function that normalizes all input values to values between 0 and 1 and has the characteristic that the sum of the output values is always 1.
  • the pooling layer is a layer that selects features representing the feature map by performing subsampling or pooling on the feature map obtained as the output of the convolutional layer.
  • Max pooling which extracts the largest value for a certain area of the feature map
  • average pooling which extracts the average value
  • the pooling layer can perform max pooling of 2X2 size.
  • An artificial neural network based on a CNN may include multiple connection structures of the aforementioned convolutional layers, activation layers, and pooling layers.
  • an artificial neural network based on a CNN may be used, such as a CNN-based S-CNN (shallow convolutional neural network), YOLO (You Look Only Once), SSD (Single Shot MultiBox Detector), Faster R-CNN, ResNet, U-Net, etc.
  • a model with a somewhat simplified (or lightweight) hierarchical structure based on U-Net is used.
  • the skin feature map may include a hemoglobin feature map predicting a hemoglobin distribution constituting the user's facial skin, a melanin feature map predicting a melanin distribution constituting the user's facial skin, a shadow feature map predicting a shadow distribution appearing on the user's facial skin, and a reflection feature map predicting a surface morphological characteristic distribution due to reflected light from the user's facial surface.
  • the hemoglobin feature map and the melanin feature map are feature maps that represent the skin distribution of hemoglobin and melanin components among the components of the user's facial skin
  • the shading feature map and the reflection feature map are feature maps that represent the shading distribution and light reflection distribution of the face according to the user's face shape and light source.
  • the skin analysis model can be initially trained in advance so that it can output a hemoglobin feature map, a melanin feature map, a shadow feature map, and a reflection feature map, respectively.
  • a face image of a sample user is captured, and an input image is generated by performing preprocessing on the captured face image, and a hemoglobin training image, a melanin training image, a shadow training image, and a reflection training image representing the distribution of hemoglobin among skin components, the distribution of melanin among skin components, the shadow distribution of the skin due to a light source, and the reflection distribution due to a light source are acquired using specialized skin analysis equipment for the captured face image, and when the input image is input to the skin analysis model, the hemoglobin training image, the melanin training image, the shadow training image, and the reflection training image can be initially trained to output the answer as an answer.
  • a skin feature map including a hemoglobin feature map, a melanin feature map, a shading feature map, and a reflectance feature map is obtained as an output of the skin analysis model, an image reproducing light reflectance characteristics can be generated using the obtained skin feature map.
  • These images reproducing light reflection characteristics can be images that reproduce the input image by considering the spectral shape by the light source, the degree of skin reflection depending on the skin, and the color characteristics according to the sensor characteristics of the camera.
  • the loss function is calculated using the input image and the image reproducing the light reflection characteristics, and the skin analysis model can be optimized by supervised learning so that the result value of the calculated loss function is minimized.
  • the loss function may be mean square error (MSE) or cross entropy error.
  • the loss function can be defined by the following mathematical expression 1.
  • k is the number of pixels in the input image of the skin analysis model (or the image reproducing the light reflection characteristics)
  • di is the value of the ith pixel in the image reproducing the light reflection characteristics
  • pi is the value of the ith pixel in the input image.
  • the loss function can be defined by the following mathematical expression 2.
  • k is the number of pixels in the input image of the skin analysis model (or the image reproducing the light reflection characteristics)
  • di is the value of the ith pixel in the image reproducing the light reflection characteristics
  • pi is the value of the ith pixel in the input image.
  • FIG. 5 is a conceptual diagram illustrating a preprocessing process for generating an input image from an initial image according to one embodiment.
  • control unit of the kiosk device (1000) can generate a skin detection mask from an initial image in a preprocessing process.
  • a skin detection mask may be an image in which pixels corresponding to facial skin are marked with pre-specified pixel values (e.g., a white color value or a maximum pixel value of 255 in 8-bit terms) for an initial image of a user's face, and pixels corresponding to areas other than the facial skin are marked with other pre-specified pixel values (e.g., a black color value or a pixel value of 0).
  • pre-specified pixel values e.g., a white color value or a maximum pixel value of 255 in 8-bit terms
  • control unit of the kiosk device (1000) may generate a skin detection mask by extracting only the area corresponding to the skin from an initial image of a user's face using a pre-trained deep learning-based skin detection model.
  • the skin detection model may be pre-trained to output a skin detection mask that removes areas other than the facial skin from the face when an image of the face is input.
  • the skin detection model can be supervised by using images of the faces of sample users as input images, and using training images (answer sheets) in which pixels corresponding to facial skin in the images of the faces of sample users are marked with a first pixel value designated in advance (e.g., white color value), and pixels corresponding to the remaining areas excluding the facial skin are marked with a second pixel value designated in advance (e.g., black color value).
  • a first pixel value designated in advance e.g., white color value
  • a second pixel value designated in advance e.g., black color value
  • training images can be manually generated by a person, or collected by applying various other known object detection algorithms (e.g., an algorithm that detects parts of the face other than the skin, such as the eyes and nostrils).
  • object detection algorithms e.g., an algorithm that detects parts of the face other than the skin, such as the eyes and nostrils.
  • the control unit of the kiosk device (1000) can divide the skin detection mask and the initial image into a plurality of patches each having a preset size.
  • the control unit of the kiosk device (1000) can divide the initial image into a preset size (e.g., 256X256 pixel size) to obtain a plurality of first patches, and can divide the skin detection mask into a preset size (same size as the division size of the initial image, for example, 256X256 pixel size) to obtain a plurality of second patches.
  • control unit of the kiosk device (1000) can create a plurality of third patches by making the first patches and the second patches correspond 1:1 to each other in the same position within the corresponding image, and multiplying the pixel values of the same position for each of the two corresponding patches, and using the result as the pixel value of the corresponding position.
  • the control unit of the kiosk device (1000) can generate an input image by arranging a plurality of generated third patches in the same positions as the initial image and connecting them to each other.
  • the input image may be an image in which the remaining areas, excluding areas corresponding to the skin of the face, are marked with a pre-specified second pixel value (e.g., a black color value) by a skin detection mask.
  • a pre-specified second pixel value e.g., a black color value
  • FIG. 6 is a drawing for explaining a concept for generating an image reproducing light reflection characteristics according to one embodiment.
  • the characteristics of the light source e.g., spectrum according to light wavelength
  • the characteristics of the light source e.g., spectrum according to light wavelength
  • the degree of reflection from the surface of the skin of the face i.e., skin reflectivity
  • some of the light is scattered and absorbed within the skin, and some of the light is also scattered from the skin surface, which can affect the facial image.
  • the quantum efficiency which indicates the rate at which light transmitted to the camera lens is converted into a digital signal such as pixels, varies depending on the characteristics of the shooting module (camera), and this quantum efficiency can have a three-dimensional effect on the facial image.
  • a method is proposed to mathematically model the first to third optical factors that have an influence in the process of acquiring a facial image containing facial skin as a photographing result, and to use the modeling result to reproduce optical factors included in an input image of a skin analysis model from a skin characteristic map which is an output of the skin analysis model, thereby generating a light reflection characteristic reproduction image, and to optimize (or supervised learning) the skin analysis model so that the difference between the generated light reflection characteristic reproduction image and the input image is minimized.
  • Figure 7 is a representative flowchart for explaining a process for generating an image reproducing light reflection characteristics according to one embodiment.
  • the control unit of the kiosk device (1000) can first generate skin light reflection spectrum data using a melanin feature map and a hemoglobin feature map (S100).
  • the epidermis the outermost layer of the skin, absorbs light through melanin, and in the dermis, inside the epidermis, light is absorbed through hemoglobin and light is scattered through collagen fibers and elastic fibers.
  • the light absorption and light scattering in the skin layer can be calculated depending on the wavelength of light.
  • step S100 the control unit of the kiosk device (1000) can calculate the light absorbance according to the wavelength of light according to the following mathematical expression 3.
  • ⁇ a ( ⁇ ) is the light absorbance according to the wavelength ( ⁇ ) of light
  • i is a natural number greater than or equal to 1 and less than or equal to 2, where when i is 1, it refers to the melanin component of facial skin, and when i is 2, it refers to the hemoglobin component of facial skin
  • S i is the ratio (volume fraction) of the skin component (i) per unit volume of facial skin.
  • the ratio of the skin component per unit volume known on average for the facial skin of a person (or a specific race) can be applied, or the ratio of the skin component per unit volume measured experimentally can be applied.
  • the unit volume ratio of human melanin is 1.3% to 43%, and the unit volume ratio of human hemoglobin is 2% to 7%. It is preferable to apply a known ratio for a person having a similar race, place of origin, etc. to the user to be measured, and ⁇ i ( ⁇ ) is an extinction coefficient of the corresponding skin component (i) according to the wavelength ( ⁇ ) of light. The theoretically known extinction coefficient of the skin component can be applied, or a result value precisely measured using a spectrophotometer can be applied.
  • C i is a concentration of the corresponding skin component (i).
  • the concentration of melanin can be determined from the melanin feature map output from the skin analysis model, and the concentration of hemoglobin can be determined from the hemoglobin feature map. That is, the output of the skin analysis model can be substituted into the concentration of the skin component in Mathematical Formula 3 to calculate the light absorbance according to Mathematical Formula 3.
  • the public literature on the absorption coefficient of hemoglobin according to concentration and wavelength (Prahl, Scott. "Optical absorption of hemoglobin. 1999.” URl: https:/omlc.org/spectra/hemoglobin/ ( notedt am 04. 09. 2018) (2014).) can be referenced and applied, and the public literature on the absorption coefficient of melanin according to concentration and wavelength (Jacques, Steven. "Optical absorption of melanin.” Oregon Medical Laser Center, URI: https:/omlc.org/spectra/melanin/extcoeff.html (1988).) can be referenced and applied to mathematical expression 3.
  • the skin component (i) is divided into oxy-hemoglobin, deoxy-hemoglobin, eumelanin, and pheomelanin, and the mathematical expression 3 can be calculated by applying each variable according to mathematical expression 3 according to the divided skin component (i).
  • ICA Independent Component Analysis
  • PCA Principal Component Analysis
  • a skin analysis model generates the concentration (Ci) of skin components in the form of a two-dimensional feature map (melanin feature map, hemoglobin feature map), and based on this, generates an image reproducing light reflection characteristics to verify the accuracy of the feature map that can determine the concentration (Ci) of skin components, thereby approaching the optical forward problem, and ultimately proposing a method for improving the performance of a deep learning-based skin analysis model that receives a skin image as input and generates a skin component feature map (melanin feature map, hemoglobin feature map).
  • the concentration of skin components is theoretically calculated based on absolute values, but the relative distribution of values may change during the calculation and imaging processes, and correction using specialized equipment measurements may be required for precise diagnosis based on absolute values in the future.
  • control unit of the kiosk device (1000) can calculate the light scattering according to the wavelength of light according to the following mathematical expression 4.
  • ⁇ 0 is a preset reference wavelength
  • a is a known scattering attenuation coefficient at a preset reference wavelength ( ⁇ 0 )
  • fRay is a fraction of scattering events
  • b Mie is a scattering intensity in Mie scattering.
  • the front end which is inversely proportional to the fourth power of the wavelength ( ⁇ ) of light, may correspond to a part reflecting the characteristics according to Rayleigh scattering, and the rear end may correspond to a part reflecting the characteristics according to Mie scattering.
  • the light scattering can also be calculated by precisely measuring the Henyey-Greenstein scattering phase function according to the scattering angle formed by the incident light and the scattered light using a spectrometer, and the light scattering according to the wavelength of light known through other known precise measurement methods can also be applied.
  • step S100 the control unit of the kiosk device (1000) calculates the light absorption and light scattering according to mathematical expressions 3 and 4 within a wavelength range of visible light (e.g., 400 to 720 nm) at a predetermined wavelength interval (10 nm), thereby calculating the light absorption and light scattering of the skin for the entire wavelength range of visible light.
  • a wavelength range of visible light e.g. 400 to 720 nm
  • step S100 the control unit of the kiosk device (1000) can calculate the light reflectance (R), which represents the ratio of light reflected according to the wavelength of light on the entire facial skin, as skin light reflection spectrum data based on the light absorbance and light scattering of the skin calculated in the visible light wavelength range.
  • R the light reflectance
  • control unit of the kiosk device (1000) can calculate the light reflectance (R) according to the following mathematical expression 5 based on the light absorbance and light scattering of the skin.
  • R( ⁇ ) is the light reflectance according to the wavelength of light
  • T epidermis ( ⁇ ) is the light transmittance indicating the degree of light penetration (transmittance) in the epidermis according to the wavelength ( ⁇ ) of light
  • R dermis ( ⁇ ) is the light reflectance (or light reflectance) indicating the degree of light reflection in the dermis according to the wavelength ( ⁇ ) of light.
  • the light transmittance of the epidermis (T epidermis ( ⁇ )) and the light reflectance of the dermis (R dermis ( ⁇ )) can be calculated by mathematically calculating the diffusion equation or the radiative transfer equation under previously studied skin layer thickness conditions using the light absorbance and light scattering of the skin according to mathematical expressions 3 and 4, or by repeatedly performing a simulation in which a large number of photons are incident on the skin layer by applying a method such as Monte Carlo simulation.
  • the process of calculating the degree of light intensity attenuation using the modified Beer-Lambert law and the process of calculating the degree of light reflection and refraction according to the difference in refractive index using the Fresnel equation may be included.
  • the skin layer thickness condition varies depending on the race and site, and may be, for example, 0.005 cm to 0.035 cm for the epidermis and 0.1 cm to 0.4 cm for the dermis.
  • the optical reflectance (R) according to mathematical expression 5 is the optical reflectance representing the entire skin layer including the epidermis and dermis, and is a simple form of Kubelka-Munk Theory that applies that light passing through the epidermis is reflected in the opposite direction by the dermis and passes through the epidermis to be detected from the outside.
  • mathematical expression 5 may be modified and applied to consider the case where it is directly reflected and detected from the epidermis or reflected multiple times from the epidermis and dermis to be detected from the outside.
  • the optical characteristics of the epidermis and dermis may be modeled and Monte Carlo simulation may be applied to directly measure the optical reflectance of the entire skin layer, thereby replacing the optical reflectance (R) according to mathematical expression 5.
  • Monte Carlo simulation the simulation may be repeated under various optical absorbance and optical scattering conditions, and the values that were not simulated may be inferred using interpolation, thereby including a process of tabulating the optical reflectance of the skin layer for the entire range of optical absorbance and optical scattering conditions.
  • the skin reflectance can be obtained by applying the light absorbance and light scattering according to Equations 3 and 4 to various published literatures (Example 1. Krishnaswamy, Aravind, and Gladimir V. G. Baranoski. "A biophysically-based spectral model of light interaction with human skin.” 2004., Example 2. Preece, Stephen J., and Ela Claridge. "Spectral filter optimization for the recovery of parameters which describe human skin.” 2004., Example 3. Dawson, J. B., et al. "A theoretical and experimental study of light absorption and scattering by in vivo skin.”, 1980) that calculate the skin reflectance according to Equation 5, and the skin reflectance obtained in this way can be used instead of the light reflectance according to Equation 5.
  • control unit of the kiosk device (1000) can generate a camera color space image by reflecting the camera characteristic information into the skin light reflection spectrum data (S110).
  • the camera characteristic information may include the spectral power distribution (SPD) according to the wavelength of the light source that captures the facial image and the quantum efficiency (QE) of the shooting module (camera) that captures the facial image.
  • SPD spectral power distribution
  • QE quantum efficiency
  • the spectral power distribution can be experimentally obtained by measuring the light source used to capture the face image using a spectrometer.
  • the quantum efficiency can be obtained through a known specification sheet for the corresponding photographing module or by using other known measurement methods.
  • step S110 the control unit of the kiosk device (1000) can generate a camera color space image for each color channel (R, G, B) by integrating the result of applying the camera characteristic information to the skin light reflection spectrum data over the visible light wavelength range.
  • step S110 the control unit of the kiosk device (1000) can generate a camera color space image for each color channel according to the following mathematical expression 6.
  • Im is a pixel value according to the color channel (m)
  • m is a symbol indicating the color channel (r, g, b), for example, when m is r, it means the red channel, when m is g, it means the green channel, and when m is b, it means the blue channel.
  • ⁇ max and ⁇ min are the maximum and minimum wavelengths in the visible light range
  • L( ⁇ ) is the spectral power distribution (SPD) according to the wavelength of the light source that captures the facial image
  • R( ⁇ ) is the light reflectance of the entire skin calculated according to the above mathematical expression 5
  • C m ( ⁇ ) is the quantum efficiency (QE) according to the color channel (m) of the shooting module (camera) that captures the facial image.
  • the light entering the lens of the shooting module may include not only light reflected from the skin but also light scattered, and therefore, when considering scattering inside the skin, the camera color space image for each color channel in mathematical expression 6 may be produced according to mathematical expression 7 below.
  • Mshading( ⁇ ) is a shading feature map obtained as an output of the skin analysis model
  • Mspecular( ⁇ ) is a reflection feature map obtained as an output of the skin analysis model.
  • I d(m) ( ⁇ ) models the process in which light scattered inside the skin and exits the skin is detected by the camera sensor, and is formulated by reflecting the influence of the spectral characteristics (SPD) of the light source, the skin light reflectance representing the optical characteristics inside the skin, and the shading feature map generated by the facial structure and the irradiation direction of the external light source that are not related to the components inside the skin.
  • SPD spectral characteristics
  • I s(m) ( ⁇ ) represents the process in which light is reflected from the skin surface and detected by the camera sensor, and is formulated by applying the reflection feature map to the spectral characteristics (SPD) of the light source since it does not optically interact with the components inside the skin.
  • SPD spectral characteristics
  • step S110 the control unit of the kiosk device (1000) can generate a camera color space image by combining the camera color space images (I r , I g , I b ) generated by each color channel according to the following mathematical expression 6 or mathematical expression 7 into one.
  • the camera color space image corresponds to an image in which color expression according to the specification sheet of the shooting module (camera) is reflected. It may be desirable to convert this to a common color space so as not to be dependent on the color expression characteristics of the shooting module.
  • control unit of the kiosk device (1000) can perform color correction based on the camera color space image to generate an image reproducing light reflection characteristics (S120).
  • color correction of the control unit of the kiosk device (1000) includes an operation of generating an image reproducing light reflection characteristics obtained by converting a camera color space image into a common color space image.
  • the control unit of the kiosk device (1000) can adjust the white balance in the camera color space image.
  • the white balance adjustment can be a step of performing normalization on the camera color space image using the color value of light directly input into the shooting module by the light source used for shooting the image.
  • the control unit of the kiosk device (1000) can convert the camera color space image on which white balance adjustment has been performed into a common color space image using a virtual color checker.
  • the virtual color checker means the light reflectance spectrum of each patch of the color checker, and various externally disclosed color checker APIs based on a known paper (N. Ohta, The Basis of Color Reproduction Engineering (Japanese), Corona-sha Co of Japan, published 1997) can be used, or the virtual color checker can be obtained by calculating the ratio of the reflectance spectrum measured experimentally using a spectrophotometer and the light source spectral power distribution.
  • FIG. 8 is a diagram comparing an image obtained as an output of an optimized skin analysis model using a method for optimizing a skin analysis model by using an image of light reflection characteristic reproduction reconstructed through an optical approach according to one embodiment with an image captured using professional diagnostic equipment.
  • the skin analysis model is optimized to minimize deviations in light reflection characteristics according to the shooting environment (shooting light source, shooting module, shooting location, shooting time, etc.), so there is an advantage of being able to obtain high-precision skin analysis results even without using expensive diagnostic equipment.
  • FIG. 9 is a diagram exemplarily showing the hardware configuration of a device that performs a method for optimizing a skin analysis model using an image of light reflection characteristics reconstructed through an optical approach according to one embodiment.
  • a method for optimizing a skin analysis model using an image of a reconstructed optical reflection characteristic through an optical approach may be performed by a kiosk device (1000) as described above or by other various forms of computing devices (100).
  • a kiosk device (1000) or other various types of computing devices (100) may include at least one processor (110) and a memory (120) that stores instructions that instruct the at least one processor (110) to perform at least one operation.
  • At least one processor (110) may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed.
  • CPU central processing unit
  • GPU graphics processing unit
  • dedicated processor on which methods according to embodiments of the present invention are performed.
  • the memory (120) may be composed of at least one of a volatile storage medium and a nonvolatile storage medium.
  • the memory (120) may be one of a read only memory (ROM) and a random access memory (RAM).
  • a kiosk device (1000) or other various types of computing devices may further include a storage device (160) that stores initial data, preprocessing data, intermediate processing data, temporary data, output data, etc. used in the process of performing at least one of the above operations.
  • a storage device 160 that stores initial data, preprocessing data, intermediate processing data, temporary data, output data, etc. used in the process of performing at least one of the above operations.
  • the storage device (160) may be a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or various memory cards (e.g., a micro SD card).
  • HDD hard disk drive
  • SSD solid state drive
  • various memory cards e.g., a micro SD card
  • the kiosk device (1000) or other various forms of computing devices may include a transceiver (130) that performs communication via a wireless network.
  • the kiosk device (1000) or other various forms of computing devices may further include an input interface device (140), an output interface device (150), a storage device (160), etc.
  • Each component included in the kiosk device (1000) or other various forms of computing devices may be connected by a bus (170) and communicate with each other.
  • the methods according to the present invention may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium.
  • the computer-readable medium may include program commands, data files, data structures, etc., alone or in combination.
  • the program commands recorded on the computer-readable medium may be those specially designed and configured for the present invention or may be known and available to those skilled in the art of computer software.
  • Examples of computer-readable media may include hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like.
  • Examples of program instructions may include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, and the like.
  • the above-described hardware devices may be configured to operate with at least one software module to perform the operations of the present invention, and vice versa.
  • the above-described method or device may be implemented by combining all or part of its configuration or function, or may be implemented separately.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Public Health (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Biomedical Technology (AREA)
  • Pathology (AREA)
  • Primary Health Care (AREA)
  • Epidemiology (AREA)
  • Surgery (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biophysics (AREA)
  • Veterinary Medicine (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Optics & Photonics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Dermatology (AREA)

Abstract

L'invention concerne un procédé et un dispositif pour optimiser un modèle d'analyse de peau à l'aide d'une image de reproduction de caractéristique de réflexion de la lumière reconstruite par une approche optique. Un dispositif informatique comprend au moins un processeur et des instructions ordonnant audit processeur d'effectuer au moins une opération. Ladite opération comprend les étapes de : acquisition d'une image initiale par imagerie d'une région faciale de l'utilisateur ; génération d'une image d'entrée par prétraitement de l'image initiale ; entrée de l'image d'entrée générée dans un modèle d'analyse de peau et acquisition d'une carte de caractéristique de peau à l'aide d'une sortie du modèle d'analyse de peau ; génération d'une image de reproduction de caractéristique de réflexion de la lumière à partir de la carte de caractéristique de peau acquise ; et l'utilisation de l'image d'entrée et de l'image de reproduction de caractéristique de réflexion de la lumière générée pour calculer une fonction de perte et effectuer un entraînement supervisé du modèle d'analyse de peau pour réduire au minimum les valeurs de sortie de la fonction de perte calculée.
PCT/KR2024/003455 2023-03-20 2024-03-20 Procédé et dispositif pour optimiser un modèle d'analyse de peau à l'aide d'une image de reproduction de caractéristique de réflexion de la lumière reconstruite par approche optique Ceased WO2024196128A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
KR20230035793 2023-03-20
KR10-2023-0035793 2023-03-20
KR10-2023-0087015 2023-07-05
KR1020230087015A KR102648659B1 (ko) 2023-03-20 2023-07-05 광학 접근법을 통해 재구성한 광반사 특성 재현 영상을 이용하여 피부 분석 모델을 최적화하기 위한 방법 및 장치

Publications (1)

Publication Number Publication Date
WO2024196128A1 true WO2024196128A1 (fr) 2024-09-26

Family

ID=90474029

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2024/003455 Ceased WO2024196128A1 (fr) 2023-03-20 2024-03-20 Procédé et dispositif pour optimiser un modèle d'analyse de peau à l'aide d'une image de reproduction de caractéristique de réflexion de la lumière reconstruite par approche optique

Country Status (2)

Country Link
KR (1) KR102648659B1 (fr)
WO (1) WO2024196128A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119745378A (zh) * 2025-03-06 2025-04-04 湖南安瑜健康科技有限公司 基于预测模型的血脂监测方法

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102648659B1 (ko) * 2023-03-20 2024-03-18 주식회사 룰루랩 광학 접근법을 통해 재구성한 광반사 특성 재현 영상을 이용하여 피부 분석 모델을 최적화하기 위한 방법 및 장치

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20170018666A (ko) * 2015-08-10 2017-02-20 삼성전자주식회사 얼굴 인식 방법 및 장치
KR20220046186A (ko) * 2020-10-07 2022-04-14 삼성전자주식회사 전자 장치 및 전자 장치에서 피부 형광 측정 방법
KR20220113298A (ko) * 2021-02-05 2022-08-12 성균관대학교산학협력단 하이퍼 스펙트럴 카메라를 이용한 비접촉식 바이탈 사인 예측 방법, 비접촉식 바이탈 사인 예측 장치, 예측 방법을 수행하기 위한 컴퓨터 프로그램 및 컴퓨터 판독가능한 기록 매체
KR20220128015A (ko) * 2021-03-12 2022-09-20 주식회사 매직내니 피부 분석 시스템
KR102648659B1 (ko) * 2023-03-20 2024-03-18 주식회사 룰루랩 광학 접근법을 통해 재구성한 광반사 특성 재현 영상을 이용하여 피부 분석 모델을 최적화하기 위한 방법 및 장치

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20170018666A (ko) * 2015-08-10 2017-02-20 삼성전자주식회사 얼굴 인식 방법 및 장치
KR20220046186A (ko) * 2020-10-07 2022-04-14 삼성전자주식회사 전자 장치 및 전자 장치에서 피부 형광 측정 방법
KR20220113298A (ko) * 2021-02-05 2022-08-12 성균관대학교산학협력단 하이퍼 스펙트럴 카메라를 이용한 비접촉식 바이탈 사인 예측 방법, 비접촉식 바이탈 사인 예측 장치, 예측 방법을 수행하기 위한 컴퓨터 프로그램 및 컴퓨터 판독가능한 기록 매체
KR20220128015A (ko) * 2021-03-12 2022-09-20 주식회사 매직내니 피부 분석 시스템
KR102648659B1 (ko) * 2023-03-20 2024-03-18 주식회사 룰루랩 광학 접근법을 통해 재구성한 광반사 특성 재현 영상을 이용하여 피부 분석 모델을 최적화하기 위한 방법 및 장치

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119745378A (zh) * 2025-03-06 2025-04-04 湖南安瑜健康科技有限公司 基于预测模型的血脂监测方法

Also Published As

Publication number Publication date
KR102648659B1 (ko) 2024-03-18

Similar Documents

Publication Publication Date Title
WO2024196128A1 (fr) Procédé et dispositif pour optimiser un modèle d'analyse de peau à l'aide d'une image de reproduction de caractéristique de réflexion de la lumière reconstruite par approche optique
WO2016159523A1 (fr) Procédé d'acquisition d'informations biométriques et dispositif associé
KR102146561B1 (ko) 모바일 초분광 카메라 시스템 및 모바일 초분광 카메라 시스템을 이용한 인간 피부 모니터링
US20040125996A1 (en) Skin diagnostic imaging method and apparatus
CA2690952C (fr) Systeme, methode et programme informatique permettant l'analyse des anomalies cutanees du visage
WO2016099099A1 (fr) Dispositif de capture d'images et dispositif de protection de détection
WO2023055025A1 (fr) Procédé et dispositif électronique pour déterminer des informations cutanées à l'aide d'une reconstruction hyper spectrale
WO2019059632A1 (fr) Procédé et système de reconstruction d'une image hyperspectrale à l'aide d'un prisme
WO2020101380A1 (fr) Dispositif de diagnostic optique destiné à une maladie de la peau
WO2021112447A1 (fr) Dispositif électronique et son procédé de commande
WO2022149720A1 (fr) Procédé, système et support lisible par ordinateur pour fournir des informations de diagnostic préliminaire de dépression à l'aide d'un modèle d'apprentissage automatique
EP3235242A1 (fr) Dispositif de capture d'images et dispositif de protection de détection
WO2021235566A1 (fr) Système de détection de lésions cutanées multiples, procédé de détection de lésions cutanées multiples et support d'enregistrement lisible par ordinateur sur lequel est enregistré un programme pour la mise en œuvre de celui-ci
WO2021153858A1 (fr) Dispositif d'aide à l'identification à l'aide de données d'image de maladies cutanées atypiques
WO2024014853A1 (fr) Procédé et dispositif de détection de rides faciales à l'aide d'un modèle de détection de rides basé sur un apprentissage profond entraîné selon un marquage semi-automatique
WO2023239110A1 (fr) Dispositif d'expérience virtuelle pour recommander un style personnalisé à un utilisateur
WO2024210322A1 (fr) Dispositif électronique, serveur et système pour fournir des biosignaux hautement précis sur la base d'informations acquises sans contact, et procédé de fonctionnement associé
WO2020105954A1 (fr) Appareil portable pour diagnostiquer une maladie de la peau à l'aide d'informations d'image de la peau sur la base d'une longueur d'onde variable
Hasegawa et al. Skin visualization using smartphone and deep learning in the beauty industry
EP3896609A1 (fr) Procédé d'estimation de l'âge faciale de la peau et dispositif électronique
WO2025110598A1 (fr) Système d'intégration et d'augmentation de données multimodales, et procédé de traitement de données associé
WO2024039058A1 (fr) Appareil de diagnostic de la peau et système de diagnostic de la peau et procédé comprenant celui-ci
Jung et al. Integrated deep learning approach for generating cross-polarized images and analyzing skin melanin and hemoglobin distributions
WO2023054967A1 (fr) Dispositif d'estimation d'âge
JP2018180660A (ja) 生体認証装置、及び方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24775176

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 24775176

Country of ref document: EP

Kind code of ref document: A1