Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F18/00—Pattern recognition
  • G06F18/20—Analysing
  • G06F18/24—Classification techniques
  • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
  • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
  • G06V2201/03—Recognition of patterns in medical or anatomical images

Definitions

• the present invention relates to the medical field, and more particularly to the field of wound monitoring.
• One of the objects of the present invention relates to a method and a system for monitoring a patient's wound for characterizing said wound by means of image processing based on so-called machine learning methods (from English Machine Learning) implemented by neural networks.
• the present invention will find many advantageous applications, in particular for the monitoring and characterization of chronic wounds such as, for example, diabetic wounds, bedsores, ulcers of the lower limbs or even cancerous wounds.
• Chronic wounds can be of various origins.
• diabetic wounds plantar perforating pain
• pressure sores ulcers of the lower limbs
• cancerous wounds The most frequently encountered pathologies associated with these wounds are: diabetic wounds (plantar perforating pain), pressure sores, ulcers of the lower limbs and cancerous wounds.
• the controlled healing of a chronic wound is a long process that extends over several months. It involves many health players, including home nurses, general practitioners and certain specialists such as dermatologists, diabetologists, surgeons and geriatricians.
• the supposed etiology of the wound pressure sore, venous ulcer, perforating plantar injury, cancerous wound, separation of a surgical wound, etc.
• the directed healing phase cleansing, budding, epidermization; - the beginning, the middle or the end of the phase; the concomitance of several phases according to the zones of the loss of substances and the percentage of each of these phases in the wound;
• the dimensions of the loss of substances can be measured with a paper ruler to objectify the width, length and depth. Precise calculations of the area or volume of the loss of substance are only very rarely carried out.
• the doctor summarizes the data collected (antecedents, reason for consultation, history of the disease, clinical examination) and proposes appropriate therapeutic management based on his experience and the data acquired from science.
• the treatment includes advice (hygiene, lifestyle, dietary rules), general treatments and local care. Local care will most often be entrusted to a home nurse who will be responsible for cleaning the wound and changing the dressing every day or every two days until a permanent healing.
• the prescribing doctor then sees the patient regularly at variable intervals (weekly, fortnightly, monthly) in order to observe the good progress of the controlled healing and to modify his prescription if necessary.
• the morphological parameters studied are: the location of the wound, the date, the local vascular conditions (arterial and venous), the dimension of the wound, the type of tissue observed in the wound, the presence of an exudate, the quality of peripheral skin: dryness, redness, fragility, pain, pigmentation, infection.
• Treatment assistance proposal of a treatment algorithm based on the data collected and on the recommendations of the EPUAP and the NPUAP;
• the Applicant submits that the technology proposed in this document for characterizing the wound is based on the use of different classifiers for each tissue zone corresponding to a likelihood map. A matching is performed at the end of the analysis in order to merge regions of the wound corresponding to likelihood maps that are identical.
• this document proposes the implementation of an SVM type process for “Support Vector Machine”, and more precisely the use of a combination of SVMs in order to segment budding, fibrin and necrosis.
• an SVM makes it possible to carry out binary classifications, for example a classification between budding and all the rest, a classification between fibrin and all the rest or a classification between necrosis and all the rest.
• the method proposed in this document therefore remains fragile and is not suitable for medical use because the performances obtained are not sufficiently reliable, in particular when the shooting conditions vary (case of specular reflection).
• document US 2020/0234444 A1 proposes a medical device aimed at calculating a wound surface and at segmenting the wound into different tissue zones.
• the object of the present invention is to improve the situation described above.
• the present invention therefore aims to remedy the various drawbacks mentioned above by proposing a simple solution requiring few computer resources which allows monitoring and characterization of the wound with good accuracy, regardless of the camera used and the conditions. of shooting.
• the object of the present invention relates, according to a first aspect, to a method for monitoring a wound of a patient which is implemented by computer means and which comprises the following steps:
• a characterization of the wound comprising a determination of a rate of distribution of the characteristics of the wound by counting a number of pixels for each class.
• the method described here takes into account the context of the wound, will easily detect the specular reflection and ignore it as if it does not exist. There will therefore be no impact on our results at the end of the segmentation process.
• the classification step implements a convolutional neural network.
• the characterization step includes coloring with a single color of each of the pixels of the wound belonging to the same class.
• the characteristics of the wound are among the following:
• the characterization step comprises a calculation of a percentage of each class according to the number of pixels of the same class relative to a total number of pixels of the segmented image.
• the method according to the present invention includes contouring of the wound from the acquired image.
• the method according to the present invention comprises a transmission of said acquired image to a remote server.
• the method according to the present invention comprises an analysis of the rate of distribution of the characteristics of the wound by comparison with a predetermined wound model and/or a history of values recorded for the wound.
• the method according to the present invention comprises generation of a warning signal when the analysis of the comparison detects a risk of complication.
• the method according to the present invention comprises a step of restoring on a communication terminal an analysis report comprising the rate of distribution of the characteristics of the wound.
• the restitution step comprises a display on a graphic interface of a communication terminal of the segmented image and/or of the rate of distribution of the characteristics of the wound.
• the acquisition step is performed using a lidar type camera to determine a depth map of the wound.
• the method according to the present invention comprises a first calibration of the scale of the image to determine a measurement reference.
• a computer determines the distance between each of the pixels of said image from the depth map to determine the measurement reference of the image.
• the acquisition step includes a prior provision of a reference element affixed close to the wound in the field of vision of the camera.
• the reference element has at least two fiduciary markers separated from each other by a known distance.
• the first calibration comprises detection of said at least two markers in the acquired image in order to determine the measurement reference of the image.
• the detection and extraction of markers are instantaneous, and do not require any image processing.
• each of said at least two fiduciary markers is surrounded by a white frame; this improves its detection during the first calibration step. The reliability of detection and extraction is then 100% guaranteed.
• the reference element is in the form of an adhesive, one rear face of which comprises an adhesive strip intended to be fixed to the skin of the patient.
• the reference element has a front face comprising three fiduciary markers. This allows immediate knowledge of the orientation and inclination in the plane.
• the method according to the present invention includes a step of calculating the dimensions of the wound as a function of said image measurement reference.
• the dimensions of the wound include the length of the wound and/or the width of the wound and/or the area of the wound and/or the average depth of the wound and/or the maximum depth of the wound.
• the method according to the present invention includes a second colorimetric calibration of the image to determine a colorimetric reference.
• the reference element comprises on the front face:
• the method according to the present invention comprises a calibration of the acquired image.
• the calibration step comprises a colorimetric rebalancing consisting in normalizing the colors of the image according to the colorimetric reference of the image.
• the calibration step includes detection of a deformation of the reference element.
• the calibration step comprises a comparison between the acquired image and the calibrated image to determine a quality index of said image.
• the extraction of the reference element is carried out reliably thanks to fiduciary markers and mathematical rules, where document US 2019026891 only describes an extraction carried out by color which may therefore vary considerably depending on lighting conditions and/or image quality.
• This same sticker also makes it possible to offer a more reliable process for validating the correct conditions for acquiring the image.
• the reference element according to the present invention makes it possible to verify that the image acquisition metrics (distortion, inclination, distance from the camera) are satisfactory to lead to well (or not) the process.
• the object of the present invention relates, according to a second aspect, to a computer program which comprises instructions adapted for the execution of the steps of the method as described above, this in particular when said computer program is executed by at least one processor.
• Such a computer program can use any programming language, and be in the form of source code, object code, or intermediate code between source code and object code, such as in a partially compiled form, or in any other desirable form.
• the object of the present invention relates, according to a third aspect, to a recording medium readable by a computer on which is recorded a computer program comprising instructions for the execution of the steps of the method as described above.
• the recording medium can be any entity or device capable of storing the program.
• the medium may comprise a storage means, such as a ROM memory, for example a CD-ROM or a ROM memory of the microelectronic circuit type, or even a magnetic recording means or a hard disk.
• this recording medium can also be a transmissible medium such as an electrical or optical signal, such a signal being able to be conveyed via an electrical or optical cable, by conventional or hertzian radio or by self-directed laser beam or by other ways.
• the computer program according to the invention can in particular be downloaded from an Internet-type network.
• the recording medium may be an integrated circuit in which the computer program is incorporated, the integrated circuit being adapted to execute or to be used in the execution of the method in question.
• the object of the present invention relates, according to a fourth aspect, to a system for monitoring a patient's wound.
• the system comprises computer means configured for the implementation of the steps of the method described above.
• system according to the present invention comprises:
• a processor comprising an artificial intelligence algorithm configured to segment by machine learning said acquired image into a plurality of classes according to at least one determined image datum representative of a characteristic of said wound;
• - a calculator configured to count a number of pixels for each class
• a processing circuit configured to characterize said wound by determining a rate of distribution of the characteristics of said wound as a function of the number of pixels of each of the classes.
• Figure 1 shows an image of a wound with a reference element
• Figure 2 shows a schematic view of a reference element according to Figure 1.
• Figure 3 is a flowchart of a wound monitoring method according to an exemplary embodiment of the present invention.
• Figure 4 is a schematic view of a wound monitoring system according to an exemplary embodiment of the present invention.
• One of the objectives of the present invention is to design a tool to assist in the management of chronic wounds based on image analysis and artificial intelligence.
• a first step SI which consists in acquiring an image I of a wound PL to analyze it using a camera 10.
• This acquisition SI of image I can be done in several different ways .
• the carer has a communication terminal T of the smartphone or tablet type, for example, on which a dedicated software application is installed.
• this communication terminal T is equipped with an operating system (OS) of the Android® or iOS type, preferably Android® version 24 minimum or iOS version 12 minimum.
• OS operating system
• the communication terminal T can also be a computer or equivalent.
• Such a communication terminal T is also equipped with a camera 10 capable of capturing an image I or a stream of images.
• real-time assistance integrated into the application can optionally be offered in order to assist the caregiver in carrying out this step.
• This software assistance can in particular guide the caregiver to correctly center the PL wound during the SI image capture.
• step S1 it is also possible prior to step S1 to affix during a step S2 a reference element 20 in the field of view of said camera 10.
• Such a reference element 20 is characteristic of the present invention.
• the SI acquisition step can be done directly on the mobile application by importing an image found in the personal gallery of the caregiver's communication terminal T (with the same prerequisites as those mentioned above);
• the IS acquisition step can also be done on the web interface by choosing I images to be analyzed from the caregiver's medical photo library or by digitally importing other I images from the computer gallery , or any external storage media.
• One of the objects of the present invention is to have a usable I image from which it is possible to extract the precise dimensions of the PL wound and to characterize the PL wound.
• such a reference element 20 is in the form of a sticker.
• this reference element 20 comprises in this example an adhesive with a rear face comprising an adhesive strip intended to be in contact with the skin of the patient in order to fix the reference element 20 integrally to the skin of the patient close to the wound.
• PL as shown in Figure 1.
• the reference element 20 has a surface dimension that is known and uniform over time.
• the characteristics of the reference element 20 are as follows:
• Such a depth map can be obtained from different types of 3D camera such as a Lidar, True Depth or Time of Flight (ToF) type camera.
• a Lidar True Depth
• ToF Time of Flight
• a second colorimetric calibration S4 of the image I is also provided to determine a colorimetric reference REF_2 of the image I from the known colors present on the reference element 20.
• This S5 calibration step contains several sub-steps including:
• - a copy of the original image on a remote SD server, and in particular on electronic storage means of the volatile at/or non-volatile memory type and/or on a memory storage device which may include volatile memory and /or non-volatile, such as EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic or optical disk, in order to keep track of the original image in case of any problems; - a colorimetric rebalancing consisting in particular of normalizing the colors of the image I in order to minimize the impact of the different lighting conditions as much as possible (level of exposure, color of the lights, etc.);
• a calibration circuit 60 integrated on a dedicated platform such as for example the remote server SD.
• this calibration step S5 for recording the calibrated image G on electronic storage means of the volatile at/or non-volatile memory type and/or on a memory storage device which may comprise memory volatile and/or non-volatile, such as EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic or optical disk.
• This recording aims to be able to compare calibrated wounds under the same conditions.
• This optional step aims to standardize all the images I in order to reduce as much as possible the disparities between the terminals T of the different users. It is thus possible in the long term to offer consistent and similar analyzes regardless of the communication terminal T at the origin of the image.
• this calibration step S5 provision is made to determine the quality of the calibration by comparing the acquired image I and the calibrated image G. If the quality of the calibrated images is insufficient or if the calculation of the scale is imprecise, it may be considered that it is not possible to correctly characterize the PL wound, and therefore that it will not be necessary to continue with the rest of the procedure.
• Quality measurement can be done in different ways:
• each calibration S5 carried out with the reference element 20 causes a slight modification of the colors of the digital image I such that the colors of the reference element 20 as perceived through the image I (before calibration) come as close as possible to the real (and known) colors of the reference element 20 (after calibration).
• another method also consists of dividing the image into n subdivisions, calculating the average luminance for each of these zones and determining whether there is too great a difference in luminance between two related zones (threshold also configurable). If necessary, it becomes possible to detect a poor lighting condition.
• Such contouring S6 makes it possible to calculate the different characteristics of the wound with more precision.
• this zoom corresponds to the area between the screen and the finger when the latter are in contact;
• Such an automatic S6 clipping implements one or more Machine Learning algorithms in order to first segment the digital image into two classes: wound and other, then implements one or more image processing algorithms in order to segment the wound PL with respect to color differences between wound and skin and detect the set of all contours in the image. It is then possible to combine the results of these different approaches in order to map the digital image and determine a first outline of the wound. This S6 clipping can then be adjusted later by the user if necessary.
• the S6 clipping provided in the context of the present invention is therefore more precise and more reliable because it allows the user to correct the points of the polygon obtained thanks to the clipping (manual or automatic by AI) to extract the wound from the healthy skin.
• the method provides for a classification step S8 of the calibrated image F by machine learning as a function of at least one determined image datum representative of a characteristic of the wound PL (budding, fibrin, necrosis wet, dry necrosis, neutral), in order to segment the image into a plurality of classes.
• This step S8 makes it possible to extract the different properties necessary for the characterization of the wound (budding, fibrin, wet necrosis, dry necrosis, neutral).
• performing this step S 8 implements a convolutional neural network whose main role is to segment the wound into several classes.
• the step lasts only a few seconds.
• the output image a wound identical to the input one, is colored: each of the classes used for the characterization of the wound is colored with a unique color. Thus it is easy to distinguish which pixel of the digital output image belongs to which characteristic of the wound.
• Fibrin / Wet Necrosis (yellow / white / green hues);
• Dry necrosis purple / dark gray / black shades
• Neutral anything external to the wound.
• the implementation of such a neural network is not done locally on the communication terminal T but rather on a sufficiently powerful dedicated platform (remote server in the clouds, here denoted SD) to launch analyzes in parallel and manage user requests.
• a convolutional neural network also called convolutional neural network or convolutional neural network noted CNN or ConvNet (from the English “Convolutional Neural Networks”) corresponds to a network of artificial acyclic neurons (from the English “feed-forward” ).
• Such a convolutional neural network comprises a convolutional part implementing one or more convolution layers and a densely connected part implementing one or more layers of densely connected (or fully connected) neurons ensuring the classification of information according to an MLP type model (from English “Multi Layers Perceptron” or in French “Perceptrons multicouches”) for example.
• Such a process makes it possible to pool the resources necessary to determine the characteristics of a set of input data, these characteristics then being used by several layers of a multilayer neural network.
• This classification step S 8 is characteristic of the present invention. This is implemented by a processor 30 or a microcontroller of the type CPU, GPU, TPU, FPGA, etc., for example.
• a characterization step S9 is then provided during which a processing circuit 50 characterizes the wound PL by determining a rate of distribution of the characteristics of the wound PL.
• step S9 includes in particular: the counting by a computer 40 of the quantity of pixels for each class and the calculation of the percentage of each class relative to the total quantity of segmented pixels.
• a dimension calculation step S7 making it possible to calculate according to the measurement reference REF 1 of the image I a set of characteristic dimensions such as the area (expressed in cm 2 ), the depth (if applicable) or even the greatest length/width.
• Alerts can then be generated on the SC user interface if one of the parameters indicates a risk of complication or an unfavorable evolution of the PL wound over time: increased wound size, increased percentage of necrosis, increased percentage of fibrin.
• a report (for example in .pdf format) of this analysis can also be generated on the SC interface during a final SI 1 restitution step.
• This report will contain all the previously mentioned elements (patient information, images analyzed for this patient, analysis tables, evolution curves).
• This SI 1 restitution step also makes it possible to display the results of the digital analysis directly on the SC user interface (mobile application or online medical photo library).
• three images are displayed on the SC interface: the original image, the image with the contoured wound and the calibrated image (optional).
• a second table can also be displayed; this includes the results of the calculation of the dimensions of the wound: length, width, area, depth (optional).
• the present invention thus provides caregivers and various health actors acting in this type of follow-up with an intelligent medical aid tool adapted to chronic wounds.
• the present invention makes it possible in particular to semi-automate several tasks from one or more images of wounds, in order to:
• the present invention therefore allows a semi-automated analysis of images of chronic wounds and offers in particular the following advantages: the structuring and protection of data, the reduction of treatment costs, the improvement of the quality of care and the improving the working conditions of caregivers. It should be noted that this detailed description relates to a particular embodiment of the present invention, but that in no case does this description have any limiting character to the subject of the invention; on the contrary, it aims to remove any possible imprecision or any misinterpretation of the following claims.

Landscapes

• Engineering & Computer Science (AREA)
• Data Mining & Analysis (AREA)
• Theoretical Computer Science (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Bioinformatics & Cheminformatics (AREA)
• Bioinformatics & Computational Biology (AREA)
• Artificial Intelligence (AREA)
• Evolutionary Biology (AREA)
• Evolutionary Computation (AREA)
• Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Life Sciences & Earth Sciences (AREA)
• Measuring And Recording Apparatus For Diagnosis (AREA)
• Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=77519189

Family Applications (1)

Country Status (3)

Family Cites Families (5)

• 2021
  • 2021-05-10 FR FR2104935A patent/FR3122758B1/fr active Active
• 2022
  • 2022-05-10 WO PCT/FR2022/050898 patent/WO2022238658A1/fr not_active Ceased
  • 2022-05-10 EP EP22727384.4A patent/EP4338094A1/de active Pending

Also Published As

Similar Documents

Legal Events