WO2020170160A1 - Augmentation améliorée d'une visualisation de réalité pour injection faciale - Google Patents

Augmentation améliorée d'une visualisation de réalité pour injection faciale Download PDF

Info

Publication number
WO2020170160A1
WO2020170160A1 PCT/IB2020/051386 IB2020051386W WO2020170160A1 WO 2020170160 A1 WO2020170160 A1 WO 2020170160A1 IB 2020051386 W IB2020051386 W IB 2020051386W WO 2020170160 A1 WO2020170160 A1 WO 2020170160A1
Authority
WO
WIPO (PCT)
Prior art keywords
structural
data
image
displacement
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/IB2020/051386
Other languages
English (en)
Inventor
Benoit HENDRICKX
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Augmented Anatomy bvba
Original Assignee
Augmented Anatomy bvba
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from BE20195109A external-priority patent/BE1027060A1/nl
Application filed by Augmented Anatomy bvba filed Critical Augmented Anatomy bvba
Publication of WO2020170160A1 publication Critical patent/WO2020170160A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/50—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for simulation or modelling of medical disorders
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/40—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing

Definitions

  • the invention relates to a system, a method and a computer program product for augmenting a visualisation of reality with additional information for facial injection.
  • US 2017/0 252 108 A1 describes a facial injection assist system comprising a visualisation system configured to display an augmented reality environment.
  • the visualisation system can be augmented reality glasses.
  • Patient-specific information about anatomical features (e.g. bones, nerves, blood vessels) of a patient being treated can be displayed via the visualisation system during an injection procedure. This can be obtained via a CT scan, an MRI scan, a photographic image, an X-ray image, and the like.
  • the information displayed may relate to an intended zone.
  • the information displayed may relate to an anatomical structure to be avoided.
  • the information displayed can be aligned with the face of the patient face based on alignment points.
  • the alignment points may relate to external anatomical features, such as a centre of an eye or a tip of a nose.
  • the facial injection assist system may comprise a camera for recording the location of the alignment points during the injection procedure.
  • the document further describes repeatedly obtaining the anatomical features of a specific patient while deforming the face for determining muscles and muscle deformation for optimally determining an intended zone. As a result, the anatomical features must be obtained for each of a plurality of deformations. Furthermore, the document does not provide for linking of information of the face surface to underlying anatomical features. Moreover, the document does not provide for an adequate representation of anatomical features for a distortion not previously obtained.
  • the present invention aims to find a solution for at least some of the above problems.
  • the present invention relates to a system for augmenting a visualisation of reality for facial injection, according to claim 1.
  • the system can also be used to augment a visualisation of reality for another medical or cosmetic treatment.
  • the present invention relates to a method for augmenting a visualisation of reality for facial injection, according to claim 24.
  • the method can also be used to augment a visualisation of reality for another medical or cosmetic treatment.
  • the present invention relates to a computer program product for augmenting a visualisation of reality for facial injection, according to claim 32.
  • the computer program product can also be used to augment a visualisation of reality for another medical or cosmetic treatment.
  • the present invention relates to a use of a system according to the first aspect for determining a zone to be avoided in a cosmetic injection of a patient with a filler.
  • the system according to the first aspect can be used for determining a zone to be avoided in an injection of a patient with a drug and/or autologous tissue.
  • the present invention relates to a use of a method according to the second aspect for determining a zone to be avoided in a cosmetic injection of a patient with a filler.
  • the method according to the second aspect can be used for determining a zone to be avoided in an injection of a patient with a drug and/or autologous tissue.
  • the present invention relates to a use of a computer program product according to the third aspect for determining a zone to be avoided in a cosmetic injection of a patient with a filler.
  • the computer program product according to the third aspect can be used for determining a zone to be avoided in an injection of a patient with a drug and/or autologous tissue.
  • Reference data from a head of a patient is obtained.
  • the head comprises a face and anatomical structures.
  • the reference data comprises facial data and structural data.
  • the anatomical structures comprise blood vessels.
  • the structural data comprises blood vessel data.
  • the facial data and structural data are obtained via magnetic resonance imaging (MRI) or computed tomography (CT).
  • An image of the face is recorded via a camera. There may be an actual facial deformation between the face when determining the facial data and the face when determining the image.
  • a spatial map is determined based on the facial data and the image.
  • the spatial map is calculated from the facial data and the image.
  • the spatial map comprises a distortion representation.
  • a structural displacement is determined based on the spatial map and the structural data.
  • the structural displacement comprises a vessel displacement.
  • the image is augmented on the basis of the structural displacement and represented via a visualisation tool. Augmenting the image means, among other things, updating the image on the basis of the structural displacement, which represents the displacement of (parts of) anatomical structures, and updating the position of said (parts of) anatomical structures on the image.
  • the‘old’ position of the anatomical structure is here omitted from the image, or adapted so that it becomes clear to a user from the visualisation what the updated position is.
  • the present invention is advantageous for various reasons. If reference data is obtained with a single spatial configuration of the head of a patient, the image can be correctly augmented for a plurality of spatial configurations, via the determination of the structural displacement. This not only results in a reduction in time and resources for obtainment, but also allows a visualisation of dynamically adapted anatomical structures to be projected over a dynamic image recording and thus form a dynamically augmented image.
  • the augmentation of the image based on the structural displacement thus follows, in real time, changes in the recorded image, for example the position, orientation and facial expressions of the patient.
  • Figure 1 shows an illustrative representation of an anatomy of blood vessels.
  • Figure 2 shows an illustrative representation of a subdivision of a face via movement axes (joints).
  • Figure 3 shows an illustrative representation of anchor points in a face.
  • Figure 4 shows a schematic representation of a blood vessel, a skin and a bone.
  • the invention relates to a system, a method and a computer program product for augmenting a visualisation of reality for facial injection.
  • the invention was summarised in the section provided for this purpose. In the following, the invention is described in detail, preferred embodiments are explained, and the invention is illustrated by way of examples.
  • ‘a’ and‘the’ refer to both the singular and the plural, unless the context presupposes otherwise.
  • ‘a segment’ means one or more segments.
  • face andcountenance are synonymous, and are to be interpreted within the present invention as the surface of the face or countenance.
  • An injection can be performed for cosmetic purposes.
  • An injection can be performed for medical purposes.
  • An injection can be an injection with a filler, a drug, an autologous tissue, or the like.
  • An example of a drug is a toxin for reducing muscle pain and/or muscle contraction.
  • An example of autologous tissue is fat.
  • a non- exhaustive list of examples of fillers comprises Restylane, Perlane, Radiesse, Evolence, Prevelle Silk and Juvederm.
  • a non-lim itative example of a toxin is Azzalure (Dysport).
  • A‘spatial map’ defines a spatial transform ation.
  • the transformation can com prise a translation .
  • the transformation can com prise a rotation .
  • the transformation can com prise a scaling.
  • the transformation can be a non-isom etric spatial transform ation .
  • the transform ation can in particular be a shape-changing non- isom etric spatial transform ation.
  • transform ation describes the result of a shape-changing non-isom etric spatial transform ation .
  • I n a non-lim iting illustrative example, a‘spatial map’ m ay com prise a list of origin coordinates, e.g. , reference anchor points, and corresponding target coordinates, e.g. , anchor points in an image; or comprise a spatial transformation based on such a list.
  • A‘transformation’ occurs in this example when, according to the spatial m ap, an angle based on three points and/or a ratio of distances between pairs of points changes.
  • the present invention relates to a system for augm enting a visualisation of reality for facial inj ection.
  • the present invention relates to a m ethod for augm enting a visualisation of reality for facial injection .
  • the present invention relates to a computer program product for augm enting a visualisation of reality for facial inj ection.
  • the present invention relates to a use of a system according to the first aspect for determ ining a zone to be avoided in a cosmetic injection of a patient with a filler.
  • the present invention relates to a use of a m ethod according to the second aspect for determ ining a zone to be avoided in a cosmetic injection of a patient with a filler.
  • the present invention relates to a use of a com puter program product according to the third aspect for determ ining a zone to be avoided in a cosm etic injection of a patient with a filler.
  • the m ethod according to the second aspect com prises various steps.
  • the system according to the first aspect is configured to perform the steps of the method according to the second aspect.
  • the com puter program product according to the third aspect com prises instructions which, when the computer program product is executed by a computer system com prising a processor, a cam era and a visualisation means, such as e.g . a system according to the first aspect, cause the com puter system to execute the steps of the m ethod according to the second aspect.
  • a person having ordinary skill in the art will therefore appreciate that the different aspects of the present invention are related.
  • Reference data from a head of a patient is obtained.
  • the head com prises a face and anatom ical structures.
  • the anatom ical structures comprise blood vessels, fat pads, nerves and/or m uscles.
  • the anatom ical structures com prise blood vessels.
  • the anatom ical structures are blood vessels.
  • the reference data com prises facial data and structural data and preferably also bone data.
  • the structural data comprises blood vessel data, fat pad data, nerve data and/or m uscle data.
  • the structural data comprises blood vessel data.
  • the structural data is blood vessel data.
  • the facial data and structural data are obtained via magnetic resonance im aging (MRI ) or computed tomography (CT) , or any other m edical im aging technique, or com binations of two or more techniques. Based on this, it is possible - by combining spliced im ages from different sections - to produce a discrete three-dim ensional representation of the head of a patient wherein said structures can be distinguished, also in discrete form .
  • at least bone data and blood vessel data are obtained via the imaging techniques, wherein these can be distinguished from each other and from other types of structures (wherein the other types of structures m ay or may not be further distinguishable from each other) .
  • the reference data is obtained on the basis of two or more separate im aging techniques whose data is combined. This can be useful in situations in which certain structures need to be distinguished and visualised that are difficult to distinguish in a single technique.
  • the way in which the reference data is obtained is possible via any imaging technique that is suitable for distinguishing the desired structures.
  • the position of the blood vessels is determ ined in at least a part of the face, in the three- dim ensional volume of the head. All detected structures are displayed in discrete, segm ented voxels.
  • An image of the face is recorded via a cam era.
  • a non-exhaustive list of exam ples of cam eras comprises a cam era (on the back or front) of a smartphone, a cam era connected to a desktop computer or laptop, and a cam era of augmented reality glasses.
  • a non-exhaustive list of augm ented reality glasses com prises Microsoft HoloLens, Apple AR-glasses, Google AR-glasses and Oculus AR-glasses.
  • a spatial map is determined based on the facial data and the image. The spatial map is calculated from the facial data and the image.
  • the spatial map comprises a distortion representation.
  • the spatial map defines a shape-changing non-isometric spatial transformation.
  • a structural displacement is determined based on the spatial map and the structural data.
  • the structural displacement comprises a blood vessel displacement, a fat pad displacement, a nerve displacement and/or a muscle displacement.
  • the structural displacement comprises a blood vessel displacement.
  • the structural displacement is a blood vessel displacement.
  • the image is augmented based on the structural displacement. The augmented image is displayed via a visualisation means.
  • the system comprises the camera and the visualisation means.
  • An operator can then inject the patient based on the augmented image.
  • This may involve cosmetic or aesthetic injection, for example with a filler.
  • This can alternatively also involve medical injection, for example with a treatment substance.
  • a user device comprising the camera and the visualisation means.
  • the user device can be augmented reality glasses.
  • the user device can be a portable computer system.
  • a non-exhaustive example list of portable computer systems comprises a smartphone, a tablet, and a laptop.
  • the system comprises a user device, wherein the user device is a portable computer system, the system itself is preferably the portable computer system.
  • the system is configured for:
  • obtaining reference data from a head of a patient comprising bones, a face and anatomical structures, the reference data comprising bone data, facial data and structural data, wherein the anatomical structures are blood vessels and wherein the structural data is blood vessel data;
  • augmenting the image based on the map and the structural data, and preferably the bone data wherein the augmentation of the image based on the map and the structural data, and preferably the bone data, visualises it when displaying the augmented image;
  • system being further configured for: determining a structural displacement based on the map, the facial data, the bone data and the structural data; wherein the structural displacement involves a blood vessel displacement; and
  • augmenting the image based on the structural displacement wherein augmenting the image based on the structural displacement visualises the structural displacement when displaying the augmented image
  • the method comprises the steps of:
  • obtaining reference data from a head of a patient comprising bones, a face and anatomical structures, the reference data comprising bone data, facial data and structural data; and wherein the anatomical structures are blood vessels, wherein the structural data is blood vessel data;
  • step of augmenting the image based on the map and the structural data comprises the following steps:
  • augmenting the image based on the structural displacement visualises the structural displacement when displaying the augmented image
  • the structural displacement being determined by: o determining a relative structural position d of a portion of the anatomical structures between a portion of the face and a portion of the bones based on the reference data;
  • the computer program performs the following steps:
  • obtaining reference data from a head of a patient comprising bones, a face and anatomical structures, the reference data comprising bone data, facial data and structural data;
  • step of augmenting the image based on the map and the structural data comprises the following steps:
  • augmenting the image based on the structural displacement wherein augmenting the image based on the structural displacement visualises the structural displacement when displaying the augmented image; and wherein the anatomical structures are blood vessels, wherein the structural data is blood vessel data;
  • o determining a facial deformation of the portion of the face relative to the portion of the bones based on the map; o determ ining the structural displacem ent of the portion of the anatom ical structures relative to the portion of the bones based on the facial deform ation and the relative structural position .
  • the relative structural position d is the relative depth of the portion of the anatom ical structures below the portion of the face relative to the depth of the portion of the bones below the portion of the face.
  • the structural displacem ent is determ ined on the basis of a flexibility ratio R, wherein the flexibility ratio of the portion of the anatom ical structures is defined as ( 1 - d) .
  • Figu re 1 shows an illustrative representation of blood vessels in a head of a patient.
  • the corresponding blood vessel can becom e blocked. This can lead to skin necrosis, nerve paralysis, and even blindness when the blood vessel provided a supply to the eye.
  • I t is therefore essential to avoid blood vessels when inj ecting with a filler. Due to the dense network of blood vessels it is not easy to avoid them .
  • the position of at least a portion of the blood vessels changes as the spatial configuration of the face changes, for exam ple with changes in facial expression. Different persons may also have m utual differences in the anatom ical position of the blood vessels.
  • the present invention can be used to augm ent an intended zone and/or a zone to be avoided in the image, depending on the case.
  • the im age can be correctly augmented for a plurality of spatial configurations, via the determ ination of the structural displacement based on the spatial map.
  • This not only results in a reduction in time and resources for obtainm ent, but also allows a visualisation of dynam ically adapted anatom ical structures to be proj ected over a dynam ic im age recording and thus form a dynam ically augm ented im age.
  • the augm entation of the image based on the structural displacement thus follows, in real tim e, changes in the image, for exam ple the position , orientation and facial expressions of the patient.
  • a plurality of anchor points are determ ined in the im age, and the spatial map is determ ined based on the facial data and the plurality of anchor points in the image.
  • the spatial map is determ ined based on the facial data and the plurality of anchor points in the image.
  • vis-a-vis the anchor points the three- dim ensional position information of the anchor points is recorded in the image.
  • the facial data can com prise reference anchor points.
  • the facial data can comprise three-dim ensional spatial facial inform ation.
  • Reference anchor points can be determ ined based on the three-dim ensional spatial facial inform ation .
  • the spatial map can be determ ined based on the reference anchor points and the plurality of anchor points in the im age.
  • the structural data is determ ined based on data obtained via m edical im aging, such as e.g. MRI or CT.
  • the facial data comprises reference anchor points.
  • the reference anchor points are determ ined based on the structural data.
  • the facial data com prises a subdivision into movem ent axes (joints) .
  • each of the movement axes comprises a weight.
  • a rigging based on a subdivision into movem ent axes can define a predeterm ined deformation capacity.
  • the facial data com prises reference anchor points and movement axes between the reference anchor points. With well-chosen anchor points and intermediate movem ent axes, this defined deformation capacity corresponds to the natural deformation capacity.
  • Figure 2 shows an illustrative representation of a subdivision of a face via movem ent axes.
  • the plurality of anchor points com prises at least 1 0 anchor points, more preferably at least 20 anchor points, even more preferably at least 30 anchor points, even more preferably at least 45 anchor points, and most preferably at least 60 anchor points.
  • a non-exhaustive list of examples of anchor points comprises an anchor point on a chin ; an anchor point on a lip; an anchor point at a corner of the mouth (left and/or right corner of the mouth) ; an anchor point on a nose; an anchor point on an eyelid (e.g. , medial and/or lateral canthus) ; an anchor point on an ear; and an anchor point on an eyebrow.
  • the plurality of anchor points comprises at least two, preferably at least three, more preferably at least four, even more preferably at least five, even more preferably at least six, and most preferably all seven , of the anchor points of the non-exhaustive list of examples of anchor points.
  • the anchor points are typically easily recognisable points on a face, preferably allowing them to be recognised automatically by the system itself - although manual indication and/or correction is also possible in situations with a lim ited number of anchor points.
  • anchor points are defined on (a‘historical’ image based on) reference data previously obtained.
  • an overlay of the historical image is created on the recorded im age, wherein a m apping is determ ined which determ ines the transform ation between the historical and the recorded im age, and in particular records the displacem ent of anchor points.
  • the historical image does not necessarily have to be effectively created here, since it can also be done in reverse and the recorded image is converted into ‘mom entary’ reference data and an inverse mapping can be determ ined with respect to the previously obtained reference data in order to thus record the deformations (whether or not via anchor points) . Both options, however, fall under the sam e principle.
  • Figu re 3 shows an illustrative representation of a plurality of anchor points in a face.
  • the reference anchor points are determ ined on the basis of the structural data, which is preferably obtained via a m edical im aging technique such as MRI or CT. These reference anchor points are patient-specific.
  • the plurality of anchor points in this exam ple comprises 68 anchor points.
  • Points 1 and 1 7 concern , for exam ple, anchor points on an ear helix.
  • Point 2 concerns, for example, an anchor point on the ear antitragus.
  • Point 3 concerns, for exam ple, an anchor point on an earlobe.
  • Point 9 concerns, for exam ple, an anchor point on a chin .
  • Points 1 8 , 22 , 23 and 27 concern, for exam ple, anchor points on an eyebrow.
  • Points 37 , 40 , 43 and 46 concern, for example, anchor points on an eyelid.
  • Point 28 concerns, for exam ple, an anchor point on a nasal bridge.
  • Point 31 concerns, for example, an anchor point on a tip of the nose.
  • Points 32 and 36 concern , for exam ple, anchor points at an attachment of a nostril.
  • Points 49 , 50 , 55 , 60 , 61 and 68 concern, for exam ple, anchor points on a contour of an upper or lower lip.
  • the structural data comprises three-dimensional spatial anatomical structure information.
  • the spatial anatomical structure information comprises spatial blood vessel information, spatial fat pad information, spatial nerve information and/or spatial muscle information.
  • the spatial anatomical structure information comprises spatial blood vessel information.
  • the spatial anatomical structure information is spatial blood vessel information.
  • the structural data comprises a three-dimensional spatial mesh.
  • a point cloud comprising spatial anatomical structure information is obtained.
  • the mesh can be determ in ed based on the point cloud.
  • the point cloud can be obtained via medical imaging, such as ultrasound, angiography, infrared imaging, magnetic resonance imaging (MRI) or computed tomography (CT), such as dual-energy CT.
  • CT has the advantage that it is faster than MRI, resulting in a faster obtainment, and consequently a smaller possible discrepancy between the facial data and the structural data due to movement of the patient.
  • CT computed tomography
  • CT has the advantage that it is faster than MRI, resulting in a faster obtainment, and consequently a smaller possible discrepancy between the facial data and the structural data due to movement of the patient.
  • the augmentation of the image is of significantly higher quality and the patient experiences considerably less discomfort.
  • metallic artefacts such as braces and crowns, to be suppressed using software.
  • the point cloud is obtained via MRI.
  • the point cloud can be obtained via MRI with the administration of no, one or more intravenous contrast mediums.
  • the point cloud is obtained via MRI without administration of an intravenous contrast medium or any addition to improve the quality of the representation of the anatomical structures.
  • MRI has the advantage that no radiological radiation is involved.
  • MRI is performed without administration of an intravenous contrast medium. Possible allergic reactions and complications due to an infusion are thus avoided.
  • the point cloud comprising spatial anatomical structure information is obtained via:
  • ⁇ exposing the patient, preferably the head, to an extracorporeal heat source, preferably an infrared lamp; • preferably, stimulating muscles of the patient, preferably muscles in the head, preferably by stimulation of the muscles by the patient, more preferably by voluntary stimulation of the muscles by the patient; and
  • the same embodiment can be used for the visualisation of one or more anatomical structures of another body part.
  • a non-exhaustive example list of other body parts comprises an arm, a leg, a hand, a foot. Analogous steps for the stimulation of muscles in the relevant body part can then be followed.
  • a vessel can be an artery, a vein, a lymph vessel or any other tubular anatomical structure in the human body, such as, for example, a vascular prosthesis.
  • the point cloud comprising spatial anatomical structure information is obtained without administration of an intravenous contrast medium, without administration of a drug, and without X-rays. This is advantageous to avoid health risks to the patient. Magnetic resonance angiography (MRA) offers this possibility.
  • MRA Magnetic resonance angiography
  • an extracorporeal heat source such as, for example, an infrared lamp, which is determined by the type of examination and/or the body part, such as, for example, 30 cm away from the face, parallel to the face, and for 10 minutes, for blood vessels in the face.
  • an extracorporeal heat source such as, for example, an infrared lamp
  • the body part such as, for example, 30 cm away from the face, parallel to the face, and for 10 minutes, for blood vessels in the face.
  • This induces vasodilation and an increased flow.
  • the patient is asked to activate muscles of the examined anatomical region, e.g., activating facial muscles by forming different facial expressions, such as frowning and smiling.
  • the structural data comprises a subdivision into movement axes.
  • the structural data comprises a subdivision of the mesh into motion axes.
  • each of the movement axes comprises a weight.
  • a spatial annotation map is determined of the structural data, preferably the mesh, on a template, the template comprising annotation information.
  • the motion axes, and preferably also the weight for each of the motion axes, is then determined based on the spatial annotation map and the annotation information of the template.
  • a flexibility ratio is determined for a portion of the anatomical structures based on a dimension of the portion of the anatomical structures, a location of the portion of the anatomical structures, and annotation information of the template corresponding to the spatial annotation map and the portion of the anatomical structures.
  • determining a structural displacement based on the spatial map and the structural data is determining a mesh deformation based on the spatial map and the mesh.
  • the image is then augmented based on a two- dimensional projection of at least a portion of the mesh deformation.
  • the head comprises bones (e.g., the bone structure of the skull) and the reference data comprises bone data.
  • the system is configured to determine a structural displacement based on the spatial map, the structural data, the facial data, and the bone data. Other data can also contribute to this.
  • the structural displacement is then determined based on the aforementioned data and other data.
  • a blood vessel displacement can be determined based on structural data comprising blood vessel data as well as data regarding other anatomical structures, such as, for example, structural data comprising blood vessel data, fat pad data, and muscle data.
  • the bone data comprises three-dimensional spatial bone information.
  • the bone data comprises a subdivision into movement axes.
  • each of the movement axes comprises a weight.
  • a relative structural position of a portion of the anatomical structures between a portion of the face and a portion of the bones is determined based on the reference data.
  • the relative structural position is the relative depth of the portion of the anatomical structures below the portion of the face relative to the depth of the portion of the bones below the portion of the face.
  • a facial deformation, for example as a result of facial expression via the facial muscles, i.e. a lateral or medial movement and/or cranial or caudal movement, of the portion of the face relative to the portion of the bones is determined based on the spatial map.
  • a flexibility ratio is determined for the portion of the anatomical structures based on the relative structural position.
  • a structural displacement of the portion of the anatomical structures relative to the portion of the bones is determined based on the facial deformation and the relative structural position.
  • a structural displacement of the portion of the anatomical structures relative to the portion of the bones is determined based on the facial deformation and the flexibility ratio.
  • the image is augmented based on the structural displacement of the portion of the anatomical structures.
  • the structural displacement appears to be very strongly linked to the relative depth of the anatomical structure.
  • the underlying bones are substantially rigid, and the overlying face portion is flexible (allows deformation)
  • the blood vessels are in an area anchored on one side to the underlying bones, and anchored on the other side to the overlying skin, causing the deformation of the skin to take place averaged out on the blood vessels, depending on the relative distances between blood vessel and skin on the one hand, and blood vessel and bones on the other hand.
  • Other (flexible) anatomical structures such as fat pads, nerves, muscles.
  • the displacement effect on underlying anatomical structures i.e. the structural displacement, can be determined.
  • An additional advantage is that a large part of the medical imaging methods, and in particular the more accessible ones, very easily allow blood vessels to be detected very accurately in view of their more deviating features compared to other structures/material between face and leg.
  • the reference data is typically obtained by combining 2D images into a discrete 3D representation (e.g. via separate - parallel - images or slices of an MRI, CT scan, ultrasound or angiography, but not limited to that).
  • This discretisation into full- volume voxels allows the structural displacement of an anatomical structure to be carried out in a discrete manner, wherein the anatomical structure is segmented into the voxels, and for each voxel the displacement is determined separately on the basis of a relative structural position of the voxel, and in particular the relative depth, wherein it is determined based on the distance of the voxel from the face (area thereof), and an underlying bone structure.
  • the relative structural position (of the portion of the anatomical structures below the face) is the relative depth thereof relative to the depth of the portion of the bones below the portion of the face.
  • the structural data and in particular the bone data, facial data and the blood vessel data, are provided with three-dimensional position information thereof in the obtained reference data.
  • the effective structural displacement can then also be determined based on the relative structural position.
  • the structural displacement can be determined on the basis of a flexibility ratio R, the flexibility ratio being defined as (1-d).
  • Figure 4 shows an illustrative schematic representation of a blood vessel (402), a skin (401) and a bone (403).
  • the distance from the blood vessel to the skin is Dh.
  • the distance from the blood vessel to the bone is Db.
  • a relative structural position of a portion of the anatomical structures, in this example the blood vessel (402), between a portion of the face, in this example the skin (401), and a portion of the bones, in this example the bone (403), is determined.
  • a facial displacement A of the portion of the face (401) relative to the portion of the bones (403) is determined based on the spatial map.
  • the template comprises a subdivision into zones and the annotation information of the template per zone comprises a mathematical deformation model.
  • Any mathematical deformation model can herein consider features of tissue structures of the zone.
  • the image is a dynamic image, a real-time image.
  • a sequence of images can be recorded at a predetermined frequency.
  • ‘real time’ in this document means that the relevant corresponding steps are repeatedly performed at said predetermined frequency for each image of the sequence, and preferably without visible delay for an operator.
  • the dynamic image is dynamically augmented based on a dynamically determined structural displacement. A structural displacement can be determined in real time for each image of the sequence of images, on the basis of which the image is augmented and displayed via the visualisation means.
  • the following steps are performed iteratively, with n initially 1 and increasing by 1 after an iteration.
  • An nth image of the face is recorded via the camera.
  • An nth spatial map is determined based on the nth image and the ( n - 1 ) t h image, the 0th image being the facial data.
  • An nth structural displacement is determined based on the nth map and the ( n - 1 ) t h structural displacement, the 0th structural displacement being the structural data.
  • the nth image is augmented based on the nth structural displacement.
  • the augmented nth image is displayed via the visualisation means.
  • a spatial map is determined based on two consecutive images of the sequence, on the basis of which a structural displacement corresponding to the earlier image of the two consecutive images is converted into a new structural displacement.
  • the system is configured to determine a size of the nth map.
  • the nth structural displacement is the (n-1 )th structural displacement in case the size of the nth map is smaller than a predeterm in ed dimension.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • General Health & Medical Sciences (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Databases & Information Systems (AREA)
  • Pathology (AREA)
  • Data Mining & Analysis (AREA)
  • Biomedical Technology (AREA)
  • Surgery (AREA)
  • Urology & Nephrology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)

Abstract

La présente invention comprend un système, un procédé et un produit programme d'ordinateur pour augmenter une visualisation de réalité pour une injection faciale. Des données de référence provenant de la tête d'un patient sont obtenues. La tête comprend un visage et des structures anatomiques. Les données de référence comprennent des données faciales et des données structurales. Une image du visage est enregistrée par l'intermédiaire d'une caméra. Une carte spatiale est déterminée sur la base des données faciales et de l'image. Un déplacement structural est déterminé sur la base de la carte spatiale et des données structurales. L'image est augmentée sur la base du déplacement structural et représentée par l'intermédiaire d'un outil de visualisation. Grâce à la détermination du déplacement structural, une image dynamique peut être dynamiquement augmentée sur la base d'un déplacement structural déterminé de manière dynamique. Les structures anatomiques peuvent comprendre des vaisseaux sanguins. Les données structurales peuvent comprendre des données de vaisseau sanguin. Le déplacement structural peut comprendre un déplacement de vaisseau.
PCT/IB2020/051386 2019-02-19 2020-02-19 Augmentation améliorée d'une visualisation de réalité pour injection faciale Ceased WO2020170160A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
BE20195109A BE1027060A1 (nl) 2019-02-19 2019-02-19 Verbeterde aanvulling van een visualisatie van de realiteit voor gelaatsinjectie
BEBE2019/5109 2019-02-19
BEBE2019/5785 2019-11-13
BE20195785A BE1027062A1 (nl) 2019-02-19 2019-11-13 Verbeterde aanvulling van een visualisatie van de realiteit voor gelaatsinjectie

Publications (1)

Publication Number Publication Date
WO2020170160A1 true WO2020170160A1 (fr) 2020-08-27

Family

ID=69811433

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2020/051386 Ceased WO2020170160A1 (fr) 2019-02-19 2020-02-19 Augmentation améliorée d'une visualisation de réalité pour injection faciale

Country Status (1)

Country Link
WO (1) WO2020170160A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116328081A (zh) * 2023-04-07 2023-06-27 北京艺升医疗科技有限公司 一种注射系统
WO2023162776A1 (fr) * 2022-02-28 2023-08-31 株式会社資生堂 Procédé d'évaluation, dispositif d'évaluation et programme

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003023713A1 (fr) 2001-09-10 2003-03-20 Visage Technologies Ab Clonage de mouvement facial
EP2258265A2 (fr) * 2009-06-03 2010-12-08 MINIMEDREAM Co., Ltd. Système de mesure de corps humain et son procédé de fourniture d'informations
US20150272691A1 (en) * 2014-03-31 2015-10-01 Tricubics Inc. Method and apparatus for providing virtual plastic surgery sns service
US20170252108A1 (en) 2016-03-02 2017-09-07 Truinject Medical Corp. Sensory enhanced environments for injection aid and social training
WO2019178287A1 (fr) * 2018-03-15 2019-09-19 Truinject Corp. Outils de réalité augmentée et systèmes d'injection

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2003023713A1 (fr) 2001-09-10 2003-03-20 Visage Technologies Ab Clonage de mouvement facial
EP2258265A2 (fr) * 2009-06-03 2010-12-08 MINIMEDREAM Co., Ltd. Système de mesure de corps humain et son procédé de fourniture d'informations
US20150272691A1 (en) * 2014-03-31 2015-10-01 Tricubics Inc. Method and apparatus for providing virtual plastic surgery sns service
US20170252108A1 (en) 2016-03-02 2017-09-07 Truinject Medical Corp. Sensory enhanced environments for injection aid and social training
WO2017151963A1 (fr) * 2016-03-02 2017-09-08 Truinject Madical Corp. Environnements sensoriellement améliorés pour aide à l'injection et formation sociale
WO2019178287A1 (fr) * 2018-03-15 2019-09-19 Truinject Corp. Outils de réalité augmentée et systèmes d'injection

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
ANONYMOUS: "Crisalix Virtual aesthetics.", 1 January 2016 (2016-01-01), XP055634414, Retrieved from the Internet <URL:https://vimeo.com/172972073> [retrieved on 20191022] *
GIUSEPPE SITO ET AL: "Vascular Complications after Facial Filler Injection: A Literature Review and Meta-analysis", J CLIN AESTHET DERMATOL., vol. 12, no. 6, 1 June 2019 (2019-06-01), pages e65 - e72, XP055634360 *
MARK THEODORE DRAELOS ET AL: "Robotic cosmetic dermatology : Editorial Echoes", JOURNAL OF COSMETIC DERMATOLOGY, vol. 10, no. 2, 1 June 2011 (2011-06-01), GB, pages 87 - 88, XP055693413, ISSN: 1473-2130, DOI: 10.1111/j.1473-2165.2011.00559.x *
R M KOCH ET AL: "A Framework for Facial Surgery Simulation", CS TECHNICAL REPORT #326, INSTITUTE OF SCIENTIFIC COMPUTING, 18 June 1999 (1999-06-18), XP055633780, Retrieved from the Internet <URL:http://webdoc.sub.gwdg.de/ebook/ah/2000/ethz/tech-reports/3xx/326.pdf> [retrieved on 20191018] *
STEFAN ZACHOW: "Computational Planning in Facial Surgery", FACIAL PLASTIC SURGERY., vol. 31, no. 05, 18 November 2015 (2015-11-18), STUTTGART, DE, pages 446 - 462, XP055633741, ISSN: 0736-6825, DOI: 10.1055/s-0035-1564717 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023162776A1 (fr) * 2022-02-28 2023-08-31 株式会社資生堂 Procédé d'évaluation, dispositif d'évaluation et programme
JPWO2023162776A1 (fr) * 2022-02-28 2023-08-31
EP4487760A4 (fr) * 2022-02-28 2025-01-15 Shiseido Company, Ltd. Procédé d'évaluation, dispositif d'évaluation et programme
CN116328081A (zh) * 2023-04-07 2023-06-27 北京艺升医疗科技有限公司 一种注射系统

Similar Documents

Publication Publication Date Title
TWI828701B (zh) 用於肺體積閘控ｘ射線成像系統和方法，及非暫態電腦可讀取儲存媒介
KR102946653B1 (ko) 방사선 파라미터 결정을 위한 시스템 및 방법
Mah et al. Technical aspects of the deep inspiration breath-hold technique in the treatment of thoracic cancer
JP4612057B2 (ja) 集束された超音波治療システム
Bibb et al. A review of the issues surrounding three-dimensional computed tomography for medical modelling using rapid prototyping techniques
Bettinardi et al. Motion management in positron emission tomography/computed tomography for radiation treatment planning
US20200008676A1 (en) Medical imaging apparatus comprising primary module and supplemental module and process thereof
Gunkel et al. Three-dimensional navigation in otorhinolaryngological surgery with the viewing wand
WO2020170160A1 (fr) Augmentation améliorée d&#39;une visualisation de réalité pour injection faciale
Rhodes Computer graphics and medicine: a complex partnership
Wang et al. Functional assessment of stenotic coronary artery in 3D geometric reconstruction from fusion of intravascular ultrasound and X-ray angiography
CN114947908A (zh) 辐射剂量的管理方法和装置
US20210251549A1 (en) Method of visualization of the patient&#39;s body surface and determining the coordinates of ecg electrodes during non-invasive electrophysiological mapping of the heart
JP7703573B2 (ja) 拡張現実を使用した解剖学的構造の動的位置合わせ
Chan et al. Three-dimensional computed craniofacial tomography (3D-CT): potential uses and limitations
Linte et al. Image-guided procedures: tools, techniques, and clinical applications
BE1027062A1 (nl) Verbeterde aanvulling van een visualisatie van de realiteit voor gelaatsinjectie
CN114980970B (en) System and method for determining radiation parameters
JP7433927B2 (ja) 放射線治療計画装置
Mara et al. Medical Imaging for Use Condition Measurement
Rhodes et al. Guest Editors' Introduction: Computers in Surgery and Therapeutic Procedures
Roostaeian Facial Analysis
Corcoran Medical electronics: Pacemaker responds to changes in body motion worldwide use of artificial hearts increases three-dimensional diagnostic imaging matures
HK40079965A (en) System and method for determining radiation parameters
HK40047128B (zh) 用於肺体积门控x射线成像的系统和方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20711320

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20711320

Country of ref document: EP

Kind code of ref document: A1