EP4634871A1 - Appareil pour aider un utilisateur dans une planification basée sur une image et effectuer une procédure chirurgicale - Google Patents

Appareil pour aider un utilisateur dans une planification basée sur une image et effectuer une procédure chirurgicale

Info

Publication number
EP4634871A1
EP4634871A1 EP23821654.3A EP23821654A EP4634871A1 EP 4634871 A1 EP4634871 A1 EP 4634871A1 EP 23821654 A EP23821654 A EP 23821654A EP 4634871 A1 EP4634871 A1 EP 4634871A1
Authority
EP
European Patent Office
Prior art keywords
image
volume
projection
anatomical
interest
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP23821654.3A
Other languages
German (de)
English (en)
Inventor
Rafael Wiemker
Claas Bontus
Tom BROSCH
Hongxin Chen
Jose Alejandro MATUTE FLORES
Alexandra Groth
Harald Sepp Heese
Wei Ni
Hannes NICKISCH
Joukje Garrelina Orsel
Jochen Peters
Mark BAR
Rolf Jürgen WEESE
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Koninklijke Philips NV
Original Assignee
Koninklijke Philips NV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from EP23156444.4A external-priority patent/EP4386683A1/fr
Application filed by Koninklijke Philips NV filed Critical Koninklijke Philips NV
Publication of EP4634871A1 publication Critical patent/EP4634871A1/fr
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10132—Ultrasound image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/41—Medical

Definitions

  • the invention relates to an apparatus, a system comprising the apparatus, a method and a computer program product for assisting a user in an imaged-based planning and performing of a surgical procedure.
  • an apparatus for assisting a user in an image-based planning and performing of a surgical procedure, wherein the apparatus comprises a) an image proving unit for providing an anatomical volume image of an anatomical structure of a patient to be subject of the surgical procedure, b) a volume of interest definition unit for defining a volume of interest in the anatomical volume image comprising at least a part of the anatomical structure shown in the anatomical volume image, c) a contour enhancement unit for enhancing anatomical contours in the defined volume of interest based on the anatomical volume image resulting in an enhanced volume image of the volume of interest, d) a projection image generation unit for generating an enhanced projection image of the anatomical structure for a predetermined projection position, angle and direction, wherein the enhanced projection image is generated by determining a projection through the enhanced volume image along respective projection rays determined by projection position, angle and direction, e) a composite image generation unit for generating a composite image based on the enhanced projection image and an original
  • anatomic contours are enhanced in the volume image, i.e. the 3D image, enhancement algorithms with low computational cost can be utilized and anatomical structure can be accurately enhanced.
  • a projection image is generated and since a composite image is generated based on the enhanced projection image and an original projection image, for instance, a projection image generated based on the volume image without enhancement or directly acquired from a projection image acquisition unit, the composite image can be provided such that it shows not only accurately the enhanced anatomic contours but also other anatomic contours, wherein the presentation of the enhanced anatomic contours with respect to the other structures within the composite image can be adjusted very flexible in order to allow a user, for instance, a surgeon, to find the best view of the planning or reviewing of the surgical procedure.
  • the apparatus allows for an improved assistance of a user in planning and performing of an imaged-based surgical procedure.
  • the apparatus is configured for assisting a user in an imaged-based planning and performing of a surgical procedure, in particular, by presenting an improved composite image to a user.
  • the apparatus can be realize any form of any hardware and/or software provided by a general or dedicated computer system.
  • the apparatus can also be realized in distributed computing, for example, can be realized as part of a computer network, in which the functions of the apparatus are provided by different processors, servers or computer systems.
  • the surgical procedure can be any surgical procedure that is planed and/or performed utilizing medical projection images.
  • the surgical procedure refers to a transcatheter aortic valve implementation or replacement procedure, in particular, a minimal invasive procedure, in which a new cardiac valve is inserted, for instance, without removing an old damaged valve.
  • the image providing unit is configured for providing an anatomic volume image of an anatomic structure of a patient to be subject to the surgical procedure.
  • the image providing unit can refer to a storage unit or can be communicatively coupled to the storage unit, wherein anatomical volume images are already stored on the storage unit and the image providing unit is configured for providing anatomical volume image stored on the storage unit.
  • the image providing unit can also receive the anatomic volume image, for instance, from an input unit or via an interface directly from a respective imaging apparatus and the image providing unit is then configured for providing the received anatomical volume image.
  • the image providing unit can also refer to a respective image acquisition apparatus, for example, an CT apparatus, MRI apparatus, etc.
  • An anatomical volume image can refer to any image that shows an anatomical structure of a patient that is of interest for the surgical procedure.
  • anatomical volume image is a CT image or a MRI image.
  • the anatomical structure can be any anatomical structure that is of interest for the surgical procedure, preferably, the anatomical structure refers to at least a part of a heart of the patient, for example, an area of a valve of a heart of the patient.
  • the image is a contrast agent enhanced image of a heart of the patient.
  • the anatomical volume image is a pre-procedural image acquired for planning the surgical procedure.
  • the volume of interest definition unit is configured for defining a volume of interest in the anatomical volume image.
  • the volume of interest comprises at least a part of the anatomical structure shown in the anatomical volume image.
  • the volume of interest can also refer to the whole anatomical image, wherein in this case the volume of interest definition unit simply defines the whole image as the volume of interest.
  • the defining of the volume of interest by the volume of interest definition unit can be performed automatically, via a user input or a machine-guided user interaction process.
  • the volume of interest definition unit can be configured to define the volume of interest based on a predetermined anatomical structure of interest utilizing a segmentation of an anatomical structure or one or more known image characteristics of the predefined anatomical structure of interest.
  • other techniques like machine learning techniques for defining a volume of interest can be utilized.
  • the definition of the volume of interest has not to be very accurate, for example, a rough outline of the volume of interest is suitable for the following procedure.
  • the automatic methods and algorithms for defining a volume of interest do not have to be very sophisticated.
  • the volume of interest definition unit can define the volume of interest, for instance, by presenting the anatomical volume image to a user, wherein the user can then input the respective outline of the volume of interest into the volume image using respective input means.
  • an the interaction process between the user and the volume of interest definition unit can be utilized, for instance, the volume of interest definition unit can automatically determine a volume of interest in the anatomical volume image and present the automatic determination to a user, wherein the user can then adjust the volume of interest in an anatomical volume image.
  • the contour-enhanced unit is configured for enhancing anatomical contours in a defined volume of interest based on the anatomical volume image.
  • all known contour enhancement algorithms can be utilized for enhancing the anatomical contours in the defined volume of interest.
  • the contour enhancement unit is configured to enhance the anatomical contours in the volume of interest based on the anatomical volume image by applying a noise suppression contour filter to the anatomical volume image in the volume of interest.
  • a noise suppression contour filter has the advantage that an enhancement can be performed with low computational costs, i.e. very fast.
  • any known noise suppression contour filter can be utilized and applied to an anatomical volume image for enhancing the respective anatomical contours.
  • a bilateral edge-detection algorithm can be utilized as described, for example, in the articles “A 3D Image Filter for Parameter-Free Segmentation of Macromolecular Structures from Electron Tomograms.”, A. RA et. al., PLoS ONE 7(3): e33697 (2012) and “Bilateral edge detectors.”, Jose et. al., International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE. 1449-1453 (2013).
  • IICASSP International Conference on Acoustics, Speech and Signal Processing
  • the application of the noise suppressing contour filter comprises utilizing a local Hessian matrix calculated for a voxel of the volume of interest and determining an enhancement value for the voxel of the volume of interest based on the calculated Hessian matrix of the respective voxel for enhancing the anatomical contours in the volume of interest.
  • a Hessian matrix is a sparse matrix of second order partial derivatives of a scalar field.
  • the Hessian matrix of a voxel of the anatomical volume image can be calculated based on the value of the voxel and also based on the values of voxels neighboring the voxel utilizing known numerical method for calculating respective derivatives. Accordingly, the Hessian matrix of the voxel does not only take the value of the voxel into account, but also the values of neighboring voxels, which allows for increased suppression of noise.
  • the calculating of the enhancement value for a voxel comprises utilizing the highest positive eigenvalue of the Hessian matrix for this voxel clamped by zero.
  • the contour enhancement unit is in this preferred embodiment configured for calculating the eigenvalues of the Hessian matrix for a voxel.
  • the three eigenvalues of the Hessian matrix comprise information on the local gray-value curvature around a respective voxel, for which the Hessian matrix has been calculated.
  • cardiac structures like cardiac valve leaflets that are represented, for instance, in a contrast agent enhanced image as dark contours, particularly as dark planes, it has been found that these contours can be recognized by utilizing positive eigenvalues in the respective Hessian matrix of the voxel belonging to the cardiac valves.
  • the value of the voxel is set to the highest positive eigenvalue of the Hessian matrix for this voxel clamped by zero. Accordingly, if none of the eigenvalues is positive for a voxel, this voxel will be set in an enhanced volume image to zero, whereas voxels comprising one or more positive eigenvalues are set to the highest positive eigenvalue. This allows, in particular, for a fast and easy calculation of enhanced contours for structures they can comprise a darker grey value in the respective anatomic volume image, like heart valves.
  • the contour enhancement unit can be configured for enhancing the anatomical contours in the volume of interest based on the anatomical volume image by utilizing a trained machine learning based model configured for enhancing anatomical contours in an image based on the provided volume image.
  • a trained machine learning based model configured for enhancing anatomical contours in an image based on the provided volume image.
  • Respective trained machining learning based models configured for enhancing anatomical contours in an image based on the provided volume image are known.
  • trained machine learning algorithms can be parameterized utilizing a plurality of labeled volume images, for instance, volume images that have already been provided with an enhancement, wherein the respective machine learning model can then be trained based on this plurality of labeled volume images.
  • An example, of such a machine learning algorithm that can be utilized here can be found in the article “Hough-CNN: Deep Learning for Segmentation of Deep Brain Regions in MRI and Ultrasound.”, F. Milletari, et. al., arXiv: 1601.07014 (2016).
  • the enhancement of the anatomical volume image results in an enhanced volume image.
  • the projection image generation unit is configured for generating an enhanced projection image of the anatomic structure for a predetermined projection position, angle and direction.
  • the enhanced projection image is generated by determining a projection through the enhanced volume image along respective projection rays determined by the predetermined projection position, angle and direction.
  • methods for generating a projection image from a volume image along respective predetermined projection positions, angles and directions are known and can be utilized by the projection image generation unit.
  • IGSTK Image-Guided Surgery Toolkit
  • the enhanced anatomical contours will also be enhanced in the enhanced projection image.
  • not enhanced anatomical structures will be less visible in the enhance projection image.
  • the composite image generation unit is configured for generating a composite image based on the enhanced projection image and an original projection image of the anatomical structure.
  • the original projection image is acquired by projecting through the anatomical structure along respective projection rays determined by the predetermined projection position, angle and direction that were also utilized for generating the enhanced projection image.
  • the original projection image can refer to a projection image that is acquired by a projection image acquisition unit Trent or during a surgical procedure.
  • a projection image can be utilized showing the anatomical structures very accurately.
  • the projection image in this case can be acquired utilizing a fluoroscope or an x- ray system, or any other type of projection acquisition unit.
  • the projection image generation unit is further configured to generate the original projection image of the anatomical structure for a predetermined projection position, angle and direction, wherein the original projection image is generated by determining the projection through the anatomical volume image along respective projection rays determined by projection position, angle and direction.
  • the same or any other known algorithms for determining a projection image from a volume image can be utilized for generating the original projection image based on the provided anatomical volume image.
  • the original projection image and the enhanced projection image are then utilized to generate a composite image comprising aspects of both projection images and the enhanced projection image.
  • any known algorithm that allows a fusing of two images, in particular, two medical images, can be utilized.
  • a user can input respective preferences for the composite image.
  • the enhanced projection image and the original projection image can be utilized during the fusion in different ways.
  • different colors for the images can be utilized, or the respective projection images can be fused with different grey scales.
  • voxels mainly defines by the enhanced projection image can be provided with a color different from voxels mainly defined by the original projection image. This allows to customize the composite image such that an image can be provided that is most suitable for the intention of the user.
  • the composite image allows to show an important anatomical structure with an enhance contrast in the projection image, but at the same time also shows other anatomical structures that were not enhanced during enhancement process.
  • the composite image allows for providing an easy orientation in the anatomical structures for a user, whereas at the same time making it easier to find the relevant anatomical structures, i.e. then enhanced anatomical contours. Accordingly, when the interface unit configured for presenting the composite image to a user presents this composite image, the user is assisted in the planning or performing of the surgical procedure.
  • the contour enhancement unit is configured to further apply a Gaussian smoothing before the enhancement of the anatomical contours in the volume of interest.
  • the Gaussian smoothing allows for a more accurate determination of the anatomical contours, by removing artifacts that could be due to noisy images.
  • the apparatus further comprises a registration unit for registering the composite projection image and anatomical volume image such that a position of a point of interest in the anatomical volume image is associated with a position of the point of interest in the combined projection image and such that a point of interest in the composite projection image is associated with a projection ray comprising the point of interest in the anatomical volume image.
  • the connection between the composite image and the anatomical volume image is already known.
  • the registration can be easily performed by tracing the position of the voxels in the anatomical volume image during the generation of the enhanced volume image and projection through the enhanced image to the position in the composite projection image.
  • the original projection image is also generated by projecting through the anatomical volume image also the connection between these two images can be utilized for registering the composite projection image and the anatomical volume image.
  • the registration is provided such that the point of the interest in the composite project image is associated with a projection ray comprising the point of interest in the anatomical volume image.
  • the apparatus further comprises an interactive navigation unit for providing an interactive navigation of the anatomical image volume based on the composite projection image and the registration to a user, wherein the interactive navigation comprises determining a position of a point of interest in the composite projection image in the anatomical volume image by determining the most contributing voxel along a projection ray associated with the point of interest in the composite projection image.
  • a user indicating a point of interest in the composite projection image refers to a structure presents at the point of interest that is well visible in the composite projection image, for instance, enhanced, instead of any anatomical structure surrounding the most visible structure, providing the most contributing voxel along the projection ray as a point of interest in the anatomical volume image, to which the indicated point of interest in the composite projection image refers, is the most useful and easiest way of associating a point of interest in the composite projection image with a point of interest and anatomical volume image.
  • voxels belonging to an enhanced contour are provided with a higher weight than other voxels along the projection ray.
  • the apparatus further comprises a simulated volume image generation unit for generating a simulated volume image based on an acquired projection image and based on the registration between the anatomical volume image and the composite image. Since registration between the anatomical volume image and the composite image is known, in particular, since the projection ray associated with a point of interest in the composite image is known, an acquired projection image, for instance, acquired during the surgical procedure, can be utilized to generate a simulated volume image. In particular, based on the registration, the anatomical volume image can be amended to fit the acquired projection image in order to generate the simulated volume image.
  • iterative procedures can be utilized, in which through the registration it is easy to iterate back and forth from an amended anatomical volume image to the projection image, until the projection image is equal to the acquired projection image and the amended anatomical volume image can thus be regarded as simulated volume image.
  • This allows, in particular, during a complex surgical procedure, for example, when a 3D replanning is necessary, to acquire a fast up-to-date volume image without having to interrupt the surgical procedure for acquiring a real 3D volume image of the anatomical structure.
  • a method for assisting a user in an imagebased planning and performing of a surgical procedure, wherein the method comprises a) providing an anatomical volume image of an anatomical structure of a patient to be subject of the surgical procedure, b) defining a volume of interest in the anatomical volume image comprising at least a part of the anatomical structure shown in the anatomical volume image, c) enhancing anatomical contours in the defined volume of interest based on the anatomical volume image resulting in an enhanced volume image of the volume of interest, d) generating an enhanced projection image of the anatomical structure for a predetermined projection position, angle and direction, wherein the enhanced projection image is generated by determining the projection through the enhanced volume image along respective projection rays determined by the projection position, angle and direction, e) generating a composite image based on the enhanced projection image and an original projection image of the anatomical structure, wherein the original projection image is acquired projecting through the anatomical structure along respective projection rays determined by the projection
  • a system for assisting a user in an imagebased planning and performing of a surgical procedure, wherein the system comprises a) an image acquisition unit configured for acquiring an anatomical volume image, and b) an apparatus as described above.
  • a computer program product for assisting a user in an image-based planning and performing of a surgical procedure, wherein the computer program product causes an apparatus as described above to carry out the method as described above.
  • Fig. 1 shows schematically and exemplarily a system for assisting a user in an imaged- based planning and performing of a surgical procedure.
  • Fig. 2 shows schematically and exemplarily a flowchart of a method for assisting a user in an imaged-based planning and performing of a surgical procedure
  • Fig. 3 shows schematically and exemplarily a CT image generated utilizing the above apparatus and/or method.
  • Fig. 1 shows schematically and exemplarily a system for assisting a user in an imaged- based planning and performing of a surgical procedure.
  • the system 100 comprises a volume image acquisition unit 120 and an apparatus 110 for assisting a user in an imaged-based planning and/or performing of a surgical procedure.
  • the volume image acquisition unit 120 for instance, a CT imaging unit, acquires an anatomical volume image of a patient 121 lying on the patient table 122.
  • the acquired anatomic volume image is then provided to the apparatus 110.
  • the apparatus 110 comprises an image providing unit 111, a volume of interest definition unit 112, a contour enhancement unit 113, a projection image generation unit 114, a composite image generation 115 and an interface unit 116.
  • the apparatus can comprise an input unit 117 and an output unit 118.
  • the apparatus 110 can be realized in any form of software/hardware combination of a computing device, in particular, the apparatus 110 can be realized and in form of distributed computing, in which a plurality of processors at different locations perform the functions of apparatus 110 described in the following.
  • the image providing unit 111 is configured for providing an anatomical volume image acquired, for instance, from the volume image acquisition unit 120.
  • the volume image acquisition unit 120 can provide the acquired anatomical volume image to a storage unit and the image providing unit 111 can be configured for accessing the storage unit in order to provide the anatomical volume image, for instance, to the volume of interest definition unit 112.
  • the image providing unit 111 can also directly receive, for instance, via a wireless communication interface, an anatomical volume image from the anatomical volume image acquisition unit 120.
  • the volume of interest definition unit 112 is then configured for defining a volume of interest in the anatomical volume image comprising at least a part of the anatomical structure shown in the anatomical volume image.
  • the volume of interest definition unit 112 can present the volume of interest to users utilizing an output unit, for instance, a display 117.
  • a user can then define the respective volume of interest in the anatomical volume image.
  • the volume of interest definition unit 112 can also automatically define the volume of interest, for instance, utilizing predetermined rules or algorithms, in particular, a machine learning algorithm, a segmentation algorithm, or a respective filter algorithm.
  • the contour enhancement unit 113 is then configured for enhancing anatomical contours in the defined volume of interest based on the anatomical volume image resulting in an enhanced volume image of the volume of interest.
  • the contour enhancement unit 113 can utilize any known contour enhancement algorithm, for instance, contour enhancement filters or trained machine learning algorithms. Details on preferred examples will be provided further below.
  • the projection image generation unit 114 is then configured for generating an enhanced projection image of the anatomical structure based on the enhanced volume image.
  • a projection position, angle and direction are predetermined, for instance, utilizing input unit 118 realized in form of a keyboard, mouse, or any other user interface, for determining the projection through the enhanced volume image along the respective projection rays, determined by the projection position, angle and direction.
  • methods and algorithms for determining projection images from volume images are known and can be utilized by the projection image generation unit 114.
  • the composite image generation 115 can then generate a composite image based on the enhanced projection image and based on an original projection image.
  • the original projection image of an anatomical structure can be a projection image that has been acquired by a projection image acquisition unit, for instance, an X-ray imaging unit or a fluoroscope imaging unit or can be a generated projection image that has been generated, for instance, from the anatomical volume image before the enhancement.
  • the composite image then refers to a fusion of the enhanced projection image and the original projection image such that in addition to the enhanced contours also other anatomical structures are visible in the composite image.
  • the composite image can be generated, for instance, as an overlay of the respective enhanced projection image and original projection image utilizing predetermined colors, weights, and/or grey value scales.
  • the value can be determined by adding or weighted adding the respective image values, by multiplying the respective image values, selecting a maximum/minimum image value, etc.
  • a user can utilize input unit 118 and output unit 117, to determined different kinds of composition, for instance, to determine different colors, schemes, different grey scales, etc. for the respective projection images for the fusing.
  • a such generated composite image can then be presented via the interface unit 116 to the user, utilizing the output unit 117, in particular realized in form of a display.
  • the method 200 comprises a step 110 of providing an anatomical volume image of an anatomical structure of the patient on which the surgical procedure should be performed. Further, in step 120, a volume of interest in an anatomical volume image is defined comprising at least a part of anatomical structure shown in the anatomical volume image. In step 230, anatomical contours in the defined volume of interest are enhanced based on the anatomical volume image resulting in an enhanced volume image of the volume interest.
  • an enhanced projection image of anatomical structure is generated for a predetermined projection positon, angle and direction.
  • the enhanced projection images are generated by determining a projection through the enhanced volume image along respective projection rays determined by the projection position, angle and direction.
  • the method comprises a step 250 of generating a composite image based on the enhanced projection image and the original projection image of the anatomical structure.
  • the original projection image is an acquired projection through the anatomical structure along respective projection rays determined by the projection position, angle and direction.
  • the composite image is then presented to the user allowing to assist the user in the planning/performing of a respective surgical procedure.
  • Transcatheter aortic valve implantation or replacement is a minimally invasive procedure where a new cardiac valve is inserted without removing the old, damaged valve.
  • TAVI procedures are usually carried out under intra-operative image guidance based on fluoroscopy, preferably, dynamic X-ray projection images, but the pre-operative planning is usually carried out in preoperative CT image volumes. While there are techniques aiming to render interventional imaging as close to 3D CT imaging, this information is often not available prior to the intervention.
  • the composite image, of the region of interest which is just enriched to a subtle and optionally adjustable degree by additionally emphasizing the anatomic contours, while not cluttering or overburdening the standard familiar representation.
  • an automatic generation of a region of interest can be utilized that does not need to be a precise, flawless delineation of the structure of interest, in this example, valve, aorta and outflow tract. Rather, an approximate envelope is sufficient.
  • the enhancement in the composite image allows then for accelerated and intuitive navigation to points of interest to be visually appreciated and measured. Since preferably for the enhancement an analytical algorithm, like a contour filter is utilized, no training data for machine learning is required with its accompanying annotation, sampling, imaging protocol coverage, and regulatory efforts, respectively.
  • an aim for this application of the invention is to provide an intuitive graphical presentation for visual appreciation and navigational aid to points of interest for pre-operative measurement and planning.
  • An example of a preferred embodiment of a method of the invention comprises the following computer-implemented steps that can be performed, for example, by the apparatus discussed in Fig. 1.
  • an X-ray projection forward simulation can be utilized to compute fluoroscopy-like projection images from CT image volumes under respective predetermined projection angles.
  • a respective algorithm can be utilized to defined an approximate volume of interest, e.g., around the aortic valve, in the CT volume image.
  • model-based segmentation or other machine learning based semantic segmentations like deep CNNs can be utilized for the region of interest definition.
  • a noise suppressing contour filter is applied to the CT image volume to collect filter responses along anatomical contours within the region of interest with preferably intrinsic noise suppressing properties and invariance against contrast agent concentration.
  • the contour filter can comprise determining eigenvalues of a Hessian matrix of second derivatives.
  • a composite volume rendering for an unobtrusive embedding of the contour filter response into the simulated projection images can be used for generating the composite image.
  • a soft alpha-blending tapering off at the regions of interest margins can be used.
  • the generated projection images can then optionally by spatially co-registered to link the projection images with the three-dimensional CT image volume, considering in particular the enhanced contours, for interactive navigation, e.g., by mouse click, to the structures of interest locations in 3D CT volume image.
  • An application of the above method to a cardiac CT image is shown in Fig. 3.
  • the top row shows image slices that are reformatted obliquely from the original scanner grid, in order to rotate around an axis through the aortic cardiac valve.
  • a generated fluoroscopy projection image prior to enhancement, showing the valve structures, e.g., bulbus and tri -cuspid leaflets in a subtle way, due to superpositions of other adjacent anatomies is sown.
  • the right column the same simulated rotated projection images are shown, this time with an embedded contour enhancement within the region of interest around the valve.
  • the filter response at the contours for enhancing the contours in the volume image is computed from differential geometry properties.
  • a local Hessian matrix of partial spatial derivatives is computed for each voxel in the region of interest after applying an optional initial Gaussian smoothing.
  • the derivatives of the Hessian matrix are invariant against the absolute intensity level, e.g., from varying contrast agent concentration in the ascending aorta and left cardiac ventricle.
  • certain local anatomical structures like, planar, tubular, or spherical structures combinations or conditionals of the three real eigenvalues of a symmetric Hessian matrix can be utilized.
  • the corresponding 3D location is shown in a standard slice view port of the volume image, e.g., indicated by a cross-hair and switch to the location containing slice of the volume image.
  • the correspondence between a projection ray referring to a pixel in the enhanced projection image and a 3D location of a voxel in the CT image volume is not bijective.
  • the most contributing point, e.g. voxel, along a projective ray can be utilized as point of interest.
  • a measure for identifying the most contributing point can be, for example, the point along the ray with the highest radio-opacity of all point belonging to the ray.
  • a measure is used that provides more weight to enhanced contour points along the projection rays, such that a user selection on the enhanced projection image leads most likely to an enhanced contour point in the three-dimensional CT volume.
  • enhanced contours are also integrated into intra-operative images.
  • an intra-operative projection image can be used for generating the composite image.
  • a forward coordinate correspondence obtained from an image-registration between the pre-operative CT image and the interventional X-ray images can be utilized to combine the enhanced projection image with the intra-operative projection image such that the CT-based contours can be added as an overlay into the interventional X-ray images to enhance those.
  • 2D interventional X-ray images can be rendered into 2.5D or 3D images using the forward coordinate correspondence obtained from image-registration between the preoperative CT volume image and the interventional X-ray images.
  • 2D+t interventional X-ray images/sequences can be augmented into 2.5D+t or 3D+t interventional X-ray sequences by using the available geometrical context from the pre-operative 3D CT volume images. This allows to virtually change the perspective of a 2D+t interventional X-ray acquisition during the intervention.
  • a toggling between automatic segmentation and edge-enhancing visualization is provided.
  • a user cab be enabled to toggle between a suggested analytical contour enhancement within the region of interest, as described above, versus an overlay of an automatic segmentation or planning result, in order to visually check the consistency/accuracy/certainty.
  • the above described invention can, in particular, be integrated into imaging workstations and PACS-viewers dedicated to diagnosis and or pre-operative surgical planning, e.g., of structural heart disease, SHD or Valvular Heart Disease, VHD, transcatheter aortic valve implantation or replacement, TA VI or TAVR.
  • imaging workstations and PACS-viewers dedicated to diagnosis and or pre-operative surgical planning, e.g., of structural heart disease, SHD or Valvular Heart Disease, VHD, transcatheter aortic valve implantation or replacement, TA VI or TAVR.
  • imaging modalities relevant cardiac 3D imaging such as MRI, US, SPECT, PET.
  • a single unit or device may fulfill the functions of several items recited in the claims.
  • the mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
  • Procedures like the providing of an anatomical volume image, the defining of volume of interest, the enhancing of anatomical contours, the generating of an enhanced projection image, the generating of the composite image, the presentation of the composite image et cetera, performed by one or several units or devices can be performed by any other number of units or devices.
  • These procedures can be implemented as program code means of a computer program and/or as dedicated hardware.
  • a computer program product may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. Any reference signs in the claims should not be construed as limiting the scope.
  • An image proving unit provides an anatomical volume image of an anatomical structure of a patient.
  • a contour enhancement unit enhances anatomical contours in a defined volume of interest based on the anatomical volume image resulting in an enhanced volume image.
  • a projection image generation unit generates an enhanced projection image of the anatomical structure.
  • a composite image generation unit generates a composite image based on the enhanced projection image and an original projection image of the anatomical structure, wherein the original projection image is acquired projecting through the anatomical structure along respective projection rays determined by a projection position, angle and direction.
  • An interface unit presents the composite image to a user.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Epidemiology (AREA)
  • General Health & Medical Sciences (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Primary Health Care (AREA)
  • Public Health (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

Un objet de l'invention est de fournir un appareil (110) qui permet une assistance améliorée d'un utilisateur dans une planification basée sur l'imagerie et la réalisation d'une procédure chirurgicale. Une unité de preuve d'image (111) fournit une image de volume anatomique d'une structure anatomique d'un patient. Une unité d'amélioration de contour (113) améliore les contours anatomiques dans un volume d'intérêt défini sur la base de l'image de volume anatomique résultant en une image de volume améliorée. Une unité de génération d'image de projection (114) génère une image de projection améliorée de la structure anatomique. Une unité de génération d'image composite (115) génère une image composite sur la base de l'image de projection améliorée et d'une image de projection d'origine de la structure anatomique, l'image de projection d'origine étant acquise en saillie à travers la structure anatomique le long de rayons de projection respectifs déterminés par une position, un angle et une direction de projection. Une unité d'interface (116) présente l'image composite à un utilisateur.
EP23821654.3A 2022-12-14 2023-12-12 Appareil pour aider un utilisateur dans une planification basée sur une image et effectuer une procédure chirurgicale Pending EP4634871A1 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
CN2022138920 2022-12-14
EP23156444.4A EP4386683A1 (fr) 2022-12-14 2023-02-14 Appareil pour aider un utilisateur à planifier et exécuter une procédure chirurgicale sur la base d'une image
PCT/EP2023/085291 WO2024126458A1 (fr) 2022-12-14 2023-12-12 Appareil pour aider un utilisateur dans une planification basée sur une image et effectuer une procédure chirurgicale

Publications (1)

Publication Number Publication Date
EP4634871A1 true EP4634871A1 (fr) 2025-10-22

Family

ID=89168188

Family Applications (1)

Application Number Title Priority Date Filing Date
EP23821654.3A Pending EP4634871A1 (fr) 2022-12-14 2023-12-12 Appareil pour aider un utilisateur dans une planification basée sur une image et effectuer une procédure chirurgicale

Country Status (3)

Country Link
EP (1) EP4634871A1 (fr)
CN (1) CN120390942A (fr)
WO (1) WO2024126458A1 (fr)

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2015204201B2 (en) * 2014-01-06 2019-11-28 Body Vision Medical Ltd. Surgical devices and methods of use thereof

Also Published As

Publication number Publication date
CN120390942A (zh) 2025-07-29
WO2024126458A1 (fr) 2024-06-20

Similar Documents

Publication Publication Date Title
EP3800617B1 (fr) Une méthode mise en oeuvre par ordinateur pour enregistrer des images en basse dimension avec une image en haute dimension, une méthode pour former un réseau neuronal artificiel utile pour trouver des points de repère dans des images en basse dimension, un programme informatique et un système pour enregistrer des images en basse dimension avec une image en haute dimension
US20250315964A1 (en) Registration projection images to volumetric images
US9240046B2 (en) Method and system to assist 2D-3D image registration
Wagner et al. 4D interventional device reconstruction from biplane fluoroscopy
CN109472835B (zh) 处理医学图像数据的方法和医学图像数据的图像处理系统
US20110052026A1 (en) Method and Apparatus for Determining Angulation of C-Arm Image Acquisition System for Aortic Valve Implantation
JP2009536857A (ja) 画像誘導放射線医療のための画像の変形可能な登録
CN112862833A (zh) 血管分段方法、电子装置和存储介质
EP4631028A1 (fr) Système et procédé de génération d'image pour guidage de remplacement de valve aortique transcathéter
Penney et al. Postoperative calculation of acetabular cup position using 2-D–3-D registration
Toth et al. 3D/2D Registration with superabundant vessel reconstruction for cardiac resynchronization therapy
Tran et al. 3D catheter shape reconstruction using electromagnetic and image sensors
EP3828836A1 (fr) Procédé et système de traitement de données pour la fourniture d'une image bidimensionnelle dépliée d'au moins une structure tubulaire
JP2020099696A (ja) 再配向手術のための術前計画：模擬ｘ線を用いた表面モデルフリーアプローチ
Wagner et al. A dynamic model‐based approach to motion and deformation tracking of prosthetic valves from biplane x‐ray images
EP4386683A1 (fr) Appareil pour aider un utilisateur à planifier et exécuter une procédure chirurgicale sur la base d'une image
US10417765B2 (en) Adaptive segmentation for rotational C-arm computed tomography with a reduced angular range
WO2024126458A1 (fr) Appareil pour aider un utilisateur dans une planification basée sur une image et effectuer une procédure chirurgicale
Malinda et al. Lumbar vertebrae synthetic segmentation in computed tomography images using hybrid deep generative adversarial networks
JP6472112B2 (ja) 投影ｘ線画像の改善された視覚化
CN120500705A (zh) 用于训练基于ai的ct到cbct配准和cbct分割的cbct模拟
CN112790778B (zh) 采集错误对齐
Kamiya et al. Usefulness of Isosurface Geometric Measurement on Volume‐Rendered Images for Quantitative Measurements of Complex Cardiac Anatomical Features
Schwerter et al. A novel approach for a 2D/3D image registration routine for medical tool navigation in minimally invasive vascular interventions
JP2023549954A (ja) 医用撮像の際に患者の位置を調整するための誘導

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20250714

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)