WO2023006021A1 - 一种配准方法和系统 - Google Patents

一种配准方法和系统 Download PDF

Info

Publication number
WO2023006021A1
WO2023006021A1 PCT/CN2022/108552 CN2022108552W WO2023006021A1 WO 2023006021 A1 WO2023006021 A1 WO 2023006021A1 CN 2022108552 W CN2022108552 W CN 2022108552W WO 2023006021 A1 WO2023006021 A1 WO 2023006021A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
target
dimensional
transformation matrix
pose
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/108552
Other languages
English (en)
French (fr)
Inventor
付春萌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan United Imaging Healthcare Surgical Technology Co Ltd
Original Assignee
Wuhan United Imaging Healthcare Surgical Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202110875674.6A external-priority patent/CN113570648B/zh
Priority claimed from CN202210147197.6A external-priority patent/CN114529594B/zh
Application filed by Wuhan United Imaging Healthcare Surgical Technology Co Ltd filed Critical Wuhan United Imaging Healthcare Surgical Technology Co Ltd
Priority to EP22848626.2A priority Critical patent/EP4365838B1/en
Publication of WO2023006021A1 publication Critical patent/WO2023006021A1/zh
Priority to US18/427,758 priority patent/US20240221190A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10116—X-ray image
    • G06T2207/10124—Digitally reconstructed radiograph [DRR]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30008—Bone
    • G06T2207/30012—Spine; Backbone

Definitions

  • This specification relates to the field of computer-aided surgery, and in particular to a method and system for registering a three-dimensional image taken before operation and a two-dimensional image taken during operation.
  • 3D images of patients before surgery such as CT (Computed Tomography, computerized tomography) images or MRI (Magnetic Resonance Imaging, magnetic resonance imaging) images, and make surgical plans based on 3D images .
  • CT Computerized Tomography
  • MRI Magnetic Resonance Imaging, magnetic resonance imaging
  • the operation plan formulated in the preoperative 3D image is mapped to the operation space to achieve the purpose of guiding the operation process during the operation.
  • multiple 2D images can be taken during the operation, and the 2D-3D (2D-3D) registration of the preoperative 3D images and 2D images can be performed to indirectly establish the relationship between the 3D image space and the surgical space. Mapping relations.
  • the registration method includes: acquiring a three-dimensional image of the target object taken before the operation and at least one two-dimensional image taken during the operation; based on the at least one two-dimensional image, performing pose transformation on the three-dimensional image to obtain a registered three-dimensional image , and the first transformation matrix between the three-dimensional image coordinate system corresponding to the three-dimensional image and the operation space coordinate system corresponding to the operation; determine the two-dimensional corresponding to the target part of the target object in the at least one two-dimensional image
  • the target image determining the 3D target image corresponding to the target part in the registered 3D image; based on the 3D target image and the 2D target image in each 2D image, positioning the registered 3D image pose transformation to optimize the first transformation matrix to obtain a target transformation matrix.
  • the 3D image and the at least one 2D image may be down-sampled based on a preset multiple to obtain a down-sampled 3D image and at least one down-sampled 2D image; Downsampling the pose of the 3D image to obtain an adjusted downsampled 3D image; for each of the 2D images, project the adjusted downsampled 3D image based on the shooting pose of the 2D image to obtain the corresponding the first projected image; in response to at least one of the first projected image and the at least one downsampled two-dimensional image satisfying a first preset condition, the adjusted downsampled three-dimensional image is adjusted according to the preset multiple Up-sampling is performed to obtain the registered 3D image; and the first conversion matrix is determined based on a pose transformation process from the 3D image to the registered 3D image.
  • the first similarity may be determined based on at least one of the first projected image and the at least one down-sampled two-dimensional image; in response to the first similarity being greater than a similarity threshold, determining the The at least one first projected image and the at least one downsampled 2D image satisfy a first preset condition.
  • the adjusted down-sampled 3D image after projecting the adjusted down-sampled 3D image based on the shooting pose of the 2D image for each of the 2D images to obtain the corresponding first projected image, it may also respond to If at least one of the first projected image and the at least one down-sampled 2D image do not meet the first preset condition, adjusting the pose of the adjusted down-sampled 3D image according to the preset step size , so as to repeat the above process of obtaining the first projected image corresponding to each two-dimensional image until at least one of the first projected image and the at least one downsampled two-dimensional image satisfy the first preset condition.
  • the adjusted down-sampled 3D image in response to at least one of the first projected image and the at least one downsampled 2D image satisfying the first preset condition, and the preset multiple does not satisfy the second preset condition If the conditions are set, the adjusted down-sampled 3D image can be up-sampled according to the preset multiple to update the 3D image; The updated 3D image and the at least one 2D image are down-sampled to repeat the process of obtaining the first projection image corresponding to each 2D image until at least one of the first projection image and the at least one The down-sampled 2D image satisfies the first preset condition, and the preset multiple satisfies the second preset condition.
  • the second transformation matrix between the two-dimensional imaging device coordinate system and the operation space coordinate system can be obtained; based on the pose transformation process from the three-dimensional image to the registered three-dimensional image, the obtained A third transformation matrix between the three-dimensional image coordinate system and the two-dimensional imaging device coordinate system; the first transformation matrix is obtained based on the second transformation matrix and the third transformation matrix.
  • the 3D target image corresponding to the target part may be determined in the registered 3D image. For each of the at least one two-dimensional image: obtaining a fourth transformation matrix between the two-dimensional image coordinate system corresponding to the two-dimensional image and the operation space coordinate system; based on the three-dimensional target image, the The first transformation matrix and the fourth transformation matrix determine the 2D target image in the 2D image.
  • the three-dimensional coordinates of the representative point of the target part in the three-dimensional image coordinate system and the size parameters of the target part can be determined; based on the three-dimensional coordinates, the The first transformation matrix and the fourth transformation matrix determine the two-dimensional coordinates of the representative point in the two-dimensional image coordinate system; based on the two-dimensional coordinates and the size parameter, determine in the two-dimensional image The 2D target image.
  • the second transformation matrix between the coordinate system of the two-dimensional imaging device and the coordinate system of the operation space can be obtained; the second transformation matrix between the coordinate system of the two-dimensional imaging device and the coordinate system of the two-dimensional image can be obtained Five transformation matrices; obtaining the fourth transformation matrix based on the second transformation matrix and the fifth transformation matrix.
  • the three-dimensional target image may be projected based on the shooting pose of the two-dimensional image to obtain a corresponding second projected image; based on at least one of the second Determining a second similarity between the projected image and the at least one two-dimensional target image; performing pose adjustment on the registered three-dimensional image based on the second similarity to optimize the first transformation matrix to obtain the target transformation matrix .
  • One of the embodiments of this specification provides a registration system, including an image acquisition module, a pose transformation module, a target image determination module, and a target matrix determination module;
  • the image acquisition module is used to acquire a three-dimensional image of a target object taken before surgery and at least one two-dimensional image taken during the operation;
  • the pose transformation module is used to perform pose transformation on the three-dimensional image based on the at least one two-dimensional image to obtain a registered three-dimensional image, and the three-dimensional image corresponding to the three-dimensional image A first transformation matrix between the image coordinate system and the operation space coordinate system corresponding to the operation;
  • the target image determination module is used to determine the two-dimensional target corresponding to the target part of the target object in the at least one two-dimensional image Image, determining the 3D target image corresponding to the target part in the registered 3D image;
  • the target matrix determination module is used to calculate the 3D target image based on the 3D target image and the 2D target image in each 2D image performing pose transformation on the registered 3D image to
  • One of the embodiments of the present specification provides a computer-readable storage medium, the storage medium stores computer instructions, and after the computer reads the computer instructions in the storage medium, the computer executes the registration method.
  • the rough registration between the 2D image and the 3D image is realized by performing pose transformation on the 3D image taken before the operation based on the 2D image taken during the operation; based on the 2D target corresponding to the target part Image and 3D target image, continue to perform pose transformation on the coarsely registered 3D image, and realize fine registration between 2D image and 3D image.
  • the 2D target image corresponding to the target part can be determined from the 2D image based on the 3D target image. Because 2D images usually have lower imaging resolution than 3D images, and 2D images may contain surgical devices, direct segmentation of target parts on 2D images may not yield good segmentation results. By determining the two-dimensional target image based on the three-dimensional target image, the accuracy of target part segmentation can be improved.
  • Fig. 1 is a schematic diagram of an application scenario of a registration system according to some embodiments of this specification
  • FIG. 2 is a block diagram of a processing device according to some embodiments of the present specification.
  • Fig. 3 is an exemplary flowchart of a registration method according to some embodiments of the present specification.
  • Fig. 4 is an exemplary flow chart of a process for coarse registration of a 3D image and at least one 2D image according to some embodiments of the present specification
  • Fig. 5 is an exemplary flowchart of a process for generating a 3D target image and a 2D target image according to some embodiments of the present specification
  • Fig. 6 is an exemplary flow chart of the process of optimizing the first transformation matrix to obtain the target transformation matrix according to some embodiments of the present specification
  • FIG. 7A and 7B are schematic diagrams of a two-dimensional image of a first shooting pose according to some embodiments of the present specification.
  • 8A and 8B are schematic diagrams of a two-dimensional image of a second shooting pose according to some embodiments of the present specification.
  • Fig. 9 is a schematic diagram of a registration method according to some embodiments of the present specification.
  • Fig. 10 is a schematic diagram of a registration method according to some embodiments of the present specification.
  • Fig. 11 is a schematic structural diagram of a registration system according to some embodiments of the present specification.
  • system means for distinguishing different components, elements, parts, parts or assemblies of different levels.
  • the words may be replaced by other expressions if other words can achieve the same purpose.
  • Fig. 1 is a schematic diagram of an application scenario of a registration system according to some embodiments of the present specification.
  • a registration system 100 may include a medical imaging device 110 , a processing device 120 , a storage device 130 , a terminal 140 , and a network 150 .
  • the registration system 100 can realize two-dimensional-three-dimensional registration of images acquired by the medical imaging device 110 through the processing device 120 implementing the methods and/or processes disclosed in this specification.
  • the 2D-3D registration refers to the registration of the 3D image of the target object taken before the operation and the 2D image taken during the operation.
  • the pose of the 3D image is adjusted (e.g., by means of translation and/or rotation) such that the pose of the target object in the adjusted 3D image (i.e., the registered 3D image) is the same as the photographed
  • the poses and poses of the two-dimensional images are the same.
  • the mapping relationship between the 3D image space (also called the 3D image coordinate system) and the surgical space (also called the surgical space coordinate system) corresponding to the 3D image can be established, so that the 3D image based on the 3D image
  • the established operation plan is accurately transformed into the operation space, so as to ensure the smooth progress of the operation and improve the effect of the operation.
  • the medical imaging device 110 refers to a device that reproduces the internal structure of a target object (eg, a human body, an animal, etc.) as an image by using different methods in medicine.
  • the medical imaging device 110 can be any medical device capable of imaging or treating a designated body part of a patient, for example, a computerized tomography (Computed Tomography, CT) device, a magnetic resonance (Magnetic Resonance Imaging, MRI) Equipment, Positron Emission Computed Tomography (PET) equipment, Direct Digital Radiography (DDR) equipment, X-ray imaging equipment, etc.
  • CT computerized tomography
  • MRI Magnetic Resonance Imaging
  • PET Positron Emission Computed Tomography
  • DDR Direct Digital Radiography
  • X-ray imaging equipment X-ray imaging equipment
  • the medical imaging device 110 can be used to capture 2D and/or 3D images of the target object.
  • the medical imaging device 110 may include a first medical imaging device and a second medical imaging device.
  • the first medical imaging device can be used to acquire three-dimensional images
  • the second medical imaging device can be used to shoot two-dimensional images.
  • the first imaging device may be a CT device or an MRI device, which can be used to take a three-dimensional image of the target object before the operation.
  • the second imaging device can be an X-ray imaging device, which can be used to take a two-dimensional image of the target object during the operation.
  • the 3D image can be used to formulate a surgical plan (eg, surgical path), and the 2D image can be used to register with the 3D image to obtain a transformation matrix, and the transformation matrix is used to map the surgical plan into the surgical space.
  • medical imaging device 110 may comprise a single device that may be capable of capturing both 3D and 2D images.
  • data and/or information such as images captured by the medical imaging device 110 may be stored in the storage device 130 .
  • the medical imaging device 110 may receive instructions and the like sent by the doctor through the terminal 140, and perform related operations according to the instructions, such as irradiation and imaging.
  • the medical imaging device 110 can exchange data and/or information with other components in the registration system 100 (eg, the processing device 120 , the storage device 130 , and the terminal 140 ) through the network 150 .
  • the medical imaging device 110 can be directly connected with other components in the registration system 100 .
  • one or more components in registration system 100 may be included within medical imaging device 110 .
  • the processing device 120 may process data and/or information obtained from other devices or system components, and execute the registration methods shown in some embodiments of this specification based on these data, information and/or processing results, so as to complete one or more functions described in some embodiments of this specification. For example, the processing device 120 may perform 2D-3D registration based on the 2D image and the 3D image collected by the medical imaging device 110, so as to determine the transformation relationship between the 3D image space and the operation space. In some embodiments, the processing device 120 may send the processed data, for example, the conversion matrix between coordinate systems, projection images, etc., to the storage device 130 for storage.
  • the processing device 120 can obtain pre-stored data and/or information from the storage device 130, for example, two-dimensional images, three-dimensional images, etc., so as to execute the registration method shown in some embodiments of this specification , for example, perform 2D-3D registration, etc.
  • processing device 120 may include one or more sub-processing devices (eg, a single-core processing device or a multi-core multi-core processing device).
  • processing device 120 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), an application specific instruction processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor ( DSP), field programmable gate array (FPGA), programmable logic circuit (PLD), controller, microcontroller unit, reduced instruction set computer (RISC), microprocessor, etc. or any combination of the above.
  • CPU central processing unit
  • ASIC application specific integrated circuit
  • ASIP application specific instruction processor
  • GPU graphics processing unit
  • PPU physical processing unit
  • DSP digital signal processor
  • FPGA field programmable gate array
  • PLD programmable logic circuit
  • controller microcontroller unit, reduced instruction set computer (RISC), microprocessor, etc. or any combination of the above.
  • Storage device 130 may store data or information generated by other devices.
  • the storage device 130 may store data and/or information captured by the medical imaging device 110 , such as two-dimensional images, three-dimensional images, and the like.
  • the storage device 130 may store data and/or information processed by the processing device 120 , for example, a conversion matrix between coordinate systems, a projected image, and the like.
  • the storage device 130 may include one or more storage components, and each storage component may be an independent device or a part of other devices. Storage can be local or via the cloud.
  • the terminal 140 can control the operation of the medical imaging device 110 .
  • a doctor may issue an operation instruction to the medical imaging device 110 through the terminal 140, so that the medical imaging device 110 completes a specified operation, for example, imaging a specified body part of a patient.
  • the terminal 140 can instruct the processing device 120 to execute the registration method as shown in some embodiments of this specification.
  • the terminal 140 can receive the registered three-dimensional image from the processing device 120, so that the user can perform effective and targeted examination and/or treatment on the patient.
  • the terminal 140 may be one of a mobile device 140-1, a tablet computer 140-2, a laptop computer 140-3, a desktop computer and other devices with input and/or output functions or any of them. combination.
  • Network 150 may connect various components of the system and/or connect parts of the system with external resources. Network 150 enables communication between the various components and with other components outside the system, facilitating the exchange of data and/or information.
  • one or more components in the registration system 100 eg, the medical imaging device 110 , the processing device 120 , the storage device 130 , and the terminal 140
  • the network 150 may be any one or more of a wired network or a wireless network.
  • the processing device 120 may be based on a cloud computing platform, such as public cloud, private cloud, community and hybrid cloud, and the like. However, these changes and modifications do not depart from the scope of this specification.
  • Fig. 2 is a block diagram of a processing device according to some embodiments of the present specification.
  • the processing device 120 may include an image acquisition module 210 , a pose transformation module 220 , a target image determination module 230 and a target matrix determination module 240 .
  • the image acquisition module 210 may be used to acquire a three-dimensional image of a target object (eg, a human body, an animal, etc.) taken before the operation and at least one two-dimensional image taken during the operation.
  • a target object eg, a human body, an animal, etc.
  • the pose transformation module 220 can be used to perform pose transformation on the 3D image based on at least one 2D image to obtain a registered 3D image, and the first coordinate system between the 3D image coordinate system corresponding to the 3D image and the operation space coordinate system corresponding to the operation. transformation matrix.
  • the target image determination module 230 can be used to determine the 2D target image corresponding to the target part of the target object in at least one 2D image, and determine the 3D target image corresponding to the target part in the registered 3D image.
  • the target site may include at least one of a target spine, a target hip joint, a target elbow joint, a target knee joint, and a target finger joint.
  • the target matrix determination module 240 can be used to perform pose transformation on the registered 3D image based on the 3D target image and the 2D target image in each 2D image, so as to optimize the first transformation matrix and obtain the target transformation matrix.
  • the processing device 120 may further include a further route information acquisition module and a guidance information generation module (not shown in FIG. 2 ).
  • the path information acquiring module can be used to acquire surgical path information corresponding to the three-dimensional image.
  • the guidance information generation module can be used to convert the operation path information into the operation space coordinate system based on the target transformation matrix, so as to generate operation guidance information.
  • Fig. 3 is an exemplary flowchart of a registration method according to some embodiments of the present specification.
  • process 300 may be performed by registration system 100 .
  • the process 300 may be stored in a storage device (eg, the memory 130 ) in the form of an instruction set (eg, an application program).
  • the processing device 120 eg, one or more modules shown in FIG. 2
  • the process 300 may include the following steps.
  • Step 310 acquiring a 3D image of the target subject taken before the operation and at least one 2D image taken during the operation.
  • step 310 may be performed by the image acquisition module 210 .
  • a target object refers to an object undergoing surgery or inspection, for example, a living body, a phantom, and the like.
  • a target object may include a human body or a portion thereof.
  • a target object may include a patient requiring surgery or a portion thereof (eg, spine, hip, elbow, knee, finger, etc.).
  • the target object includes a target site.
  • the target part refers to the organ, tissue and other body regions that need to undergo surgery and/or inspection in the target object, for example, the target spine, target hip joint, target elbow joint, target knee joint, and target finger joint.
  • a three-dimensional image refers to an image obtained by shooting a target object through a three-dimensional medical imaging device, for example, a CT image, an MRI image, a PET image, and the like.
  • the image acquisition module 210 can scan the target object with a 3D medical imaging device before the operation to obtain a 3D image.
  • the three-dimensional image can be used for inspection, diagnosis, preoperative preparation and intraoperative path positioning, etc.
  • a two-dimensional image refers to an image obtained by shooting a human body through two-dimensional medical imaging equipment, for example, a DDR image, an X-ray machine image, and the like.
  • the image acquisition module 210 may scan the target object with a two-dimensional medical imaging device during surgery to obtain a two-dimensional image.
  • the 2D image can be used to register the 2D image with the 3D image, so that the pose of the target object in the 3D image is consistent with the pose of the target object in the operation, and the transformation matrix between the 3D image space and the operation space is obtained, Transform the surgical plan into surgical space using a transformation matrix.
  • the 2D images may include at least two, wherein the shooting angles of these 2D images may be different.
  • the 2D image may include a frontal 2D image and a lateral 2D image.
  • the radiation source When taking anteroposterior 2D images, the radiation source is positioned directly in front of the target object for irradiation, and when shooting lateral 2D images, the radiation source is positioned for irradiation on the side of the target object.
  • there are at least two 2D images among the 2D images and the angle between each two 2D images is within a preset angle range.
  • the angle between two two-dimensional images refers to the angle between the two groups of rays emitted by the two-dimensional medical imaging equipment when shooting the two two-dimensional images, and is also the angle between the imaging planes of the two two-dimensional images.
  • the preset angle range may be 50 degrees to 130 degrees.
  • the image acquisition module 210 may also acquire the 3D image of the target object taken before the operation and at least one 2D image taken during the operation in other ways, for example, from various data sources such as memory.
  • Step 320 based on at least one 2D image, perform pose transformation on the 3D image to obtain a registered 3D image, and a first transformation matrix between the 3D image coordinate system corresponding to the 3D image and the operation space coordinate system corresponding to the operation.
  • step 320 may be performed by the pose transformation module 220 .
  • the pose adjustment of the 3D image in step 320 is performed based on the analysis of the whole 3D image, and this process can be regarded as a process of rough registration of the 3D image.
  • the pose (ie, position and attitude) of the target object in the 3D image can be adjusted to be roughly consistent with the pose when the 2D image was taken (ie, the current pose during surgery).
  • the registered 3D image refers to the image obtained by performing rough registration on the 3D image.
  • the first transformation matrix refers to a transformation matrix for transformation between the three-dimensional image coordinate system corresponding to the three-dimensional image obtained through the rough registration result and the operation space coordinate system corresponding to the operation.
  • the surgical space coordinate system corresponding to the surgery is also called the surgical space or the surgical coordinate system, which is a coordinate system generated with a preset position as a reference point for guiding surgery.
  • the surgical space coordinate system may include a coordinate system based on an optical tracking system (Optical Tracking System).
  • OTS can photograph the entire surgical scene through optical imaging technology.
  • an OTS system may include binocular cameras.
  • the surgical space coordinate system can take a certain point on the binocular camera as the origin, the horizontal direction as the x-axis, the vertical direction as the z-axis, and the front-back direction as the y-axis.
  • the 3D image coordinate system corresponding to the 3D image is also called the 3D image space, which is a 3D coordinate system established on the basis of the 3D image.
  • the 3D image space which is a 3D coordinate system established on the basis of the 3D image.
  • the target object in the 3D image is a patient
  • the center point of the patient can be used as the origin
  • the left and right directions of the patient can be used as the x-axis
  • the front-to-back direction of the patient can be used as the y-axis
  • the patient's head and feet can be used as the z-axis to establish a 3D image coordinate system .
  • the processing device 120 may adjust the pose of the 3D image to obtain the adjusted 3D image. Further, the processing device 120 may project the adjusted 3D images in different projection poses to obtain a first projection image corresponding to each 2D image. Wherein, for the corresponding 2D image and the first projection image, the shooting pose of the 2D image is the same as the projection pose of the first projection image. Wherein, the shooting pose refers to the pose of the two-dimensional medical imaging device when shooting a two-dimensional image, and the projection pose refers to the pose of the virtual shooting device when the projected image is obtained through projection.
  • the pose of the C-arm in the surgical coordinate system (that is, the position coordinates of the ray source and the center of the imaging plane of the C-arm in the coordinate system) can be obtained through the OTS device.
  • the virtual C-arm The position coordinates of the ray source of the arm and the center of the imaging plane are also set to be consistent with those in the surgical coordinate system, so that the projection pose and the shooting pose can be guaranteed to be the same.
  • the processing device 120 may determine a first similarity based on each 2D image and the corresponding first projection image, and judge whether to continue adjusting the pose of the adjusted 3D image according to the first similarity.
  • the similarity between each two-dimensional image and the corresponding first projected image may be determined, and the first similarity may be obtained based on all the determined similarities.
  • the first similarity may be one of the sum, average, maximum value, etc. of all determined similarities. If the first similarity is small, continue to adjust the pose of the adjusted 3D image to update the adjusted 3D image, and repeat the above process of determining the degree of similarity until it is judged based on the degree of similarity that the adjusted The pose of the 3D image is adjusted to obtain a registered 3D image.
  • steps 410-460 which will not be repeated here.
  • the first transformation matrix may be determined based on the registered 3D images. Specifically, in the rough registration process, the pose adjustment of the 3D image can be performed based on the rotation step and the translation step, and the first transformation matrix can be determined according to the total rotation step and the total translation step in the whole adjustment process. For more details on how to determine the first transformation matrix based on the registered 3D image, refer to the relevant description of step 470, and details will not be repeated here.
  • the subsequent step 330 and step 340 can be used to fine-register the registered 3D image and the 2D image. Different from coarse registration, the pose adjustment of the registered 3D image in fine registration is based on the target part.
  • Step 330 Determine the 2D target image corresponding to the target part of the target object in at least one 2D image, and determine the 3D target image corresponding to the target part in the registered 3D image. In some embodiments, step 330 may be performed by the target image determining module 230 .
  • the 2D target image refers to a 2D image including a target part, for example, a 2D image of a target spine.
  • the 3D target image refers to a 3D image including a target part, for example, a 3D image of a target spine.
  • the 2D target image and/or the 3D target image may only include the image of the target part (or a small number of its surrounding parts), while the original 2D image and 3D image may include other body parts other than the target part. parts.
  • the target site is the L3 spine
  • the 2D target image and the 3D target image only include the L3 spine
  • the original 3D image and 2D image include the L1 spine, L2 spine, L3 spine, L4 spine and L5 spine.
  • the 2D target image and the 3D target image corresponding to the target part of the target object may be respectively determined in at least one 2D image and the registered 3D image.
  • a first image segmentation algorithm for example, a 2D image segmentation model
  • a second image segmentation algorithm for example, a 3D image segmentation model
  • a 3D target image including a target part may be determined in coordinating the 3D images.
  • the fourth transformation matrix between the two-dimensional image coordinate system corresponding to the two-dimensional image and the surgical space coordinate system can be obtained, and based on the three-dimensional target image, the first transformation matrix and the fourth transformation matrix, determine the two-dimensional target image. That is to say, the 3D target image may be determined first, and then the 2D target image may be determined based on the 3D target image.
  • the two-dimensional image coordinate system is also called the two-dimensional image space, which is a two-dimensional coordinate system established on the basis of two-dimensional images.
  • the upper left corner of the 2D image can be taken as the origin
  • the direction from left to right of the 2D image can be taken as the positive direction of the x-axis
  • the direction from top to bottom of the 2D image can be taken as the positive direction of the y-axis.
  • Step 340 based on the 3D target image and the 2D target image in each 2D image, perform pose transformation on the registered 3D image to optimize the first transformation matrix to obtain the target transformation matrix.
  • step 340 may be performed by the target matrix determination module 240 .
  • the target transformation matrix refers to the transformation matrix between the three-dimensional image coordinate system corresponding to the three-dimensional image obtained according to the fine registration result and the operation space coordinate system corresponding to the operation.
  • the 3D target image may be projected based on the shooting pose of the 2D image to obtain a corresponding second projected image.
  • the second similarity can be determined based on at least one second projected image and at least one two-dimensional target image.
  • a pose adjustment can be performed on the registered 3D image based on the second similarity to optimize the first transformation matrix to obtain a target transformation matrix.
  • the processing device 120 may further acquire surgical path information corresponding to the 3D image.
  • the operation path information may include the coordinates in the three-dimensional image coordinate system of the target point of the target object that the surgical instrument passes through during the operation.
  • the processing device 120 may convert the surgical path information into the surgical space coordinate system based on the obtained target transformation matrix, so as to generate surgical guidance information.
  • the coordinates of the target point in the three-dimensional image coordinate system can be transformed into coordinates in the surgical space coordinate system as surgical guidance information.
  • the surgical guidance information may include a judgment result of whether the position of the surgical instrument is correct and/or prompt information for guiding the moving position of the surgical instrument, and the like.
  • the registration of the 3D image and the transformation matrix between the coordinate system of the 3D image and the coordinate system of the operation space are obtained to realize the conversion of the 3D image and the 2D image.
  • Coarse registration between two-dimensional images Further, based on the 2D target image and 3D target image corresponding to the target part, continue to perform pose transformation on the coarsely registered 3D image, optimize the transformation matrix, and realize the fine registration of the 2D image and the 3D image.
  • the accuracy of the registration can be improved, thereby improving the accuracy of the determined target transformation matrix between the three-dimensional image space and the operation space, and the accuracy of the operation guidance.
  • the 2D-3D registration method in this manual can be implemented without human intervention or with a small amount of human intervention, without human-computer interaction (such as manual selection of anatomical points in 3D images and 2D images), which greatly simplifies At the same time, it also avoids errors caused by a large number of human-computer interactions, thereby improving the accuracy of registration between 3D images and 2D images.
  • Fig. 4 is an exemplary flowchart of a process for performing rough registration on a 3D image and at least one 2D image according to some embodiments of the present specification.
  • the pose transformation module 220 can use the method shown in process 400 to perform pose transformation on the 3D image based on at least one 2D image to obtain a registered 3D image, and determine the 3D image coordinate system corresponding to the 3D image and The first transformation matrix between the surgical space coordinate systems corresponding to the surgery.
  • the process 400 can be used to implement step 320 shown in FIG. 3 . As shown in Fig. 4, the process 400 may include the following steps.
  • Step 410 Downsampling the 3D image and at least one 2D image based on a preset multiple to obtain a downsampled 3D image and at least one downsampled 2D image.
  • the pose transformation module 220 may down-sample the 3D image and at least one 2D image according to a preset multiple, that is, reduce the 3D image and the 2D image according to a preset multiple.
  • the preset multiple can be any value such as 8, 4, 2, etc.
  • Step 420 adjust the pose of the downsampled 3D image based on the preset step size, and obtain the adjusted downsampled 3D image.
  • the preset step size may include a preset rotation step size and/or a preset translation step size.
  • the pose transformation module 220 may rotate the downsampled 3D image according to a preset rotation step, and/or translate the downsampled 3D image according to a preset translation step.
  • the preset rotation step size can be any angle such as 10°, 5°, 2°, etc.
  • the preset translation step can be any length such as 20mm, 10mm, 5mm, etc.
  • the preset multiple and the preset step size may be the default settings of the registration system 100, or be manually set by the user, or be determined by the pose transformation module 220 according to actual needs.
  • the preset multiple and the preset step size have a specific corresponding relationship. For example, the preset rotation step corresponding to the preset multiple of 8 is 10°, and the preset translation step is 20mm; the preset rotation step corresponding to the preset multiple of 4 is 5°, and the preset translation step is 10mm; Set the preset rotation step corresponding to preset multiple 2 to 5°, and the preset translation step to 5mm.
  • Step 430 for each 2D image, project the adjusted down-sampled 3D image based on the shooting pose of the 2D image to obtain a corresponding first projected image.
  • the pose transformation module 220 can project the adjusted down-sampled three-dimensional image at a specific projection pose to obtain its corresponding first projected image, wherein the first projected image
  • the projection pose of is the same as the shooting pose of the 2D image.
  • the down-sampling process does not change the shooting pose of the 2D image. Therefore, the shooting pose of the 2D image is the same as the shooting pose of the corresponding down-sampled 2D image.
  • the projection of the first projected image corresponding to the 2D image The pose and the shooting pose of the downsampled 2D image are also the same.
  • the 2D image may include an orthographic 2D image and a lateral 2D image
  • the pose transformation module 220 may respectively project the adjusted down-sampled 3D image from the anterior and lateral to obtain the orthographic first A projected image and a first lateral projected image.
  • the pose transformation module 220 may project the adjusted down-sampled 3D image into the first projection image by means of digitally reconstructed radiograph (Digitally Reconstructed Radiograph, DRR) projection or the like.
  • digitally reconstructed radiograph Digitally Reconstructed Radiograph, DRR
  • the first projected image corresponding to each two-dimensional image it may be determined whether at least one first projected image and at least one downsampled two-dimensional image satisfy a preset condition. In some embodiments, it may be determined that at least one first projected image and at least one downsampled 2D image satisfy a preset condition through the following steps 440 and 450 .
  • Step 440 Determine a first similarity based on at least one first projected image and at least one downsampled 2D image.
  • the first similarity is used to measure the degree of similarity between the first projected image and the corresponding downsampled 2D image.
  • the pose transformation module 220 may determine the similarity between each first projected image and the corresponding downsampled 2D image in various ways, and obtain the first similarity based on all the determined similarities.
  • the first similarity may be one of a sum, an average value, a maximum value, and the like of all determined similarities.
  • the similarity between the first projected image and its corresponding downsampled 2D image can be determined by various similarity algorithms.
  • Exemplary similarity algorithms may include mutual information, pattern strength, gradient difference, and the like.
  • MI Mutual Information
  • Mutual Information is used to represent the statistical correlation between two systems, or the amount of information contained in another system in one system.
  • the mutual information between two images can be expressed by the following formula:
  • p(x), p(y) represent the marginal probability distribution of the two images to be registered; p(x, y) represents the joint probability distribution of the two images to be registered; S represents the Mutual information value between two images.
  • Pattern Intensity is calculated based on the difference image between images to be registered, where the object to be registered is called a "pattern". Specifically, two images are subtracted to obtain I dif , and when the registration state is reached, the pattern to be registered in I dif will disappear, and the intensity of the pattern will be minimized.
  • the pattern strength between two images can be expressed by the following formula:
  • the gradient difference (GD, Gradient Difference) is also realized based on the difference image, but the difference image is obtained from the gradient image.
  • the two images are processed by horizontal and vertical Sobel operators to generate four gradient images, which respectively represent the change rates of the two registered images in the directions of two orthogonal coordinate axes.
  • the gradient difference measure between two images can be expressed by the following formula:
  • dI fl /di represents the gradient image of the 2D image to be registered in the horizontal direction
  • dI DRR /di represents the gradient image of the DRR image in the horizontal direction
  • I diffV represents the gradient image of the 2D image to be registered and the gradient of the DRR image
  • the image of image subtraction; s, A v , A h represent the weight of the function; G(s) represents the final gradient difference;
  • I diffV (I,j) represents the subtraction image of the horizontal gradient images of the two images to be registered
  • I diffH (I, j) represents the pixel value at the coordinate (I, j) of the subtraction image of the vertical gradient image of the two images to be registered.
  • the DRR projection method is used to project the 3D image into 2D images with the same number as the 2D images to be registered, and to calculate the similarity, so that the similarity can be obtained more accurately, and the registration can be improved. Accuracy.
  • Step 450 in response to the first similarity being greater than the similarity threshold, determine that at least one first projected image and at least one downsampled 2D image satisfy a first preset condition.
  • the similarity threshold can have any value.
  • the similarity threshold may be a default setting of the registration system 100, or be manually set by a user, or be determined by the pose transformation module 220 according to actual needs.
  • the similarity threshold has a corresponding relationship with a preset multiple. For example, when there are two 2D images, the similarity threshold corresponding to the preset multiple of 8 can be set to 0.2, the similarity threshold corresponding to the preset multiple of 4 is 0.4, and the similarity threshold corresponding to the preset multiple of 2 can be set to 0.6.
  • the first preset condition may include that the first similarity is greater than a similarity threshold or the like.
  • the first similarity when the first similarity is greater than a similarity threshold, it may be determined that the first projected image and the corresponding downsampled 2D image satisfy a first preset condition. When it is determined that the first projected image and the corresponding downsampled 2D image satisfy the first preset condition, continue to execute step 460 . If not, repeat step 420-step 440 until the first preset condition is met.
  • step 420 determines the first similarity based on the obtained at least one first projected image and at least one downsampled 2D image, and repeat the above process until the first similarity is greater than the similarity threshold.
  • Step 460 in response to at least one first projected image and at least one downsampled 2D image satisfying a first preset condition, upsampling the adjusted downsampled 3D image according to a preset multiple to obtain a registered 3D image .
  • the up-sampling operation can enlarge the adjusted down-sampled 3D image so that the resulting registered 3D image has the same size as the original 3D image.
  • the pose transformation module 220 may further determine whether the preset multiple satisfies the second preset condition . If the preset multiple satisfies the second preset condition, the pose transformation module 220 may upsample the adjusted downsampled 3D image, and use the upsampled image as a registered 3D image.
  • the second preset condition may include a preset value that the preset multiple is equal to the multiple, and the like. Wherein, the preset value of the multiple is a preset smaller value.
  • the preset multiplier is greater than the preset value of the multiplier, it means that the preset multiplier is large, and the size difference between the downsampled 3D image and the 3D image is large. It is necessary to reduce the preset multiplier according to the preset downsampling step size to reduce the downsampled 3D image The size gap with the 3D image.
  • the adjusted down-sampled 3D image may be up-scaled. Sampling to obtain an updated 3D image.
  • the updated 3D image has a different pose from the original 3D image. Further, the preset multiple can be reduced, and steps 410-450 are performed again.
  • the updated 3D image and the corresponding 2D image can be down-sampled based on the reduced preset multiple, and the process of obtaining the first projected image corresponding to each 2D image can be repeated until at least one first projection image
  • the projected image and the corresponding at least one down-sampled 2D image satisfy a first preset condition, and the preset multiple satisfies the second preset condition.
  • the preset multiple may be reduced according to the preset down-sampling step. Specifically, the preset multiple may be divided by the down-sampling step to obtain the reduced preset multiple.
  • the preset downsampling step size can be set according to requirements. For example, the preset down-sampling step size is 2, and the down-sampling process is performed on the preset multiple according to the preset down-sampling step size, which is to reduce the preset multiple by 2 times.
  • the adjusted down-sampled 3D image already satisfies the first preset condition. Therefore, continuing to adjust the pose based on the adjusted down-sampled 3D image can reduce the duration of rough registration. Restore the size of the adjusted down-sampled 3D image to the same size as the original 3D image according to the preset multiple (that is, up-sampling), and obtain an updated 3D image, so that the updated 3D image can be updated through the reduced preset multiple.
  • the 3D image is down-sampled to narrow the size gap between the down-sampled 3D image and the 3D image.
  • a global optimizer may also be used to adjust space transformation parameters such as a preset multiple and a preset step size.
  • Optimization problems include many types of problems, such as how to allocate resources most efficiently, fitting problems, min-max problems, and so on. Optimization problems are generally divided into local optimization and global optimization. Local optimization is to find the minimum value in a limited area of the function value space; while global optimization is to find the minimum value in the entire area of the function value space. The local minimum point of the function, that is, the function point whose function value is less than or equal to the value of the nearby point, but may be greater than the value of the farther point; the global minimum point, that is, the function point whose function value is less than or equal to all feasible points.
  • the pose transformation module 220 can adjust the space transformation parameters such as the preset multiple and the preset step size in a global optimization manner, so as to achieve faster and more accurate It is more efficient to find a spatial parameter that can make the first similarity meet the first preset threshold.
  • the pose transformation module 220 may also use other optimization methods to adjust space transformation parameters such as preset multiples and preset step sizes, which are not specifically limited here.
  • Step 470 Determine a first transformation matrix based on the pose transformation process from the 3D image to the registered 3D image.
  • the pose transformation module 220 can obtain a second transformation matrix between the coordinate system of the two-dimensional imaging device and the coordinate system of the operation space.
  • the two-dimensional imaging device coordinate system is a coordinate system generated with reference to the medical imaging device (for example, two-dimensional medical imaging device, etc.) during operation.
  • the coordinate system of the two-dimensional imaging device can be based on a specific position on the two-dimensional imaging device that shoots the two-dimensional image (such as a certain point on the C-arm) as the origin, the horizontal direction as the x-axis, and the vertical direction as z Axis, front and back direction as the y-axis.
  • the first tracking array and the second tracking array may be respectively placed on the two-dimensional medical imaging device (for example, on the C-arm of the DR device) and the body of the target subject in advance.
  • the tracking array can include several marker points (such as reflective balls).
  • a device space coordinate system corresponding to the two-dimensional medical imaging device ie, a two-dimensional imaging device coordinate system
  • an intraoperative space coordinate system ie, an operation space coordinate system
  • the coordinates of the first tracking array and the coordinates of the second tracking array can be obtained by using the OTS, so as to determine the transformation matrix between the coordinate system of the two-dimensional imaging device and the coordinate system of the operation space as the second transformation matrix.
  • the pose transformation module 220 may obtain a third conversion matrix between the coordinate system of the 3D image and the coordinate system of the 2D imaging device based on the registration of the 3D image.
  • the process of adjusting the 3D image to obtain the registration of the 3D image is the process of establishing the transformation relationship from the coordinate system of the 3D image to the coordinate system of the 2D imaging device. Therefore, the first adjustment value can be determined based on the adjustment process, and the third transformation matrix can be determined according to the first adjustment value.
  • the first adjustment value includes a total rotation step and a total translation step.
  • step 420 may be performed multiple times.
  • the pose transformation module 220 may add up the rotation step size used each time step 420 is executed to obtain the total rotation step size, and add up the translation step size used each time step 420 is executed to obtain the total translation step size.
  • the pose transformation module 220 can obtain the first transformation matrix based on the second transformation matrix and the third transformation matrix. For example, the product of the second conversion matrix and the third conversion matrix can be used as the first conversion matrix.
  • the following provides a specific example of performing pose adjustment on a 3D image to obtain a registered 3D image.
  • the 2D image consists of two images
  • the preset multiple is 2
  • the preset downsampling step is 2
  • the similarity threshold corresponding to the preset multiple 8 is 0.2
  • the similarity threshold corresponding to the preset multiple 4 is 0.4.
  • the pose transformation module 220 can obtain the registered 3D image through the following steps, and determine the third transformation matrix.
  • the 3D image P01, the 2D image P11 corresponding to the first shooting pose, and the 2D image P21 corresponding to the second shooting pose are respectively subjected to downsampling processing with a preset multiple of 8 to obtain the downsampled 3D image P02, downsampled two 2D image P12 and downsampled 2D image P22.
  • Project P02 at the first projection pose and the second projection pose respectively to obtain the first projection image D1 and the first projection image D2, wherein the first shooting pose is the same as the first projection pose, and the second shooting pose The pose is the same as the second projected pose. Calculate the first similarity based on P12, P22, D1, and D2.
  • the first similarity is not greater than 0.2, adjust the pose of P02 according to the preset rotation step of 10° and the preset translation step of 20mm corresponding to 8, and get the adjustment.
  • the final down-sampled 3D image P02 repeat the above-mentioned process of determining the first similarity until the determined first similarity is greater than 0.2. Further, it may be determined whether the current preset multiple is equal to the preset value 2 of the multiple. Since the preset multiple of 8 is not equal to the preset multiple of 2, the P02 is up-sampled according to the preset multiple of 8 to update the 3D image P01 to obtain the 3D image P01'. Reduce the preset factor to 4 according to the preset downsampling step size of 2.
  • the 3D image P01', the 2D image P11 of the first shooting pose, and the 2D image P21 of the second shooting pose are respectively subjected to downsampling processing with a preset multiple of 4 to obtain the downsampled 3D image P02', downsampled two
  • the three-dimensional image P12' and the down-sampled two-dimensional image P22' are respectively projected on P02' in the first projection pose and the second projection pose to obtain the first projection image D1' of the first projection pose, and the second projection
  • the first similarity is calculated based on P12', P22', D1' and D2'.
  • the preset rotation step of 4 corresponding to 5° and Adjust the pose of P02' with a preset translation step of 10 mm to obtain the adjusted down-sampled 3D image P02', repeat the above-mentioned process of determining the first similarity until the determined first similarity is greater than 0.4, and determine the current preset multiple Whether it is equal to the preset value 2 of the multiple, in this step, since the preset multiple 4 is not equal to the preset value 2 of the multiple, up-sample P02' according to the preset multiple 4 to update the 3D image P01 to obtain a 3D image P01", reduce the preset multiple to 2 according to the preset downsampling step size 2.
  • the 3D image P01", the 2D image P11 of the first projection pose, and the 2D image P21 of the second projection pose are respectively subjected to downsampling processing with a preset multiple of 2 to obtain the downsampled 3D image P02", the downsampled two Two-dimensional image P12" and down-sampled two-dimensional image P22", respectively project P02" in the first projection pose and the second projection pose to obtain the first projection image D1" in the first projection pose, and the second projection For the first projected image D2" of the pose, the first similarity is calculated based on P12", P22", D1" and D2".
  • the corresponding preset rotation step of 2° and Adjust the pose of P02” with a preset translation step of 5 mm to obtain the adjusted down-sampled 3D image P02 repeat the above process of determining the first similarity until the determined first similarity is greater than 0.6, and judge the current preset multiple Whether it is equal to the preset value 2 of the multiple, in this step, since the preset multiple 2 is equal to the preset value 2 of the multiple, P02" is up-sampled according to the preset multiple 2 to obtain the adjusted three-dimensional image, and determine the first Three transformation matrices.
  • the coarse registration between the 3D image and the 2D image is realized by performing pose transformation on the 3D image based on at least one 2D image.
  • the rough registration process there is no need to manually select anatomical points in the preoperative 3D images and multiple 2D images during the operation, and a large amount of human-computer interaction is not required, which reduces the complexity of registration.
  • the amount of data analysis can be reduced, the speed of rough registration can be improved, and the rapid and automatic completion of 3D images and 2D images can be achieved. Coarse registration between .
  • Fig. 5 is an exemplary flowchart of a process for generating a 3D object image and a 2D object image according to some embodiments of the present specification.
  • the target image determination module 230 can determine the 2D target image and the 3D target image corresponding to the target part of the target object in at least one 2D image and the registered 3D image through the method shown in the process 500 .
  • the process 500 can be used to implement step 330 shown in FIG. 3 .
  • the process 500 can be performed separately for each target part. As shown in FIG. 5 , the process 500 may include the following steps.
  • Step 510 determine the 3D target image corresponding to the target part in the registered 3D image.
  • the 3D target image refers to an image intercepted from the registered 3D image that only contains the target part, or contains the target part and a small amount of its surrounding parts.
  • the target site may include at least one of a target spine, a target hip joint, a target elbow joint, a target knee joint, and a target finger joint.
  • the target image determining module 230 may segment the registered 3D image through an image segmentation algorithm to obtain a segmentation mask corresponding to the target part.
  • the target image determining module 230 may determine the 3D target image including only the target part in the registered 3D image based on the segmentation mask.
  • step 520 can be used to obtain the fourth transformation between the two-dimensional image coordinate system corresponding to the two-dimensional image and the operation space coordinate system Matrix: through step 530-step 550, determine the 2D target image based on the 3D target image, the first transformation matrix and the fourth transformation matrix.
  • a two-dimensional image is taken as an example below to describe the implementation process of steps 520-550.
  • Step 520 acquiring a fourth transformation matrix between the coordinate system of the two-dimensional image corresponding to the two-dimensional image and the coordinate system of the operation space.
  • the target image determination module 230 may obtain a second transformation matrix between the coordinate system of the two-dimensional imaging device and the coordinate system of the operation space. For more details on how to obtain the second transformation matrix, refer to the relevant description of step 470, and details are not repeated here.
  • the target image determining module 230 may also obtain a fifth transformation matrix between the coordinate system of the 2D imaging device and the coordinate system of the 2D image. Specifically, the target image determination module 230 may acquire internal references (for example, the distance from the radiation source to the imaging plane) when the two-dimensional medical imaging equipment captures two-dimensional images, and determine the fifth transformation matrix based on the internal references.
  • the target image determining module 230 may obtain a fourth transformation matrix between the two-dimensional image coordinate system corresponding to the two-dimensional image and the operation space coordinate system based on the second transformation matrix and the fifth transformation matrix.
  • Step 530 based on the 3D target image, determine the 3D coordinates of the representative point of the target part in the 3D image coordinate system and the size parameters of the target part.
  • the representative point of the target part refers to a representative feature point in the target part, for example, a central point, a boundary point, and the like.
  • the representative point of the target site may include a center point of the target site, eg, the center of the spine.
  • the size parameter of the target part refers to a parameter related to the size of the target part, for example, length, width, distance from a center point to a side, and the like.
  • the size parameter of the target part may include a set of distances from the center point of the target part to the edge of the target part.
  • the edge of the target site may be represented by each side of its circumscribed rectangle.
  • the target image determining module 230 may identify a representative point (eg, a central point) of the target part in the 3D target image, so as to obtain the 3D coordinates of the representative point of the target part in the 3D image coordinate system.
  • a representative point eg, a central point
  • the target image determining module 230 may perform projection (such as DRR projection) on the 3D target image to obtain a second projected image corresponding to the 2D image.
  • the projection pose of the second projected image is the same as the shooting pose of the 2D image.
  • other projected images corresponding to the two-dimensional images eg, second projected images, etc.
  • the target image determining module 230 may determine representative points of the target part and size parameters of the target part in the second projected image based on representative points of the target part in the 3D target image. For example, the center point (i.e. the second center point) projected by the center point (i.e.
  • the first center point) of the target part in the three-dimensional target image can be determined in the second projection image, and the minimum value of the target part in the second projection image can be determined.
  • Circumscribed rectangle get the distance from the second center point to each side of the smallest circumscribed rectangle.
  • the distance set composed of these distances can be used as the size parameter of the target part. That is to say, the distance from the second center point (ie, the center point of the target part in the second projection image) to the edge of the target part refers to the distance from the second center point to each side of the smallest circumscribed rectangle of the target part.
  • the 2D image includes a first 2D image and a second 2D image.
  • the target image determining module 230 may project the three-dimensional target image at the first projection pose and the second projection pose to obtain a second projection image X1 corresponding to the first two-dimensional image, and a second projection image X1 corresponding to the second two-dimensional image.
  • Image X2 determine the second center point f1 obtained by the projection of the first center point in X1, determine the second center point f2 obtained by the projection of the first center point in X2; determine the minimum circumscribed rectangle C1 of the target site in X1, and determine The distance from f1 to the four sides of C1 is obtained to obtain the distance set A1 corresponding to X1; the minimum circumscribed rectangle C2 of the target part is determined in X2, and the distance from f2 to the four sides of C2 is determined to obtain the distance set A2 corresponding to X2.
  • A1 may be used as a size parameter corresponding to the first two-dimensional image.
  • A2 may be used as a size parameter corresponding to the second two-dimensional image.
  • Step 540 determine the two-dimensional coordinates of the representative point in the two-dimensional image coordinate system based on the three-dimensional coordinates, the first transformation matrix and the fourth transformation matrix.
  • the three-dimensional coordinates can be determined through step 530
  • the fourth transformation matrix can be determined through step 520
  • the first transformation matrix can be determined through step 320 .
  • the target image determination module 230 can convert the three-dimensional coordinates of the representative points of the target part in the three-dimensional image coordinate system into the operation space coordinate system through the first transformation matrix to obtain the first transformation point; according to the two The fourth conversion matrix corresponding to the 2D image can convert the first conversion point in the surgical space coordinate system into the 2D image coordinate system corresponding to the 2D image, so as to obtain the 2D coordinates of the representative point.
  • the target image determination module 230 can convert the three-dimensional center point m1 to the surgical space coordinate system through the first transformation matrix T1 to obtain the first transformation point m2; through the fourth transformation matrix T4a corresponding to the first two-dimensional image, the m2 is transformed into the first two-dimensional image coordinate system corresponding to the first two-dimensional image, and the two-dimensional center point n1 in the first two-dimensional image is obtained (ie, the corresponding point of the representative point of the target part in the first two-dimensional image) .
  • the target image determination module 230 can transform m2 into the second two-dimensional image coordinate system corresponding to the second two-dimensional image through the first transformation matrix T4b corresponding to the second two-dimensional image, and obtain the two-dimensional The center point n2 (that is, the corresponding point of the representative point of the target part in the second two-dimensional image).
  • Step 550 determine the 2D target image based on the 2D coordinates and size parameters.
  • the size parameter may include a set of distances from the center point of the target part to the circumscribed rectangle of the target part.
  • the target image determination module 230 may determine the interception area according to a set of distances centered on the two-dimensional coordinates.
  • the intercepted area can be used as a two-dimensional target image.
  • the 2D target image may be determined from the first 2D image based on the second center point f1 and the distance set A1 corresponding to the first 2D image.
  • the 2D target image may be determined from the second 2D image based on the second center point f2 and the distance set A2 corresponding to the second 2D image.
  • the distance from the center point to one side of the circumscribed rectangle may be represented by the number of pixels.
  • the set of distances may include a left distance, a right distance, an upper distance, and a lower distance.
  • the target image determination module 230 can determine the coordinate point of the upper left corner according to the two-dimensional coordinates, the left distance and the upper distance of the representative point of the target part in the two-dimensional image coordinate system; according to the two-dimensional coordinates, the right Determine the coordinate point of the upper right corner according to the side distance and the upper side distance; determine the coordinate point of the lower left corner according to the two-dimensional coordinates, the left distance and the lower side distance; determine the lower right corner coordinate point according to the two-dimensional coordinates, the right distance and the lower side distance; Then determine the interception area according to the coordinate points of the upper left corner, the coordinate points of the upper right corner, the coordinate points of the lower left corner and the coordinate points of the lower right corner.
  • the target image determination module 230 may perform an expansion process on the distance set, so that the two-dimensional target image may include a complete target part.
  • the left distance, the right distance, the upper distance and the lower distance can be increased by a preset number of pixels, for example, the left distance, the right distance, the upper distance and the lower distance can be increased by 40 respectively pixel.
  • the 2D image may be segmented by a segmentation algorithm to obtain a 2D target image instead of a 3D image.
  • the imaging resolution of 2D images is usually not as high as that of 3D images, and the images may contain surgical devices, so the segmentation effect obtained by directly segmenting the target spine from 2D images may not be very good.
  • the segmentation accuracy can be improved.
  • Fig. 6 is an exemplary flowchart of a process of optimizing a first transformation matrix to obtain a target transformation matrix according to some embodiments of the present specification.
  • the target matrix determination module 240 can perform pose transformation on the registered 3D image based on the 3D target image and the 2D target image through the method shown in the process 600, so as to optimize the first transformation matrix and obtain the target transformation matrix .
  • the process 600 can be used to implement step 340 shown in FIG. 3 .
  • the process 600 may be performed by the target matrix determination module 240 . As shown in FIG. 6 , the process 600 may include the following steps.
  • Step 610 for each 2D image, project the 3D target image based on the shooting pose of the 2D image to obtain a corresponding second projected image.
  • the process of generating the second projected image is similar to the process of generating the first projected image described in step 430 , and will not be repeated here.
  • Step 620 Determine a second similarity based on at least one second projected image and at least one 2D target image.
  • the second similarity degree can be used to measure the degree of similarity between the second projected image and the corresponding 2D target image.
  • the target matrix determination module 240 can determine each second projection image and the corresponding The similarities between the two-dimensional target images are obtained based on all the similarities to obtain the second similarity, for example, summation, average value, maximum value, etc. For more information about how to determine the similarity, refer to the relevant description of step 440, and details will not be repeated here.
  • Step 630 adjust the pose of the registered 3D image based on the second similarity to optimize the first transformation matrix to obtain a target transformation matrix.
  • the target matrix determination module 240 may use a method similar to that used to obtain the registered 3D image to perform pose adjustment on the registered 3D image to obtain the target transformation matrix. For example, steps similar to steps 450-470 can be performed, wherein the registered 3D image is equivalent to the original 3D image in steps 450-470, the target transformation matrix is equivalent to the first transformation matrix in steps 450-470, and the second similarity The degree is equivalent to the first similarity degree in steps 450-470.
  • the target matrix determining module 240 may determine whether the second similarity satisfies a third preset condition.
  • the third preset condition may be that the similarity difference between the second similarity and the second similarity determined when adjusting the pose of the registered 3D image last time is smaller than the difference threshold.
  • the threshold can be set according to requirements, for example, 0.005 and so on.
  • the third preset condition may be that the second similarity is greater than a similarity threshold.
  • the target matrix determination module 240 may determine whether the current iteration number satisfies a fourth preset condition.
  • the fourth preset condition may be whether the current iteration number is equal to the preset iteration number.
  • the current number of iterations is the number of times for adjusting the pose of the registered 3D image, which can be set according to requirements, for example, 40 times and so on.
  • the target matrix determination module 240 may adjust the registered 3D image based on the second similarity pose, to optimize the first transformation matrix, and repeat the process of determining the second similarity until the determined second similarity satisfies the third preset condition and/or the current number of iterations meets the fourth preset condition, then stop the iteration,
  • the optimized first transformation matrix at this time is determined as the target transformation matrix.
  • an optimizer for example, Powell optimization algorithm, etc.
  • the optimization is completed, and the first transformation matrix when the optimization is completed is determined as the target transformation matrix.
  • the basis for judging that the pose of the 3D image is optimal is whether the optimizer reaches the iteration stop condition, for example, whether the maximum number of iterations (for example, 40, etc.) is reached, whether the minimum step size (for example, 0.05, etc.) is reached ), whether the allowable size of the step size change is reached (for example, 0.005, etc.).
  • the target matrix determination module 240 may iteratively adjust the poses of the registered 3D images based on multiple 2D target images, so as to iteratively optimize the first transformation matrix until the currently determined second similarity is the same as the last adjusted
  • the similarity difference between the second similarities determined when registering the poses of the 3D images is smaller than a threshold (for example, 0.005), or the current number of iterations is equal to a preset number of iterations (for example, 40 times).
  • a plurality of second projection images are determined based on the adjusted registration 3D image, and a second similarity Si is determined based on the plurality of second projection images and the plurality of 2D target images, and the calculation The similarity difference between Si and Si-1 (the second similarity calculated after adjusting the pose of the registered 3D image for the i-1th time), and obtain the current iteration number i, if the similarity difference is not less than the threshold, And if i is not equal to the preset number of iterations, the pose of the adjusted registered 3D image is adjusted again based on Si.
  • the target matrix determination module 240 may obtain a similarity difference between the second similarity and the second similarity determined when the pose of the registered 3D image was adjusted last time.
  • the target matrix determination module 240 may determine an adjustment step according to the similarity difference, and adjust the adjusted pose of the registered 3D image again according to the adjustment step.
  • adjusting the step size includes adjusting the rotation step size and adjusting the translation step size.
  • the similarity difference is positively correlated with the adjustment step, and the larger the similarity difference is, the larger the adjustment step is.
  • the target transformation matrix can be obtained.
  • the difference threshold can be set according to requirements, for example, 0.005 and so on.
  • the process of adjusting the pose of the registered 3D image according to the adjustment step is included multiple times, and also includes Determines the total adjustment steps made during this process.
  • the total adjustment step includes a total adjustment rotation step and a total adjustment translation step.
  • the target matrix determining module 240 may determine the target transformation matrix according to the first transformation matrix, the total adjusted rotation step and the total adjusted translation step.
  • the target transformation matrix can realize fine registration of the target part in the three-dimensional image coordinate system and the intraoperative space coordinate system.
  • the operation is simple.
  • the second similarity can be determined based on the second projected image and the 2D target image, and then the pose of the registered 3D image can be adaptively and iteratively adjusted based on the second similarity, so as to iteratively optimize the first transformation matrix until the target
  • the transformation matrix speeds up the iterative optimization of the first transformation matrix, and can obtain a more accurate target transformation matrix.
  • Fig. 9 is a schematic diagram of a registration method according to some embodiments of the present specification.
  • the registration method shown in FIG. 9 may be executed by the processing device 120 .
  • the target site is a vertebra in the spine.
  • the registration method shown in Figure 9 includes the following steps:
  • Step 901 acquire the three-dimensional image Y0 of the spine taken before the operation.
  • Step 902 preprocessing Y0 to obtain the segmentation mask of each spine and the label of the segmentation mask of each spine.
  • the spine in Y0 can be segmented by an image segmentation algorithm, a segmentation mask for each spine is obtained, and a label is configured for each segmentation mask.
  • Step 903 acquiring the 2D image Y1 and the 2D image Y2 of the spine taken in the first shooting pose and the second shooting pose during the operation.
  • Y1 may include the 2D image of the spine taken from the front as shown in FIG. 7A
  • Y2 may include the 2D image of the spine taken from the side as shown in FIG. 8A .
  • Step 904 based on Y1 and Y2, determine a fourth transformation matrix T1a corresponding to Y1, and a fourth transformation matrix T1b corresponding to Y2.
  • Step 905 perform rough registration according to Y0, Y1 and Y2, obtain the first transformation matrix T2 between the coordinate system of the 3D image and the coordinate system of the operation space, and register the 3D image Y4.
  • Step 906 designate the label of the target part to determine the segmentation mask of the target part, and determine the 3D target image y0 and the first center point s1 of the target part in Y4 according to the segmentation mask of the target part.
  • the processing device 120 may determine the 2D target image including only the target part in the 2D image by performing steps 907 to 910 .
  • Step 907 Projecting the pair y0 in the first projected pose and the second projected pose to obtain a second projected image y01 in the first projected pose and a second projected image y02 in the second projected pose.
  • the first projection pose is the same as the first shooting pose
  • the second projection pose is the same as the second shooting pose.
  • Step 908 determine the smallest circumscribed rectangle rec1 of the target part in y01, and the smallest circumscribed rectangle rec2 of the target part in y02.
  • Step 909 Transform s1 into the surgical space coordinate system according to T2 to obtain s2, transform s2 into the two-dimensional image coordinate system corresponding to Y1 according to T1a to obtain s3, transform s2 into the two-dimensional image coordinate system corresponding to Y2 according to T1b, get s4.
  • Step 910 determine the 2D target image y1 corresponding to the target part in Y1; according to s4 and rec2, determine the 2D target image y2 corresponding to the target part in Y2.
  • y1 may include the anteroposterior 2D image of the target spine as shown in FIG. 7B , which is segmented from the anteroposterior 2D image of the target spine as shown in FIG. 7A .
  • y2 may include a lateral 2D image of the target spine as shown in FIG. 8B , which is segmented from the lateral 2D image of the target spine as shown in FIG. 8A .
  • the processing device 120 can perform a fine registration process, that is, based on the 3D target image and the 2D target image in each 2D image, perform pose transformation on the registered 3D image to optimize the first transformation matrix, and obtain Target transformation matrix.
  • Step 911 calculate the second similarity according to y1, y2, y01 and y02;
  • Step 912 judge whether the second similarity meets the third preset condition, if not, execute step 913, and if yes, execute step 915.
  • the third preset condition may include that the similarity difference between the currently determined second similarity and the second similarity determined when the pose of the registered 3D image Y4 is adjusted last time is smaller than a threshold.
  • Step 913 obtain the current iteration number i, and judge whether the current iteration number satisfies the fourth preset condition, if not, execute step 914, and if yes, execute step 915.
  • the fourth preset condition may include that the current iteration number is equal to the preset iteration number.
  • Step 915 obtain the target transformation matrix.
  • step 914 may be omitted. If the second similarity threshold does not meet the third preset condition, 915 may be executed.
  • Fig. 10 is a schematic diagram of a registration method according to some embodiments of the present specification.
  • the processing device 120 may execute the registration method shown in FIG. 10 to perform registration on each bone in the multi-skeleton.
  • the target site is each bone in the multi-skeleton.
  • a multi-skeletal 3D image to be registered and a 2D image to be registered may be obtained.
  • DRR projection can be performed on the multi-skeleton 3D image 1010 to obtain a DRR image 1020 .
  • the multi-skeletal 3D image 1010 may include any one of CT, MRI and other images.
  • the multi-skeletal 3D image 1010 may be an image including multiple target parts, for example, a spine including multiple vertebrae.
  • the 3D image can be obtained through the following steps: obtaining the initial 3D image before operation; performing a pose search on the initial 3D image before operation based on the first pose search parameters to obtain the first 3D image, and obtaining the first 3D image The first pose similarity between the image and the 2D image; if the first pose similarity meets the third preset threshold, perform a pose search on the first 3D image based on the second pose search parameters to obtain a second 3D image, And obtain the second pose similarity between the second 3D image and the 2D image; if the second pose similarity meets the fourth preset threshold, based on the initial 3D image, the first pose search parameter and the second pose search parameter A 3D image is acquired, and the space transformation parameter from the preoperative initial 3D image to the 3D image in the above process is determined as a first space transformation parameter.
  • the first 3D image may be projected according to the shooting pose of the 2D image, and the similarity between the projection and the 2D image may be calculated as the first pose similarity.
  • the second 3D image may be projected according to the shooting pose of the 2D image, and the similarity between the projection and the 2D image may be calculated as the second pose similarity.
  • the pose search parameters may include a preset downsampling factor, a preset rotation step, and a preset translation step.
  • the pose search can be performed in a manner similar to steps 410 and 420 .
  • pose search may be performed in other ways according to actual needs, which is not specifically limited in this specification.
  • the initial poses of the joints in the initial 3D image may be estimated by means of multi-resolution and multi-stage search.
  • the first-level pose search is performed, and the initial 3D image is down-sampled by 8 times, and the 3D image is rotated every 10° around the X, Y, and Z coordinate axes in the three-dimensional space, along the X, Y, and Z axes.
  • the three coordinate axes are translated every 20mm, and then the first pose similarity calculation is performed.
  • the similarity does not meet the preset stop condition, repeat the above process until the similarity meets the preset stop condition, stop the iteration and save Space transformation parameters; then perform the second-level pose search, perform 4-fold downsampling on the initial 3D image, set the rotation interval to 5 degrees, and the translation interval to 10mm, and then perform the second pose similarity calculation, if the similarity If the preset stop condition is not satisfied, the above process is iterated repeatedly until the similarity meets the preset stop condition, the iteration is stopped and the spatial transformation parameters are retained. The initial poses of the joints are determined based on the space transformation parameters of the two-stage pose search.
  • rough registration can be performed based on the DRR image and the 2D image of the multi-skeleton overall 3D image, that is, similarity analysis is performed between the DRR image and multiple 2D images to perform multi-skeleton overall rigid registration.
  • the projection pose of the DRR of the 3D image is consistent with the shooting pose of the 3D image.
  • the DRR image 1020 and the 2D image 1030 are subjected to multi-skeleton global rigid registration in step 1040 .
  • the 2D image 1030 may include multiple X-ray images and the like.
  • the 3D image may be spatially transformed based on the first spatial transformation parameter to obtain a first registered 3D image, and a first degree of similarity between the first registered 3D image and the 2D image may be obtained.
  • the first registered 3D image can be projected (for example, DDR projection, etc.) to obtain the first registered reconstructed image, wherein the number of the first registered reconstructed image is the same as that of the 2D image, and the second The projection angle of a registered reconstructed image is the same as the shooting angle of the 2D image.
  • the similarity between the first registered and reconstructed image and the 2D image may be obtained as the first similarity.
  • the first registered and reconstructed image may be obtained by projecting the first registered 3D image through a method similar to step 430 .
  • the first similarity can be obtained by a method similar to step 440 .
  • the coarse registration that is, the multi-skeleton overall rigid registration
  • the registered multi-skeletal 3D image can be obtained through a method similar to step 460 .
  • a single-skeleton 3D image of each bone can be obtained based on the first registered 3D image, and a single-skeleton 2D image of each bone can be obtained based on the 2D image.
  • Skeleton images are obtained through bone segmentation to obtain single-skeleton images.
  • the single-skeleton 3D image can be spatially transformed based on the second spatial transformation parameters to obtain a second registered 3D image of each bone.
  • the second registered 3D image can be obtained through a method similar to that used to obtain the first registered 3D image.
  • a second degree of similarity between the second registered 3D image and the corresponding single-skeleton 2D image may be acquired, and if the second similarity meets a second preset threshold, the registration may be obtained based on the second registered 3D image. quasi-result.
  • the second similarity can be obtained through a method similar to that of obtaining the first similarity.
  • the registration result can be obtained through a method similar to step 460 and step 470 .
  • the registration results may include registration results for each bone.
  • a registered multi-skeleton 3D image can be obtained.
  • automatic bone segmentation can be performed on the registered multi-skeleton 3D image to obtain the 3D image of each bone, and the 3D image of each bone can be projected and reconstructed to obtain the DRR image of each bone.
  • automatic bone segmentation can be performed on the registered multi-skeleton 3D image through step 1050 to obtain a 3D image of each bone, that is, bone 1, bone 2, ..., bone N, where, N is a positive integer greater than or equal to 2; then perform DRR projection on bone 1, bone 2, ..., bone N respectively, to obtain corresponding projection images DRR1, DRR2, ..., DRRN.
  • the projection pose of the DRR image of each bone is consistent with the shooting pose of the 2D image 1030 .
  • automatic bone segmentation can be directly performed on the original multi-skeleton 3D image (ie, multi-skeleton 3D image without global rigid registration) to obtain a 3D image of each bone. Then, the pose of the 3D image of each bone can be adjusted based on the overall rigid registration result, and then the adjusted 3D image of each bone can be projected and reconstructed to obtain the DRR image of each bone.
  • fine registration can be performed based on the DRR image of each bone and the 2D image of each bone, that is, the similarity between the projected image of each bone and the 2D image of each bone is compared to realize the Rigid registration of each bone to obtain registration results for each bone.
  • the projection images DRR1, DRR2, ..., DRRN can be rigidly registered with the 2D images of the corresponding bones in the 2D image 1030 to obtain the corresponding bone 1 registration results, bone 2 registration results, ..., Bone N registration results.
  • the registration result of each bone may refer to the target transformation matrix (also called registration matrix) corresponding to each bone, that is, the transformation matrix from the three-dimensional image coordinate system to the intraoperative surgical space coordinate system.
  • the registration matrix of each bone as the registration result can completely reflect the transformation of each bone, and the registration accuracy is higher and the effect is better.
  • the preset threshold of similarity may be determined according to actual needs, specifically, it may be determined according to the registration site. For example, when using the above-mentioned registration method to register the pelvis and femur, and using the gradient difference as the similarity, the first preset threshold can be set to 0.5, the second preset threshold can be set to 0.88, and the third preset threshold can be set to 0.5. The threshold can be set to 0.2, and the fourth preset threshold can be set to 0.35. In some embodiments, due to different registration parts and different similarity functions selected, the corresponding preset thresholds may also be different.
  • the range of the first preset threshold can be 0.4-0.8
  • the range of the second preset threshold can be 0.7-1.0
  • the range of the third preset threshold can be 0.2-0.4
  • the range of the fourth preset threshold can be 0.3 ⁇ 0.5.
  • the first registered 3D image is obtained, and the first registration of the first registered 3D image and the 2D image is obtained.
  • the first similarity meets the first preset threshold, a single-skeleton 3D image of each bone is obtained based on the first registration 3D image, and a single-skeleton 2D image of each bone is obtained based on the 2D image; the single-skeleton 3D The image is spatially transformed based on the second spatial transformation parameters to obtain a second registration 3D image of each bone, and obtain a second similarity between the second registration 3D image and the corresponding single-skeleton 2D image; if the second similarity meets The second preset threshold value is based on the method of obtaining the registration result of the second registered 3D image, and adopts the method of multi-skeleton overall rigid registration combined with single bone rigid registration, which improves the accuracy and success rate of the registration algorithm.
  • the above multi-skeleton image registration method does not require manual interaction, and can automatically complete image registration, saving labor costs.
  • Fig. 11 is a schematic structural diagram of a registration system according to some embodiments of the present specification.
  • the registration system as shown in FIG. 11 may include a medical imaging device 1110 , a processor 1120 and a navigation device 1130 .
  • the medical imaging device 1110 can be used to capture at least one two-dimensional image of the target object (for example, human body, etc.) The three-dimensional image and/or the three-dimensional image are sent to the processor 1120.
  • the medical imaging device 1110 may include at least one of a two-dimensional imaging device, a three-dimensional imaging device, and the like.
  • the processor 1120 may be configured to acquire a 3D image of the target object taken before the operation and at least one 2D image taken during the operation, and perform 2D-3D registration on the acquired images.
  • the processor 1120 may execute the registration method described in FIGS. 3-10 of this specification.
  • navigation device 1130 may be used to navigate surgical instruments to a target surgical region based on surgical guidance information.
  • the target operation area corresponds to the target site, for example, a certain vertebra on the spine.
  • the navigation device 1130 may convert surgical path information into surgical guidance information based on the registration result (eg, target conversion matrix) to navigate the surgical instrument to the target surgical area.
  • navigation device 30 may be an NDI navigation system or other surgical navigation system.
  • the possible beneficial effects of the embodiments of this specification include but are not limited to: (1) By improving the existing 2D-3D registration method, through automatic coarse registration, automatic positioning and extraction of the anatomy to be registered in the two-dimensional image
  • the method of DRR projection of the structural area and only the anatomical structure to be registered eliminates the need for users to interact with preoperative and intraoperative images, which greatly reduces the interaction process, improves the user experience and the clinical feasibility of the algorithm, and realizes only need Semi-automatic 2D-3D registration with a small number of simple interactions and fully automatic 2D-3D registration in some registration scenarios;
  • the disadvantage of low resolution of two-dimensional image imaging, and the interference of surgical devices in the two-dimensional image is eliminated, thereby improving the segmentation accuracy and ensuring the segmentation effect;
  • the accurate mapping of the target image to the surgical space coordinate system ensures the surgical effect.
  • the possible beneficial effects may be any one or a combination of the above
  • numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of the embodiments use the modifiers "about”, “approximately” or “substantially” in some examples. grooming. Unless otherwise stated, “about”, “approximately” or “substantially” indicates that the stated figure allows for a variation of ⁇ 20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximations that can vary depending upon the desired characteristics of individual embodiments. In some embodiments, numerical parameters should take into account the specified significant digits and adopt the general digit reservation method. Although the numerical ranges and parameters used in some embodiments of this specification to confirm the breadth of the range are approximations, in specific embodiments, such numerical values are set as precisely as practicable.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computer Graphics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Radiology & Medical Imaging (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Public Health (AREA)
  • Image Processing (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

一种配准方法，由至少一个处理器执行，包括获取目标对象在手术前拍摄的三维影像和手术中拍摄的至少一张二维影像（310）；基于至少一张二维影像，对三维影像进行位姿变换以得到配准三维影像，以及三维影像对应的三维影像坐标系和手术对应的手术空间坐标系之间的第一转换矩阵（320）；在至少一张二维影像确定目标对象的目标部位对应的二维目标影像，在配准三维影像中确定目标部位对应的三维目标影像（330）；基于三维目标影像和每张二维影像中的二维目标影像，对配准三维影像进行位姿变换，以优化第一转换矩阵，得到目标转换矩阵（340）。

Description

一种配准方法和系统
相关申请的交叉引用
本申请要求2022年02月17日提交的名称为“配准方法、系统、计算机设备和存储介质”的中国专利申请202210147197.6和2021年07月30日提交的名称为“多骨骼影像配准方法、电子装置以及医学导航系统”的中国专利申请202110875674.6的优先权，上述申请的全部内容以引用方式被完全包含在此。
技术领域
本说明书涉及计算机辅助手术领域，特别涉及一种配准术前拍摄的三维影像和术中拍摄的二维影像的方法和系统。
背景技术
在计算机辅助手术中，医生往往需要在手术前对病人拍摄3D影像，比如CT(Computed Tomography，电子计算机断层扫描)影像或MRI(Magnetic Resonance Imaging，磁共振成像)影像，并基于3D影像制定手术计划。然后在手术过程中将术前3D影像中制定的手术计划映射到手术空间中去，达到引导术中手术过程的目的。在传统技术中，可以在术中拍摄多张二维影像，将术前三维影像与二维影像进行二维-三维(2dimensional-3dimensional，2D-3D)配准，间接建立起三维影像空间和手术空间的映射关系。
现有的二维-三维配准方法通常存在大量复杂的人机交互过程，包括需要在多张二维影像、三维影像中手动选取多个解剖点，手动选取待配准区域等。大量的人机交互和频繁的人工操作会影响配准的准确度和效率。另一方面，对于在对由多块骨骼组成的关节部位，比如髋关节、肘关节、膝关节、脊柱或者手指关节等进行二维-三维影像配准时，病人的关节姿态的变化会使得这些部位的骨骼发生相对运动，从而造成术前和术中拍摄影像时关节处骨骼间的相对位置不一致，降低了二维-三维配准的精度以及手术引导的准确性。
因此，希望提供一种配准术前拍摄的三维影像和术中拍摄的二维影像的方法和系统。
发明内容
本说明书实施例之一提供一种配准方法，由至少一个处理器执行。所述配准方法包括：获取目标对象在手术前拍摄的三维影像和手术中拍摄的至少一张二维影像；基于所述至少一张二维影像，对所述三维影像进行位姿变换以得到配准三维影像，以及所述三维影像对应的三维影像坐标系和所述手术对应的手术空间坐标系之间的第一转换矩阵；在所述至少一张二维影像中确定所述目标对象的目标部位对应的二维目标影像，在所述配准三维影像中确定所述目标部位对应的三维目标影像；基于所述三维目标影像和每张二维影像中的所述二维目标影像，对所述配准三维影像进行位姿变换，以优化所述第一转换矩阵，得到目标转换矩阵。
在一些实施例中，可以基于预设倍数对所述三维影像和所述至少一张二维影像进行下采样，得到下采样三维影像和至少一张下采样二维影像；基于预设步长调整所述下采样三维影像的位姿，得到调整后的下采样三维影像；对所述每一张二维影像，基于所述二维影像的拍摄位姿对所述调整后的下采样三维影像进行投影，得到对应的第一投影影像；响应于至少一张所述第一投影影像和所述至少一张下采样二维影像满足第一预设条件，则根据所述预设倍数对调整后的下采样三维影像进行上采样，得到所述配准三维影像；基于从所述三维影像到所述配准三维影像的位姿变换过程确定所述第一转换矩阵。
在一些实施例中，可以基于至少一张所述第一投影影像和所述至少一张下采样二维影像确定第一相似度；响应于所述第一相似度大于相似度阈值，确定所述至少一张第一投影影像和所述至少一张下采样二维影像满足第一预设条件。
在一些实施例中，在对所述每一张二维影像，基于所述二维影像的拍摄位姿对所述调整后的下采样三维影像进行投影，得到对应的第一投影影像之后，还可以响应于至少一张所述第一投影影像和所述至少一张下采样二维影像不满足第一预设条件，则按照所述预设步长调整所述调整后的下采样三维影像的位姿，以重复上述得到与每一张二维影像对应的第一投影影像的过程，直至至少一张所述第一投影影像和所述至少一张下采样二维影像满足所述第一预设条件。
在一些实施例中，响应于至少一张所述第一投影影像和所述至少一张下采样二维影像满足所述第一预设条件，且所述预设倍数不满足所述第二预设条件，则可以根据所述预设倍数对所述调整后的下采样 三维影像进行上采样，以更新所述三维影像；降低所述预设倍数，基于降低后的所述预设倍数对所述更新后的三维影像和所述至少一张二维影像进行下采样，以重复上述得到与每一张二维影像对应的第一投影影像的过程，直至至少一张所述第一投影影像和所述至少一张下采样二维影像满足所述第一预设条件，且所述预设倍数满足所述第二预设条件。
在一些实施例中，可以响应于所述预设倍数等于阈值，确定至少一张所述第一投影影像和所述至少一张下采样二维影像满足所述第二预设条件。
在一些实施例中，可以获取二维成像设备坐标系和所述手术空间坐标系之间的第二转换矩阵；基于从所述三维影像到所述配准三维影像的位姿变换过程，得到所述三维影像坐标系和所述二维成像设备坐标系之间的第三转换矩阵；基于所述第二转换矩阵和所述第三转换矩阵得到所述第一转换矩阵。
在一些实施例中，可以在所述配准三维影像中确定对应所述目标部位的所述三维目标影像。对所述至少一张二维影像中的每一张：获取所述二维影像对应的二维影像坐标系和所述手术空间坐标系之间的第四转换矩阵；基于所述三维目标影像、所述第一转换矩阵和所述第四转换矩阵，在所述二维影像中确定所述二维目标影像。
在一些实施例中，可以基于所述三维目标影像，确定所述目标部位的代表点在所述三维影像坐标系中的三维坐标和所述目标部位的尺寸参数；基于所述三维坐标、所述第一转换矩阵和所述第四转换矩阵确定所述代表点在所述二维影像坐标系中的二维坐标；基于所述二维坐标和所述尺寸参数，在所述二维影像中确定所述二维目标影像。
在一些实施例中，可以获取二维成像设备坐标系和所述手术空间坐标系之间的第二转换矩阵；获取所述二维成像设备坐标系和所述二维影像坐标系之间的第五转换矩阵；基于所述第二转换矩阵和所述第五转换矩阵得到所述第四转换矩阵。
在一些实施例中，可以对所述每一张二维影像，基于所述二维影像的拍摄位姿对所述三维目标影像进行投影，得到对应的第二投影影像；基于至少一张所述第二投影影像和所述至少一张二维目标影像确定第二相似度；基于所述第二相似度对所述配准三维影像进行位姿调整，以优化所述第一转换矩阵，得到所述目标转换矩阵。
本说明书实施例之一提供一种配准系统，包括影像获取模块、位姿变换模块、目标影像确定模块和目标矩阵确定模块；所述影像获取模块用于获取目标对象在手术前拍摄的三维影像和手术中拍摄的至少一张二维影像；所述位姿变换模块用于基于所述至少一张二维影像，对所述三维影像进行位姿变换以得到配准三维影像，以及所述三维影像对应的三维影像坐标系和所述手术对应的手术空间坐标系之间的第一转换矩阵；所述目标影像确定模块用于在所述至少一张二维影像中确定所述目标对象的目标部位对应的二维目标影像，在所述配准三维影像中确定所述目标部位对应的三维目标影像；所述目标矩阵确定模块用于基于所述三维目标影像和每张二维影像中的所述二维目标影像，对所述配准三维影像进行位姿变换，以优化所述第一转换矩阵，得到目标转换矩阵。
本说明书实施例之一提供一种计算机可读存储介质，所述存储介质存储计算机指令，当计算机读取存储介质中的计算机指令后，计算机执行所述配准方法。
本说明书一些实施例中，通过基于术中拍摄的二维影像对术前拍摄的三维影像进行位姿变换，实现二维影像和三维影像之间的粗配准；基于目标部位对应的二维目标影像和三维目标影像，对粗配准三维影像继续进行位姿变换，实现二维影像和三维影像之间的精配准。通过上述两次配准，无需在三维影像和二维影像中手动选取解剖点，无需大量的人机交互，大大简化了操作流程，同时也避免了大量人机交互带来误差，从而提高了三维影像与二维影像配准的准确度。进一步的，可以提高基于配准结果确定的三维影像坐标系和手术空间坐标系的转换矩阵的准确性，以及手术引导的准确性。
此外，本说明书的一些实施例中，可以基于三维目标影像从二维影像中确定目标部位对应的二维目标影像。由于二维影像通常成像清晰度没有三维影像高，且二维影像中可能会包含手术器件，直接对二维影像进行目标部位的分割可能不会得到很好的分割结果。通过基于三维目标影像确定二维目标影像，可以提高目标部位分割的准确度。
附图说明
本说明书将以示例性实施例的方式进一步说明，这些示例性实施例将通过附图进行详细描述。这些实施例并非限制性的，在这些实施例中，相同的编号表示相同的结构，其中：
图1是根据本说明书一些实施例所示的配准系统的应用场景示意图；
图2是根据本说明书一些实施例所示的处理设备的模块图；
图3是根据本说明书一些实施例所示的配准方法的示例性流程图；
图4是根据本说明书一些实施例所示的对三维影像和至少一张二维影像进行粗配准的流程的示例 性流程图；
图5是根据本说明书一些实施例所示的生成三维目标影像和二维目标影像的流程的示例性流程图；
图6是根据本说明书一些实施例所示的优化第一转换矩阵以得到目标转换矩阵的流程的示例性流程图；
图7A、7B是根据本说明书一些实施例所示的第一拍摄位姿的二维影像的示意图；
图8A、8B是根据本说明书一些实施例所示的第二拍摄位姿的二维影像的示意图；
图9是根据本说明书一些实施例所示的配准方法的示意图；
图10是根据本说明书一些实施例所示的配准方法的示意图；以及
图11是根据本说明书一些实施例所示的配准系统的结构示意图。
具体实施方式
为了更清楚地说明本说明书实施例的技术方案，下面将对实施例描述中所需要使用的附图作简单的介绍。显而易见地，下面描述中的附图仅仅是本说明书的一些示例或实施例，对于本领域的普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图将本说明书应用于其它类似情景。除非从语言环境中显而易见或另做说明，图中相同标号代表相同结构或操作。
应当理解，本文使用的“系统”、“装置”、“单元”和/或“模块”是用于区分不同级别的不同组件、元件、部件、部分或装配的一种方法。然而，如果其他词语可实现相同的目的，则可通过其他表达来替换所述词语。
如本说明书和权利要求书中所示，除非上下文明确提示例外情形，“一”、“一个”、“一种”和/或“该”等词并非特指单数，也可包括复数。一般说来，术语“包括”与“包含”仅提示包括已明确标识的步骤和元素，而这些步骤和元素不构成一个排它性的罗列，方法或者设备也可能包含其它的步骤或元素。
本说明书中使用了流程图用来说明根据本说明书的实施例的系统所执行的操作。应当理解的是，前面或后面操作不一定按照顺序来精确地执行。相反，可以按照倒序或同时处理各个步骤。同时，也可以将其他操作添加到这些过程中，或从这些过程移除某一步或数步操作。
图1是根据本说明书一些实施例所示的配准系统的应用场景示意图。
如图1所示，在一些实施例中，配准系统100可以包括医学影像设备110、处理设备120、存储设备130、终端140、网络150。配准系统100可以通过处理设备120实施本说明书中披露的方法和/或过程来实现对医学影像设备110获取的图像进行二维-三维配准。在本说明书中，二维-三维配准指的是对目标对象在手术前拍摄的三维影像和手术中拍摄的二维影像进行配准。在配准过程中，三维影像的位姿会被调整(例如，通过平移和/或旋转的方式)，使得调整后的三维影像(即配准后的三维影像)中目标对象的位姿与拍摄二维影像时的位姿一致。通过二维-三维配准，可以建立起三维影像对应的三维影像空间(也可以称为三维影像坐标系)和手术空间(也可以称为手术空间坐标系)的映射关系，从而将基于三维影像制定的手术计划准确地转化到手术空间中，从而保障手术的顺利进行，提高手术效果。
医学影像设备110是指医学上利用不同的方式，将目标物体(例如，人体、动物等)内部的结构重现为影像的装置。在一些实施例中，医学影像设备110可以是任何能够对患者的指定身体部位进行成像或治疗的医学设备，例如，计算机断层扫描(Computed Tomography，CT)设备、磁共振(Magnetic Resonance Imaging，MRI)设备、正电子发射型计算机断层(Positron Emission Computed Tomography，PET)设备、直接数字化X射线摄影(Direct Digit Radiography，DDR)设备、X光成像设备等。上面提供的医学影像设备110仅用于说明目的，而非对其范围的限制。
在一些实施例中，医学影像设备110中可以用于拍摄目标对象的二维和/或三维影像。在一些实施例中，医学影像设备110可以包括第一医学影像设备和第二医学影像设备。第一医学影像设备可以用于设备获取三维影像，第二医学影像设备可以用于拍摄二维影像。例如，第一影像设备可以是CT设备或MRI设备，其可以用于在手术前拍摄目标物体的三维影像。第二影像设备可以是X光成像设备，其可以用于在手术中拍摄目标物体的二维影像。所述三维影像可以用于制定手术规划(如，手术路径)，所述二维影像可以用于与三维影像进行配准得到变换矩阵，并使用变换矩阵将手术规划映射到手术空间中。在一些实施例中，医学影像设备110可以包括单个设备，该单个设备可以能够拍摄三维影像和二维影像。
在一些实施例中，医学影像设备110拍摄获取的影像等数据和/或信息可以被保存在存储设备130中。医学影像设备110可以接收医生通过终端140发送的指令等，并根据指令进行相关操作，例如，照射成像等。在一些实施例中，医学影像设备110可以通过网络150与配准系统100中的其它组件(例如，处理设备120、存储设备130、终端140)进行数据和/或信息的交换。在一些实施例中，医学影像设备110可以直接与配准系统100中的其它组件连接。在一些实施例中，配准系统100中的一个或多个组件(例如， 处理设备120、存储设备130)可以包括在医学影像设备110内。
处理设备120可以处理从其它设备或系统组成部分中获得的数据和/或信息，基于这些数据、信息和/或处理结果执行本说明书一些实施例中所示的配准方法，以完成一个或多个本说明书一些实施例中描述的功能。例如，处理设备120可以基于医学影像设备110采集的二维影像和三维影像进行二维-三维配准，从而确定三维影像空间和手术空间的转化关系。在一些实施例中，处理设备120可以将处理得到的数据，例如，坐标系间的转换矩阵、投影影像等，发送至存储设备130进行保存。在一些实施例中，处理设备120可以从存储设备130中获取预先存储的数据和/或信息，例如，二维影像、三维影像等，以用于执行本说明书一些实施例所示的配准方法，例如，进行二维-三维配准等。
在一些实施例中，处理设备120可以包含一个或多个子处理设备(例如，单核处理设备或多核多芯处理设备)。仅作为示例，处理设备120可以包括中央处理器(CPU)、专用集成电路(ASIC)、专用指令处理器(ASIP)、图形处理器(GPU)、物理处理器(PPU)、数字信号处理器(DSP)、现场可编程门阵列(FPGA)、可编辑逻辑电路(PLD)、控制器、微控制器单元、精简指令集电脑(RISC)、微处理器等或以上任意组合。
存储设备130可以存储其他设备产生的数据或信息。在一些实施例中，存储设备130可以存储医学影像设备110拍摄获取的数据和/或信息，例如，二维影像、三维影像等。在一些实施例中，存储设备130可以存储处理设备120处理后的数据和/或信息，例如，坐标系间的转换矩阵、投影影像等。存储设备130可以包括一个或多个存储组件，每个存储组件可以是一个独立的设备，也可以是其它设备的一部分。存储设备可以是本地的，也可以通过云实现。
终端140可以对医学影像设备110的操作进行控制。例如，医生可以通过终端140对医学影像设备110下达操作指令，以使医学影像设备110完成指定操作，例如，对患者指定身体部位拍摄成像。在一些实施例中，终端140可以通过指令使处理设备120执行如本说明书一些实施例所示的配准方法。在一些实施例中，终端140可以从处理设备120接收配准后的三维影像，以使用户可以对患者进行有效和针对性检查和/或治疗。在一些实施例中，终端140可以是移动设备140-1、平板计算机140-2、膝上型计算机140-3、台式计算机等其他具有输入和/或输出功能的设备中的一种或其任意组合。
网络150可以连接系统的各组成部分和/或连接系统与外部资源部分。网络150使得各组成部分之间，以及与系统之外其它部分之间可以进行通讯，促进数据和/或信息的交换。在一些实施例中，配准系统100中的一个或多个组件(例如，医学影像设备110、处理设备120、存储设备130、终端140)可通过网络150发送数据和/或信息给其它组件。在一些实施例中，网络150可以是有线网络或无线网络中的任意一种或多种。
应该注意的是，上述描述仅出于说明性目的而提供，并不旨在限制本说明书的范围。对于本领域普通技术人员而言，在本说明书内容的指导下，可做出多种变化和修改。可以以各种方式组合本说明书描述的示例性实施例的特征、结构、方法和其他特征，以获得另外的和/或替代的示例性实施例。例如，处理设备120可以是基于云计算平台的，例如公共云、私有云、社区和混合云等。然而，这些变化与修改不会背离本说明书的范围。
图2是根据本说明书一些实施例所示的处理设备的模块图。
如图2所示，在一些实施例中，处理设备120可以包括影像获取模块210、位姿变换模块220、目标影像确定模块230和目标矩阵确定模块240。
影像获取模块210可以用于获取目标对象(例如，人体、动物等)在手术前拍摄的三维影像和手术中拍摄的至少一张二维影像。
位姿变换模块220可以用于基于至少一张二维影像，对三维影像进行位姿变换以得到配准三维影像，以及三维影像对应的三维影像坐标系和手术对应的手术空间坐标系之间的第一转换矩阵。
目标影像确定模块230可以用于在至少一张二维影像中确定目标对象的目标部位对应的二维目标影像，在配准三维影像中确定目标部位对应的三维目标影像。在一些实施例中，目标部位可以包括目标脊椎、目标髋关节、目标肘关节、目标膝关节和目标手指关节等中的至少一种。
目标矩阵确定模块240可以用于基于三维目标影像和每张二维影像中的二维目标影像，对配准三维影像进行位姿变换，以优化第一转换矩阵，得到目标转换矩阵。
在一些实施例中，处理设备120还可以包括进一步路径信息获取模块和指导信息生成模块(图2未示出)。路径信息获取模块可以用于获取三维影像对应的手术路径信息。指导信息生成模块可以用于基于目标转换矩阵，将手术路径信息转换至手术空间坐标系中，以生成手术指导信息。
关于上述模块的更多描述可以参考本说明书其他地方，例如，图3、图4、图5等，在此不再赘述。
图3是根据本说明书一些实施例所示的配准方法的示例性流程图。
在一些实施例中，流程300可以由配准系统100执行。例如，流程300可以以指令集(例如，应用程序)的形式存储在存储设备(例如，存储器130)中。在一些实施例中，处理设备120(例如，图2中所示的一个或多个模块)可以执行指令集并相应指示配准系统100的一个或多个组件执行流程300。如图3所示，流程300可以包括下述步骤。
步骤310，获取目标对象在手术前拍摄的三维影像和手术中拍摄的至少一张二维影像。在一些实施例中，步骤310可以由影像获取模块210执行。
目标对象是指接受手术或检查的对象，例如，生物体、模体等。在一些实施例中，目标对象可以包括人体或其一部分。例如，目标对象可以包括需要进行手术的患者或其一部分(例如，脊柱、髋关节、肘关节、膝关节、手指等)。在一些实施例中，目标对象包括目标部位。目标部位是指作为目标对象中需要接受手术和/或检查的器官、组织等身体区域，例如，目标脊椎、目标髋关节、目标肘关节、目标膝关节和目标手指关节等。
三维影像是指通过三维医学影像设备拍摄目标对象得到的影像，例如，CT影像、MRI影像、PET影像等。在一些实施例中，影像获取模块210可以在手术前通过三维医学影像设备扫描目标对象，以获得三维影像。该三维影像可以用于检查、诊断、术前准备和术中路径定位等。
二维影像是指通过二维医学影像设备拍摄人体得到的影像，例如，DDR影像、X光机影像等。在一些实施例中，影像获取模块210可以在手术中通过二维医学影像设备扫描目标对象，以获得二维影像。二维影像可以用于通过二维影像与三维影像配准，使得三维影像中的目标对象位姿与手术中目标对象的位姿保持一致，并得到三维影像空间与手术空间之间的转换矩阵，利用转换矩阵将手术计划转换到手术空间中。在一些实施例中，二维影像可以包括至少两张，其中，这些二维影像的拍摄角度可以不同。例如，二维影像可以包括正位二维影像和侧位二维影像。拍摄正位二维图像时放射源位于目标对象正前方进行照射，拍摄侧位二维图像时放射源位于目标对象侧面进行照射。在一些实施例中，这些二维影像中存在至少两张二维影像，每两张二维影像之间的角度处于预设角度区间。其中，两张二维影像之间的角度指的是二维医学影像设备在拍摄两张二维影像时出射的两组射线之间的角度，也是两张二维影像的成像面之间的角度。在一些实施例中，预设角度区间可以为50度至130度。
在一些实施例中，影像获取模块210还可以通过其他方式获取目标对象在手术前拍摄的三维影像和手术中拍摄的至少一张二维影像，例如，从存储器等各种数据源获取等。
步骤320，基于至少一张二维影像，对三维影像进行位姿变换以得到配准三维影像，以及三维影像对应的三维影像坐标系和手术对应的手术空间坐标系之间的第一转换矩阵。在一些实施例中，步骤320可以由位姿变换模块220执行。
步骤320中对三维影像的位姿调整是基于对三维影像整体的分析进行，该过程可以视为对三维影像进行粗配准的过程。通过执行粗配准，可以将三维影像中目标对象的位姿(即位置和姿态)调整至与拍摄二维影像时的位姿(即手术中的当前位姿)大致一致。配准三维影像是指对三维影像进行粗配准得到的影像。第一转换矩阵是指经过粗配准结果得到的三维影像对应的三维影像坐标系和手术对应的手术空间坐标系之间进行转换的转换矩阵。
手术对应的手术空间坐标系也称为手术空间、手术坐标系，是以某个预设位置为参照点生成的用于引导手术的坐标系。在一些实施例中，手术空间坐标系可以包括基于光学跟踪系统(Optical Tracking System)设置的坐标系。OTS可以通过光学成像技术拍摄整个手术场景。例如，OTS系统可以包括双目相机。手术空间坐标系可以以双目相机上的某个点作为原点，以水平方向作为x轴，竖直方向作为z轴，前后方向作为y轴。
三维影像对应的三维影像坐标系也称为三维影像空间，是以三维影像为基础建立的三维坐标系。例如，三维影像中的目标对象为病人，可以以病人的中心点作为原点，以病人的左右方向作为x轴，病人的前后方向作为y轴，病人的头脚方向作为z轴建立三维影像坐标系。
在一些实施例中，处理设备120可以对三维影像的位姿进行调整，得到调整后的三维影像。进一步地，处理设备120可以在不同投影位姿对调整后的三维影像进行投影，得到每张二维影像对应的第一投影影像。其中，对于相对应的二维影像和第一投影影像，二维影像的拍摄位姿和第一投影影像的投影位姿相同。其中，拍摄位姿是指拍摄二维影像时二维医学影像设备的位姿，投影位姿是指通过投影获取投影影像时虚拟拍摄设备的位姿。例如，拍摄二维影像时可以通过OTS设备得到手术坐标系下的C臂位姿(即该坐标系下C臂的射线源、成像平面中心的位置坐标)，在三维影像坐标系中将虚拟C臂的射线源、成像平面中心的位置坐标也设置为与手术坐标系下的一致，这样就可以保证投影位姿与拍摄位姿相同。在得到第一投影影像后，处理设备120可以基于每张二维影像和对应的第一投影影像确定第一相似度，并根据第一相似度判断是否继续对调整后的三维影像的位姿进行调整。在一些实施例中，可以确定每张二维影像和对应的第一投影影像之间的相似度，基于所确定的所有相似度得到第一相似度。例如，第一相似度可以是 所确定的所有相似度的和、平均值、极大值等中的其中一种。若第一相似度较小，则继续对调整后的三维影像的位姿进行调整，以更新调整后的三维影像，并重复上述确定相似程度的过程，直至根据相似程度判断不再继续对调整后的三维影像的位姿进行调整，得到配准三维影像。关于如何对三维影像进行位姿变换以得到配准三维影像的更多内容，可以参见步骤410-460的相关描述，在此不再赘述。
在一些实施例中，可以基于配准三维影像确定第一转换矩阵。具体来说，粗配准过程中，三维影像的位姿调整可以基于旋转步长和平移步长进行，根据整个调整过程中的总旋转步长和总平移步长，可以确定第一转换矩阵。关于如何基于配准三维影像确定第一转换矩阵的更多内容，可以参见步骤470的相关描述，在此不再赘述。
在粗配准中，可能会存在三维影像在不同投影位姿下的投影影像与对应的二维影像未完全对准的情况。以脊柱为例，脊柱整体并不是刚性的，但脊柱中的每段脊椎是刚性的。由于术前和术中的脊柱通常会发生相对运动，粗配准后得到的配准三维影像中，虽然脊柱整体上与手术空间坐标系中的脊柱对准了，但是单个脊椎可能仍然没有完全对准。如果基于配准三维影像进行手术引导，可能会地影响手术效果，甚至导致手术失败。
在一些实施例中，在粗配准之后，可以通过后续的步骤330和步骤340来对配准三维影像与二维影像进行的精配准。与粗配准不同的是，精配准中对配准三维影像的位姿调整是基于目标部位进行的。
步骤330，在至少一张二维影像中确定目标对象的目标部位对应的二维目标影像，在配准三维影像中确定目标部位对应的三维目标影像。在一些实施例中，步骤330可以由目标影像确定模块230执行。
二维目标影像是指包括目标部位的二维影像，例如，目标脊椎的二维影像等。三维目标影像是指包括目标部位的三维影像，例如，目标脊椎的三维影像等。在一些实施例中，二维目标影像和/或三维目标影像可以只包括目标部位的影像(或少量其周围部位)，而原始的二维影像和三维影像中可能包括除了目标部位以外的其他身体部位。例如，目标部位是L3脊椎，二维目标影像和三维目标影像只包括L3脊椎，而原始的三维影像和二维影像中包括L1脊椎、L2脊椎、L3脊椎、L4脊椎和L5脊椎。
在一些实施例中，可以在至少一张二维影像和配准三维影像中分别确定目标对象的目标部位对应的二维目标影像和三维目标影像。例如，可以利用第一图像分割算法(例如，二维影像分割模型)从二维影像中将目标部位对应的部分分割出来，从而生成二维目标影像。可以利用第二图像分割算法(例如，三维影像分割模型)从配准三维影像中将目标部位对应的部分分割出来，从而生成三维目标影像。
在一些实施例中，可以在配准三维影像中确定包括目标部位的三维目标影像。对每一张二维影像，可以获取二维影像对应的二维影像坐标系和手术空间坐标系之间的第四转换矩阵，并基于三维目标影像、第一转换矩阵和第四转换矩阵，确定二维目标影像。也就是说，可以先确定三维目标影像，再基于三维目标影像确定二维目标影像。二维影像坐标系也称为二维影像空间，是以二维影像为基础建立的二维坐标系。例如，可以将二维影像的左上角作为原点，二维影像从左到右的方向作为x轴正向，二维影像从上到下的方向作为y轴正向。关于如何确定二维目标影像和三维目标影像的更多内容，可以参见图5的相关描述，在此不再赘述。
步骤340，基于三维目标影像和每张二维影像中的二维目标影像，对配准三维影像进行位姿变换，以优化第一转换矩阵，得到目标转换矩阵。在一些实施例中，步骤340可以由目标矩阵确定模块240执行。
目标转换矩阵是指根据精配准结果得到的三维影像对应的三维影像坐标系和手术对应的手术空间坐标系之间的转换矩阵。
在一些实施例中，对于每一张二维影像，可以基于二维影像的拍摄位姿对三维目标影像进行投影，得到对应的第二投影影像。进一步地，可以基于至少一张第二投影影像和至少一个二维目标影像确定第二相似度。然后，可以基于第二相似度对配准三维影像进行位姿调整，以优化第一转换矩阵，得到目标转换矩阵。关于如何得到目标转换矩阵的更多内容，可以参见图6的相关内容，在此不再赘述。
在一些实施例中，处理设备120可以进一步获取三维影像对应的手术路径信息。例如，手术路径信息可以包括手术中手术器械经过的目标对象的目标点在三维影像坐标系中的坐标。处理设备120可以基于得到的目标转换矩阵，将手术路径信息转换至手术空间坐标系中，以生成手术指导信息。例如，基于目标转换矩阵，可以将目标点在三维影像坐标系中的坐标转化为手术空间坐标系中的坐标，以作为手术指导信息。又例如，手术指导信息可以包括手术器械的位置是否正确的判断结果和/或指导手术器械移动位置的提示信息等。
本说明书一些实施例中，通过基于术中拍摄的二维影像对三维影像进行位姿变换，得到配准三维影像以及三维影像坐标系和手术空间坐标系之间的转换矩阵，实现三维影像和二维影像之间的粗配准。进一步地，基于目标部位对应的二维目标影像和三维目标影像，对粗配准三维影像继续进行位姿变换，优化转换矩阵，实现二维影像和三维影像的精配准。通过两次配准，可以提高配准的准确率，进而提高确定的三维影像空间和手术空间之间的目标转化矩阵的准确性，以及手术引导的准确性。此外，本说明书中的二 维-三维配准方法可以在无人为介入或者少量人为介入的情况下实施，无需人机交互(如在三维影像和二维影像中手动选取解剖点)，大大简化了操作流程，同时也避免了大量人机交互带来误差，从而提高了三维影像与二维影像配准的准确度。
图4是根据本说明书一些实施例所示的对三维影像和至少一张二维影像进行粗配准的流程的示例性流程图。
在一些实施例中，位姿变换模块220可以通过流程400所示的方法来基于至少一张二维影像对三维影像进行位姿变换以得到配准三维影像，以及确定三维影像对应的三维影像坐标系和手术对应的手术空间坐标系之间的第一转换矩阵。流程400可以用于实现图3所示的步骤320。如图4所示，流程400可以包括下述步骤。
步骤410，基于预设倍数对三维影像和至少一张二维影像进行下采样，得到下采样三维影像和至少一张下采样二维影像。
在一些实施例中，位姿变换模块220可以按照预设倍数对三维影像和至少一张二维影像进行下采样，即按照预设倍数缩小这些三维影像和二维影像。预设倍数可以是8、4、2等任意数值。
步骤420，基于预设步长调整下采样三维影像的位姿，得到调整后的下采样三维影像。
在一些实施例中，预设步长可以包括预设旋转步长和/或预设平移步长。位姿变换模块220可以按照预设旋转步长对下采样三维影像进行旋转，和/或按照预设平移步长对下采样三维影像进行平移。预设旋转步长可以是10°、5°、2°等任意角度。预设平移步长可以是20mm、10mm、5mm等任意长度。
在一些实施例中，预设倍数和预设步长可以是配准系统100的默认设置，或者由用户人为设定，或者由位姿变换模块220根据实际需求确定。在一些实施例中，预设倍数和预设步长具有特定的对应关系。例如，预设倍数8对应的预设旋转步长为10°，预设平移步长为20mm；设定预设倍数4对应的预设旋转步长为5°，预设平移步长为10mm；设定预设倍数2对应的预设旋转步长为5°，预设平移步长为5mm。
步骤430，对每一张二维影像，基于二维影像的拍摄位姿对调整后的下采样三维影像进行投影，得到对应的第一投影影像。
在一些实施例中，对每一张二维影像，位姿变换模块220可以在特定的投影位姿对调整后的下采样三维影像进行投影，得到其对应的第一投影影像，其中，第一投影影像的投影位姿与二维影像的拍摄位姿相同。下采样处理并未改变二维影像的拍摄位姿，因此，二维影像的拍摄位姿与对应的下采样二维影像的拍摄位姿相同，进而，二维影像对应的第一投影影像的投影位姿和下采样二维影像的拍摄位姿也相同。仅作为示例，二维影像可以包括正位二维影像和侧位二维影像，则位姿变换模块220可以从正位和侧位分别对调整后的下采样三维影像进行投影，得到正位第一投影影像和侧位第一投影影像。
在一些实施例中，位姿变换模块220可以采用数字重建放射影像(Digitally Reconstructed Radiograph，DRR)投影等方式将调整后的下采样三维影像投影为第一投影影像。
在一些实施例中，在得到每一张二维影像对应的第一投影影像之后，可以确定至少一张第一投影影像和至少一张下采样二维影像是否满足预设条件。在一些实施例中，可以通过以下步骤440和步骤450来确定至少一张第一投影影像和至少一张下采样二维影像满足预设条件。
步骤440，基于至少一张第一投影影像和至少一张下采样二维影像确定第一相似度。
第一相似度用于衡量第一投影影像和对应的下采样二维影像之间的相似的程度。在一些实施例中，位姿变换模块220可以通过各种方式确定每个第一投影影像和对应的下采样二维影像之间的相似度，基于所确定的所有相似度得到第一相似度。例如，第一相似度可以是所确定的所有相似度的和、平均值、极大值等的其中一种。
在一些实施例中，第一投影影像和其对应的下采样二维影像之间的相似度可以通过多种相似度算法确定。示例性的相似度算法可以包括互信息、模式强度、梯度差值等。
互信息(MI，Mutual Information)用来表示两个系统间的统计相关性，或者是一个系统中所包含的另一个系统中信息的多少。在一些实施例中，两幅图像间的互信息可以用以下公式来表示：
Figure PCTCN2022108552-appb-000001
其中，p(x)、p(y)分别表示待配准的两幅图像的边缘概率分布；p(x,y)表示待配准的两幅图像的联合概率分布；S表示待配准的两幅图像间的互信息值。
模式强度(PI，Pattern Intensity)是基于待配准图像间的差值图像进行计算，其中，待配准的目标称作“模式”。具体来说，将两幅图像相减得到I dif，当达到配准状态时，I dif中待配准的模式会消失，模式的强度会减到最小。在一些实施例中，两幅图像间的模式强度可以用以下公式来表示：
Figure PCTCN2022108552-appb-000002
其中，I dif表示待配准的两幅图像相减后的图像；r表示每个像素的模式强度有效计算区域半径；i、j、v、 w表示图像中的像素坐标；P r,σ表示最终的模式强度值；I dif(I,j)表示两幅待配准图像的相减图像中具体某个坐标的像素值，I dif(v,w)表示两幅待配准图像的相减图像中某个像素坐标邻域内的坐标的像素值；d表示半径，d 2表示以d为半径的圆形有效区域，其中，d 2＝(i-v) 2+(j-w) 2；常数σ是函数的权重，用来消除噪声的干扰。
梯度差值(GD，Gradient Difference)也是基于差值图像实现的，但是该差值图像是由梯度图像得到的。具体来说，用水平和垂直索贝尔(Sobel)算子对两幅图像进行处理，生成四幅梯度图像，分别表示两幅配准图像在两个正交坐标轴方向上的变化率。在一些实施例中，两幅图像间的梯度差值测度可以用以下公式来表示：
Figure PCTCN2022108552-appb-000003
Figure PCTCN2022108552-appb-000004
其中，dI fl/di表示待配准的2D图像在水平方向的梯度图像；dI DRR/di表示DRR图像在水平方向的梯度图像；I diffV表示待配准2D图像的梯度图像与DRR图像的梯度图像相减的图像；s、A v、A h表示函数权重；G(s)表示最终的梯度差值；I diffV(I,j)表示两幅待配准图像的水平梯度图像的相减图像在坐标(I,j)处的像素值；I diffH(I,j)表示两幅待配准图像的竖直梯度图像的相减图像在坐标(I,j)处的像素值。
本说明书一些实施例中，通过采用DRR投影的方式将3D影像投影为数量与待配准2D影像相同的2D影像，并进行相似度的计算，使相似度的获取更加准确，进而可以提高配准准确度。
步骤450，响应于第一相似度大于相似度阈值，确定至少一张第一投影影像和至少一张下采样二维影像满足第一预设条件。
相似度阈值可以具有任何数值。在一些实施例中，相似度阈值可以是配准系统100的默认设置，或者由用户人为设定，或者由位姿变换模块220根据实际需求确定。在一些实施例中，相似度阈值与预设倍数具有对应关系。例如，当二维影像包括两张时，可以设定预设倍数8对应的相似度阈值为0.2，预设倍数4对应的相似度阈值为0.4，预设倍数2对应的相似度阈值为0.6。在一些实施例中，第一预设条件可以包括第一相似度大于相似度阈值等。
在一些实施例中，当第一相似度大于相似度阈值时，可以确定第一投影影像和对应的下采样二维影像满足第一预设条件。当确定第一投影影像和对应的下采样二维影像满足第一预设条件时，则继续执行步骤460。若不满足，则重复执行步骤420-步骤440，直到满足第一预设条件。也就是说，若第一相似度不大于相似度阈值，则返回执行步骤420，按照预设步长再次调整调整后的下采样三维影像的位姿；执行步骤430，得到每张二维影像对应的第一投影影像；执行步骤440,基于得到的至少一张第一投影影像和至少一张下采样二维影像确定第一相似度，重复执行上述过程，直至第一相似度大于相似度阈值。
步骤460，响应于至少一张第一投影影像和至少一张下采样二维影像满足第一预设条件，则根据预设倍数对调整后的下采样三维影像进行上采样，得到配准三维影像。上采样操作可以放大调整后的下采样三维影像，使得到的配准三维影像与原始的三维影像具有相同的大小。
在一些实施例中，在确定至少一张第一投影影像和至少一张下采样二维影像满足第一预设条件后，位姿变换模块220可以进一步确定预设倍数是否满足第二预设条件。若预设倍数满足第二预设条件，则位姿变换模块220可以对调整后的下采样三维影像进行上采样，并将上采样后的图像作为配准三维影像。例如，第二预设条件可以包括预设倍数等于倍数的预设值等。其中，倍数的预设值是预先设定的较小的值。若预设倍数大于倍数的预设值，表示预设倍数较大，下采样三维影像和三维影像的尺寸差距较大，需要按照预设下采样步长降低预设倍数，以缩小下采样三维影像和三维影像的尺寸差距。通过逐级降低下采样预设倍数，在每个预设倍数下逐次调整下采样三维影像的位姿，实现了多级暴力搜索，提高了粗配准速度。
在一些实施例中，若预设倍数不满足第二预设条件，即预设倍数不等于倍数的预设值(例如，大于预设值)，则可以对调整后的下采样三维影像进行上采样得到更新后的三维影像。更新后的三维影像与原始的三维影像的位姿不同。进一步地，可以降低预设倍数，再次执行步骤410-步骤450。也就是说，可以基于降低后的预设倍数对更新后的三维影像和对应的二维影像进行下采样，并重复得到与每张二维影像对应的第一投影影像的过程，直至至少一张第一投影影像和对应的至少一张下采样二维影像满足第一预设条件，且预设倍数满足所述第二预设条件。
在一些实施例中，可以按照预设下采样步长降低预设倍数，具体来说，可以将预设倍数除以下采样步长，得到降低后的预设倍数。预设下采样步长可以根据需求设定。例如，预设下采样步长为2，按照预设下采样步长对预设倍数进行下采样处理，则是将预设倍数缩小2倍。
调整后的下采样三维影像已经满足第一预设条件，因此，基于调整后的下采样三维影像继续调整位姿，可以减少粗配准的时长。根据预设倍数将调整后的下采样三维影像的尺寸恢复至与初始的三维影像相同的尺寸(即上采样)，得到更新后的三维影像，以便于后续通过降低后的预设倍数对更新后的三维影 像进行下采样，缩小下采样三维影像和三维影像的尺寸差距。
在一些实施例中，还可以采用全局优化器调节预设倍数、预设步长等空间变换参数。优化问题包括多种类型的问题，例如，资源如何分配效益最高、拟合问题、最小最大值问题等等。优化问题一般分为局部优化和全局优化，局部优化，就是在函数值空间的一个有限区域内寻找最小值；而全局优化，是在函数值空间整个区域寻找最小值问题。函数局部最小点，即函数值小于或等于附近点的值，但是有可能大于较远距离的点的值的函数点；全局最小点，即函数值小于或等于所有的可行点的函数点。
在一些实施例中，当第一相似度不符合预设阈值，位姿变换模块220可以采用全局优化的方式对预设倍数、预设步长等空间变换参数进行调节，这样更快更准确地找到能够使第一相似度符合第一预设阈值的空间参数，效率更高。
在其它实施例中，位姿变换模块220还可以采用其他优化方式对预设倍数、预设步长等空间变换参数进行调节，此处不作具体限定。
步骤470，基于从三维影像到配准三维影像的位姿变换过程确定第一转换矩阵。
在一些实施例中，位姿变换模块220可以获取二维成像设备坐标系和手术空间坐标系之间的第二转换矩阵。二维成像设备坐标系是以手术中的医学影像设备(例如，二维医学影像设备等)为参照物生成的坐标系。例如，二维成像设备坐标系可以是以拍摄二维影像的二维成像设备上的一个特定位置(如，C臂上的某个点)作为原点，水平方向作为x轴，竖直方向作为z轴，前后方向作为y轴。
具体来说，在手术过程中或者手术前，可以预先在二维医学影像设备(例如，DR设备的C臂上)以及目标对象身体上分别放置第一追踪阵列和第二追踪阵列。追踪阵列可以包括若干个标志点(如反光球)。可以基于第一追踪阵列建立二维医学影像设备对应的设备空间坐标系(即二维成像设备坐标系)，基于第二追踪阵列建立术中空间坐标系(即手术空间坐标系)。利用OTS可以获取第一追踪阵列的坐标以及第二追踪阵列的坐标，从而确定二维成像设备坐标系与手术空间坐标系之间的转换矩阵作为第二转换矩阵。
进一步地，位姿变换模块220可以基于配准三维影像，得到三维影像坐标系和二维成像设备坐标系之间的第三转换矩阵。对三维影像进行调整得到配准三维影像的过程即建立起三维影像坐标系到二维成像设备坐标系的转换关系的过程。因此，可以基于该调整过程确定第一调整值，根据第一调整值确定第三转换矩阵。其中，第一调整值包括总旋转步长和总平移步长。例如，步骤420可能被多次进行。位姿变换模块220可以将每次执行步骤420时所用的旋转步长相加得到总旋转步长，每次执行步骤420时所用的平移步长相加得到总平移步长。
位姿变换模块220可以基于第二转换矩阵和第三转换矩阵得到第一转换矩阵。例如，第二转换矩阵和第三转换矩阵的乘积可以作为第一转换矩阵。
出于示例目的，下文提供了对三维影像进行位姿调整以得到配准三维影像的具体实例。假设二维影像包括两张，倍数的预设值为2，预设下采样步长为2，预设倍数8对应的相似度阈值为0.2，预设倍数4对应的相似度阈值为0.4，预设倍数2对应的相似度阈值为0.6，位姿变换模块220可以通过以下步骤得到配准三维影像，并确定第三转换矩阵。
分别对三维影像P01、第一拍摄位姿对应的二维影像P11和第二拍摄位姿对应的二维影像P21进行预设倍数为8的下采样处理，得到下采样三维影像P02、下采样二维影像P12和下采样二维影像P22。分别在第一投影位姿和第二投影位姿对P02进行投影，得到第一投影影像D1和第一投影影像D2，其中，第一拍摄位姿和第一投影位姿相同，第二拍摄位姿和第二投影位姿相同。基于P12、P22、D1和D2计算第一相似度，若第一相似度不大于0.2，则按照8对应的预设旋转步长10°和预设平移步长20mm调整P02的位姿，得到调整后的下采样三维影像P02，重复上述确定第一相似度的过程，直至确定的第一相似度大于0.2。进一步地，可以判断当前的预设倍数与倍数的预设值2是否相等。由于预设倍数8不等于倍数的预设值2，则根据预设倍数8对P02进行上采样，以更新三维影像P01得到三维影像P01’。按照预设下采样步长2降低预设倍数至4。
分别对三维影像P01’、第一拍摄位姿的二维影像P11和第二拍摄位姿的二维影像P21进行预设倍数为4的下采样处理，得到下采样三维影像P02’、下采样二维影像P12’和下采样二维影像P22’，分别在第一投影位姿和第二投影位姿对P02’进行投影，得到第一投影位姿的第一投影影像D1’，以及第二投影位姿的第一投影影像D2’，基于P12’、P22’、D1’和D2’计算第一相似度，若第一相似度不大于0.4，则按照4对应的预设旋转步长5°和预设平移步长10mm调整P02’的位姿，得到调整后的下采样三维影像P02’，重复上述确定第一相似度的过程，直至确定的第一相似度大于0.4，判断当前的预设倍数与倍数的预设值2是否相等，在该步骤中，由于预设倍数4不等于倍数的预设值2，则根据预设倍数4对P02’进行上采样，以更新三维影像P01得到三维影像P01”，按照预设下采样步长2降低预设倍数至2。
分别对三维影像P01”、第一投影位姿的二维影像P11和第二投影位姿的二维影像P21进行预设倍数为2的下采样处理，得到下采样三维影像P02”、下采样二维影像P12”和下采样二维影像P22”，分别 在第一投影位姿和第二投影位姿对P02”进行投影，得到第一投影位姿的第一投影影像D1”，以及第二投影位姿的第一投影影像D2”，基于P12”、P22”、D1”和D2”计算第一相似度，若第一相似度不大于0.6，则按照2对应的预设旋转步长2°和预设平移步长5mm调整P02”的位姿，得到调整后的下采样三维影像P02”，重复上述确定第一相似度的过程，直至确定的第一相似度大于0.6，判断当前的预设倍数与倍数的预设值2是否相等，在该步骤中，由于预设倍数2等于倍数的预设值2，则根据预设倍数2对P02”进行上采样，得到调整后三维影像，并确定第三转换矩阵。
本说明书一些实施例中，通过基于至少一张二维影像对三维影像进行位姿变换，实现了三维影像和二维影像之间的粗配准。在粗配准过程中，不需要在术前的三维影像和术中的多张二维影像中手动选取解剖点，不需要大量的人机交互，降低了配准复杂度。此外，通过对三维影像和二维影像进行下采样，并基于下采样后的图像进行分析，可以减少数据分析量、提高了粗配准速度，实现了快速、自动完成三维影像和二维影像之间的粗配准。
图5是根据本说明书一些实施例所示的生成三维目标影像和二维目标影像的流程的示例性流程图。
在一些实施例中，目标影像确定模块230可以通过流程500所示的方法在至少一张二维影像和配准三维影像中分别确定目标对象的目标部位对应的二维目标影像和三维目标影像。流程500可以用于实现图3所示的步骤330。在一些实施例中，当目标对象有多个目标部位时，可以对每个目标部分分别执行流程500。如图5所示，流程500可以包括下述步骤。
步骤510，在配准三维影像中确定对应目标部位的三维目标影像。三维目标影像是指从配准三维影像中截取的仅包含目标部位，或包含目标部位以及少量其周围部位的图像。例如，目标部位可以包括目标脊椎、目标髋关节、目标肘关节、目标膝关节和目标手指关节等中的至少一种。
在一些实施例中，目标影像确定模块230可以通过图像分割算法对配准三维影像进行分割，得到目标部位对应的分割掩膜。目标影像确定模块230可以基于分割掩膜，在配准三维影像中确定仅包括目标部位的三维目标影像。
在一些实施例中，在步骤510之后，对于上述至少一张二维影像中的每一张，可以通过步骤520，获取二维影像对应的二维影像坐标系和手术空间坐标系之间的第四转换矩阵；通过步骤530-步骤550，基于三维目标影像、第一转换矩阵和第四转换矩阵，确定二维目标影像。为方便描述，下文以一张二维影像为例，描述步骤520-步骤550的实施过程。
步骤520，获取二维影像对应的二维影像坐标系和手术空间坐标系之间的第四转换矩阵。
在一些实施例中，目标影像确定模块230可以获取二维成像设备坐标系和手术空间坐标系之间的第二转换矩阵。关于如何获取第二转换矩阵的更多内容，可以参见步骤470的相关描述，在此不再赘述。目标影像确定模块230还可以获取二维成像设备坐标系和二维影像坐标系之间的第五转换矩阵。具体来说，目标影像确定模块230可以获取二维医学影像设备拍摄二维影像时的内参(例如，放射源到成像平面的距离)，并基于该内参确定第五转换矩阵。目标影像确定模块230可以基于第二转换矩阵和第五转换矩阵得到二维影像对应的二维影像坐标系和手术空间坐标系之间的第四转换矩阵。
步骤530，基于三维目标影像，确定目标部位的代表点在三维影像坐标系中的三维坐标和目标部位的尺寸参数。
目标部位的代表点是指目标部位中具有代表性的特征点，例如，中心点、边界点等。在一些实施例中，目标部位的代表点可以包括目标部位的中心点，例如，脊椎中心。
目标部位的尺寸参数是指与目标部位的尺寸大小相关的参数，例如，长、宽、中心点到边的距离等。在一些实施例中，目标部位的尺寸参数可以包括目标部位的中心点到目标部位的边缘的距离的集合。可选的，目标部位的边缘可以用其外接矩形的各条边来表示。
在一些实施例中，目标影像确定模块230可以在三维目标影像中识别目标部位的代表点(例如，中心点)，从而得到目标部位的代表点在三维影像坐标系中的三维坐标。
在一些实施例中，目标影像确定模块230可以对三维目标影像进行投影(如DRR投影)，得到与二维影像对应的第二投影影像。其中，第二投影影像的投影位姿与二维影像的拍摄位姿相同。在一些实施例中，可以通过与得到第一投影影像相似的方法得到其他与二维影像对应的投影影像(例如，第二投影影像等)。目标影像确定模块230可以基于三维目标影像中目标部位的代表点确定第二投影影像的目标部位的代表点和目标部位的尺寸参数。例如，可以在第二投影影像中确定三维目标影像中目标部位的中心点(即第一中心点)投影得到的中心点(即第二中心点)，并确定第二投影影像中目标部位的最小外接矩形，获取第二中心点至最小外接矩形的每条边的距离。这些距离组成的距离集合可以作为目标部位的尺寸参数。也就是说，第二中心点(即第二投影影像中目标部位中心点)到目标部位的边缘的距离，指的是第二中心点到目标部位的最小外接矩形的每条边的距离。
仅作为示例，二维影像包括第一二维影像和第二二维影像。目标影像确定模块230可以在第一投影位姿和第二投影位姿对三维目标影像进行投影，得到第一二维影像对应的第二投影影像X1，和第二二维影像对应的第二投影影像X2；在X1中确定第一中心点投影得到的第二中心点f1，在X2中确定第一中心点投影得到的第二中心点f2；在X1中确定目标部位的最小外接矩形C1，确定f1到C1的四条边的距离，得到X1对应的距离集合A1；在X2中确定目标部位的最小外接矩形C2，确定f2到C2的四条边的距离，得到X2对应的距离集合A2。A1可以作为第一二维影像对应的尺寸参数。A2可以作为第二二维影像对应的尺寸参数。
步骤540，基于三维坐标、第一转换矩阵和第四转换矩阵确定代表点在二维影像坐标系中的二维坐标。三维坐标可以通过步骤530确定，第四转换矩阵可以通过步骤520确定，第一转换矩阵可以通过步骤320确定。
在一些实施例中，目标影像确定模块230可以通过第一转换矩阵，将三维影像坐标系中的目标部位的代表点的三维坐标，转换至手术空间坐标系中，得到第一转换点；根据二维影像对应的第四转换矩阵，可以将手术空间坐标系中的第一转换点转换至二维影像对应的二维影像坐标系中，从而得到代表点的二维坐标。
仅作为示例，目标影像确定模块230可以通过第一转换矩阵T1将三维中心点m1转换至手术空间坐标系，得到第一转换点m2；通过第一二维影像对应的第四转换矩阵T4a，将m2转换至第一二维影像对应的第一二维影像坐标系中，得到第一二维影像中的二维中心点n1(即目标部位的代表点在第一二维影像中的对应点)。目标影像确定模块230可以通过第二二维影像对应的第一转换矩阵T4b，将m2转换至第二二维影像对应的第二二维影像坐标系中，得到第二二维影像中的二维中心点n2(即目标部位的代表点在第二二维影像中的对应点)。
步骤550，基于二维坐标和尺寸参数，确定二维目标影像。
例如，如步骤530所述，尺寸参数可以包括目标部位的中心点到目标部位的外接矩形的距离的集合。目标影像确定模块230可以以二维坐标为中心，根据距离的集合确定截取区域。该截取区域可以作为二维目标影像。仅作为示例，可以基于第一二维影像对应的第二中心点f1和距离合集A1，从第一二维影像中确定二维目标影像。可以基于第二二维影像对应的第二中心点f2和距离合集A2，从第二二维影像中确定二维目标影像。
在一些实施例中，可以通过像素点个数表示中心点到外接矩形的一条边的距离。在一些实施例中，距离集合可以包括左侧距离、右侧距离、上侧距离和下侧距离。在一些实施例中，目标影像确定模块230可以根据目标部位的代表点在二维影像坐标系中的二维坐标、左侧距离和上侧距离确定左上角坐标点；根据该二维坐标、右侧距离和上侧距离确定右上角坐标点；根据该二维坐标、左侧距离和下侧距离确定左下角坐标点；根据该二维坐标、右侧距离和下侧距离确定右下角坐标点；然后根据左上角坐标点、右上角坐标点、左下角坐标点和右下角坐标点，确定截取区域。
在一些实施例中，考虑到粗配准的误差，目标影像确定模块230可以对距离集合进行扩充处理，以使得二维目标影像可以包括完整的目标部位。例如，可以将左侧距离、右侧距离、上侧距离和下侧距离分别增加预设数值个像素点，例如，将左侧距离、右侧距离、上侧距离和下侧距离分别增加40个像素点。
在一些实施例中，可以通过分割算法对二维影像进行分割，得到二维目标影像，而非借助三维影像。
二维影像通常成像清晰度没有三维影像高，且图像中会包含手术器件，直接对二维影像进行目标脊椎的分割得到的分割效果可能不会很好。本说明书一些实施例中，通过基于配准后的三维影像中的三维目标影像确定二维影像中的目标影像，可以提高分割准确度。
图6是根据本说明书一些实施例所示的优化第一转换矩阵以得到目标转换矩阵的流程的示例性流程图。
在一些实施例中，目标矩阵确定模块240可以通过流程600所示的方法基于三维目标影像和二维目标影像，对配准三维影像进行位姿变换，以优化第一转换矩阵，得到目标转换矩阵。流程600可以用于实现图3所示的步骤340。在一些实施例中，流程600可以由目标矩阵确定模块240执行。如图6所示，流程600可以包括下述步骤。
步骤610，对每一张二维影像，基于二维影像的拍摄位姿对三维目标影像进行投影，得到对应的第二投影影像。
第二投影影像的生成过程与步骤430中描述的第一投影影像的生成过程类似，在此不再赘述。
步骤620，基于至少一张第二投影影像和至少一个二维目标影像确定第二相似度。
第二相似度可以用于衡量第二投影影像和对应的二维目标影像之间的相似程度。
在一些实施例中，与确定第一相似度类似，目标矩阵确定模块240可以通过各种方式(例如，互 信息、模式强度、梯度差值等相似度算法)确定每个第二投影影像和对应的二维目标影像之间的相似度，基于所有相似度得到第二相似度，例如，求和、求平均值、求极大值等。关于如何确定相似度的更多内容，可以参见步骤440的相关描述，在此不再赘述。
步骤630，基于第二相似度对配准三维影像进行位姿调整，以优化第一转换矩阵，得到目标转换矩阵。
在一些实施例中，目标矩阵确定模块240可以使用与得到配准三维影像相似的方法对配准三维影像进行位姿调整，得到目标转换矩阵。例如，可以执行与步骤450-470类似的步骤，其中，配准三维影像相当于步骤450-470中的原始三维影像，目标转换矩阵相当于步骤450-470中的第一转换矩阵，第二相似度相当于步骤450-470中的第一相似度。
在一些实施例中，目标矩阵确定模块240可以确定第二相似度是否满足第三预设条件。例如，第三预设条件可以是第二相似度与上一次调整配准三维影像的位姿时确定的第二相似度之间的相似度差值小于差值阈值。其中，阈值可以根据需求设定，例如，0.005等。又例如，第三预设条件可以是第二相似度大于相似度阈值。
在一些实施例中，目标矩阵确定模块240可以确定当前迭代次数是否满足第四预设条件。例如，第四预设条件可以是当前迭代次数是否等于预设迭代次数。其中，当前迭代次数是调整配准三维影像的位姿的次数，可以根据需求设定，例如，40次等。
在一些实施例中，若第二相似度不满足第三预设条件和/或当前迭代次数不满足第四预设条件，则目标矩阵确定模块240可以基于第二相似度调整配准三维影像的位姿，以优化第一转换矩阵，并重复确定第二相似度的过程，直至确定的第二相似度满足第三预设条件和/或当前迭代次数满足第四预设条件，则停止迭代，将此时优化后的第一转换矩阵确定为目标转换矩阵。
在一些实施例中，可以使用优化器(例如，Powell优化算法等)调整配准三维影像，当判断达到最优时则优化完成，将优化完成时的第一转换矩阵确定为目标转换矩阵。在一些实施例中，三维影像位姿达到最优的判断依据是优化器是否达到了迭代停止条件，例如，是否达到最大迭代次数(例如，40等)、是否达到最小步长(例如，0.05等)、是否达到步长变化容许大小(例如，0.005等)。
仅作为示例，目标矩阵确定模块240可以基于多张二维目标影像，对配准三维影像的位姿进行迭代调整，以对第一转换矩阵进行迭代优化，直至当前确定的第二相似度与上一次调整配准三维影像的位姿时确定的第二相似度之间的相似度差值小于阈值(例如，0.005)，或者当前迭代次数等于预设迭代次数(例如，40次)。在第i次调整配准三维影像的位姿后，基于调整后的配准三维影像确定多张第二投影影像，基于多张第二投影影像和多张二维目标影像确定第二相似度Si，计算Si和Si-1(第i-1次调整配准三维影像的位姿后计算得到的第二相似度)的相似度差值，并获取当前迭代次数i，若相似度差值不小于阈值，并且i不等于预设迭代次数，则基于Si再次调整调整后的配准三维影像的位姿。
在一些实施例中，目标矩阵确定模块240可以获取第二相似度与上一次调整配准三维影像的位姿时确定的第二相似度之间的相似度差值。目标矩阵确定模块240可以根据该相似度差值确定调整步长，并根据该调整步长再次调整调整后的配准三维影像的位姿。其中，调整步长包括调整旋转步长和调整平移步长。相似度差值和调整步长正相关，相似度差值越大，则调整步长越大。通过限定相似度差值和调整步长正相关，可以根据相似度差值自适应调整所述调整后的配准三维影像的位姿，加快迭代优化的速度，并且得到更准确的目标转换矩阵。
在一些实施例中，若调整步长与上一次调整配准三维影像的位姿时确定的调整步长之间的差值小于差值阈值，则可以得到目标转换矩阵。其中，该差值阈值可以根据需求设定，例如，0.005等。
在基于三维目标影像和每张二维影像中的二维目标影像，对配准三维影像的位姿进行调整的过程中，包括多次根据调整步长调整配准三维影像的位姿的过程，也包括确定该过程中产生的总调整步长。其中，总调整步长包括总调整旋转步长和总调整平移步长。目标矩阵确定模块240可以根据第一转换矩阵、总调整旋转步长和总调整平移步长确定目标转换矩阵。目标转换矩阵可以实现目标部位在三维影像坐标系与所述术中空间坐标系的精配准。
本说明书一些实施例中，精配准过程中，除确定目标部位外，不需要其他人机交互，操作简单。可以基于第二投影影像和二维目标影像确定第二相似度，进而基于第二相似度对配准三维影像的位姿进行自适应迭代调整，以对第一转换矩阵进行迭代优化，直至得到目标转换矩阵，加快了迭代优化第一转换矩阵的速度，并且可以得到更准确的目标转换矩阵。
应当注意的是，上述有关流程300、400、500、600的描述仅仅是为了示例和说明，而不限定本说明书的适用范围。对于本领域技术人员来说，在本说明书的指导下可以对流程300、400、500、600进行各种修正和改变。然而，这些修正和改变仍在本说明书的范围之内。例如，步骤520和530的顺序可以交换。
图9是根据本说明书一些实施例所示的配准方法的示意图。
在一些实施例中，可以通过处理设备120执行如图9所示的配准方法。其中，目标部位为脊柱中的一个脊椎。图9所示的配准方法包括以下步骤：
步骤901，获取脊柱在术前拍摄的三维影像Y0。
步骤902，对Y0进行预处理，得到每个脊椎的分割掩膜，以及每个脊椎的分割掩膜的标签。例如，可以通过图像分割算法对Y0中的脊椎进行分割，得到每个脊椎的分割掩膜，并为每个分割掩膜配置标签。
步骤903，获取脊柱在术中以第一拍摄位姿拍摄的二维影像Y1和第二拍摄位姿拍摄的二维影像Y2。例如，Y1可以包括如图7A所示的正位拍摄的脊柱二维影像，Y2可以包括如图8A所示的侧位拍摄的脊柱二维影像。
步骤904，基于Y1和Y2，确定Y1对应的第四转换矩阵T1a，以及Y2对应的第四转换矩阵T1b。
步骤905，根据Y0、Y1和Y2，进行粗配准，得到三维影像坐标系与手术空间坐标系之间的第一转换矩阵T2，以及配准三维影像Y4。
步骤906，指定目标部位的标签，以确定目标部位的分割掩膜，根据目标部位的分割掩膜，在Y4中确定三维目标影像y0，以及目标部位的第一中心点s1。
在一些实施例中，处理设备120可以通过执行步骤907-步骤910在二维影像中确定仅包括目标部位的二维目标影像。
步骤907，在第一投影位姿和第二投影位姿对y0进行投影，得到第一投影位姿的第二投影影像y01和第二投影位姿的第二投影影像y02。其中，第一投影位姿和第一拍摄位姿相同，第二投影位姿和第二拍摄位姿相同。
步骤908，确定y01中的目标部位的最小外接矩形rec1，在y02中的目标部位的最小外接矩形rec2。
步骤909，根据T2将s1转换到手术空间坐标系，得到s2，根据T1a将s2转换到Y1对应的二维影像坐标系，得到s3，根据T1b将s2转换到Y2对应的二维影像坐标系，得到s4。
步骤910，根据s3和rec1，在Y1中确定目标部位对应的二维目标影像y1；根据s4和rec2，在Y2中确定目标部位对应的二维目标影像y2。例如，y1可以包括如图7B所示的目标脊椎的正位二维影像，该目标脊椎的正位二维影像从图7A所示的目标脊椎的正位二维图像中分割而来。y2可以包括如图8B所示的目标脊椎的侧位二维影像，该目标脊椎的侧位二维影像从图8A所示的目标脊椎的侧位二维图像中分割而来。
在一些实施例中，处理设备120可以执行精配准过程，即基于三维目标影像和每张二维影像中的二维目标影像，对配准三维影像进行位姿变换，以优化第一转换矩阵，得到目标转换矩阵。
步骤911，根据y1、y2、y01和y02计算第二相似度；
步骤912，判断第二相似度是否满足第三预设条件，若不满足则执行步骤913，若满足则执行步骤915。其中，第三预设条件可以包括当前确定的第二相似度与上一次调整配准三维影像Y4的位姿时确定的第二相似度之间的相似度差值小于阈值。
步骤913，获取当前迭代次数i，判断当前迭代次数是否满足第四预设条件，若不满足则执行步骤914，若满足则执行步骤915。其中，第四预设条件可以包括当前迭代次数等于预设迭代次数。
步骤914，i＝i+1，并根据第二相似度调整Y4的位姿，再次执行步骤906；
步骤915，得到目标转换矩阵。
应当理解，图9所示的配准方法仅是示例性，其可以包括一个或多个其他步骤，或者省略一个或多个上述步骤。例如，步骤914可以省略。如果第二相似度阈值不满足第三预设条件，可以执行915。
图10是根据本说明书一些实施例所示的配准方法的示意图。
在一些实施例中，可以通过处理设备120执行如图10所示的配准方法，对多骨骼中的每一个骨骼进行配准。其中，目标部位为多骨骼中的每一个骨骼。
在一些实施例中，可以获取多骨骼的待配准的3D影像以及待配准的2D影像。如图10所示，可以对多骨骼3D影像1010进行DRR投影，得到DRR影像1020。其中，多骨骼3D影像1010可以包括CT、MRI等影像中的任意一种。在一些实施例中，多骨骼3D影像1010可以是包括多个目标部位的图像，例如，包括多个脊椎的脊柱等。
在一些实施例中，可以通过以下步骤获取3D影像：获取术前初始3D影像；基于第一位姿搜索参数对术前初始3D影像进行位姿搜索，得到第一3D影像，并获取第一3D影像与2D影像的第一位姿相似度；若第一位姿相似度符合第三预设阈值，则基于第二位姿搜索参数对第一3D影像进行位姿搜索，得到第二3D影像，并获取第二3D影像与2D影像的第二位姿相似度；若第二位姿相似度符合第四预设阈值，则基于初始3D影像、第一位姿搜索参数以及第二位姿搜索参数获取3D影像，并将上述过程中的术前初 始3D影像到3D影像的空间变换参数确定为定第一空间变换参数。其中，可以对第一3D影像按照2D影像的拍摄位姿进行投影，并计算其投影和与2D影像之间的相似度，作为第一位姿相似度。可以对第二3D影像按照2D影像的拍摄位姿进行投影，并计算其投影和与2D影像之间的相似度，作为第二位姿相似度。位姿搜索参数可以包括降采样预设倍数、预设旋转步长和预设平移步长。在一些实施例中，可以采用与步骤410和步骤420类似的方式进行位姿搜索。在一些实施例中，可以根据实际需求采用其他方式进行位姿搜索，本说明书不作具体限定。
仅作为示例，可以采用多分辨率多级搜索的方式对初始3D影像中关节部位的初始位姿进行估计。首先进行第一级位姿搜索，对初始3D影像进行8倍降采样，并在三维空间内对3D影像围绕X、Y、Z三个坐标轴每隔10°进行旋转、沿X、Y、Z三个坐标轴每隔20mm进行平移，然后进行第一位姿相似度计算，若相似度不满足预设停止条件，则重复迭代上述过程，直至相似度满足预设停止条件时，停止迭代并保留空间变换参数；然后进行第二级位姿搜索，对初始3D影像进行4倍降采样，将旋转间隔设置为5度，平移间隔设置为10mm，然后进行第二位姿相似度计算，若相似度不满足预设停止条件，则重复迭代上述过程，直至相似度满足预设停止条件时，停止迭代并保留空间变换参数。基于两级位姿搜索的空间变换参数确定关节部位的初始位姿。
在一些实施例中，可以基于多骨骼整体3D影像的DRR影像和2D图像进行粗配准，即将DRR影像与多张2D影像进行相似度分析，以进行多骨骼整体刚性配准。3D影像的DRR的投影位姿与3D图像的拍摄位姿一致。如图10所示，将DRR影像1020和2D影像1030通过步骤1040进行多骨骼整体刚性配准。例如，2D影像1030可以包括多张X射线图像等。
在一些实施例中，可以将3D影像基于第一空间变换参数进行空间变换，得到第一配准3D影像，并获取第一配准3D影像与2D影像的第一相似度。在一些实施例中，可以将第一配准3D影像通过投影(例如，DDR投影等)得到第一配准重建影像，其中，第一配准重建影像的数量与2D影像的数量相同，且第一配准重建影像的投影角度与2D影像的拍摄角度相同。在一些实施例中，可以获取第一配准重建影像与2D影像的相似度，作为第一相似度。在一些实施例中，可以通过与步骤430类似的方法对第一配准3D影像投影得到第一配准重建影像。在一些实施例中，可以通过与步骤440类似的方法得到第一相似度。
在一些实施例中，若第一相似度符合第一预设阈值，则可以认为可以完成了粗配准，即多骨骼整体刚性配准，可以将此时的第一配准3D影像确定为配准多骨骼3D影像。在一些实施例中，可以通过与步骤460类似的方法得到配准多骨骼3D影像。
在一些实施例中，在完成多骨骼整体刚性配准后，可以基于第一配准3D影像获取每一骨骼的单骨骼3D影像，基于2D影像获取每一骨骼的单骨骼2D影像，即对多骨骼影像通过骨骼分割得到单骨骼影像。
在一些实施例中，可以将单骨骼3D影像基于第二空间变换参数进行空间变换，得到每一骨骼的第二配准3D影像。在一些实施例中，可以通过与得到第一配准3D影像类似的方法得到第二配准3D影像。
在一些实施例中，可以获取第二配准3D影像与对应的单骨骼2D影像的第二相似度，若第二相似度符合第二预设阈值，则可以基于第二配准3D影像获取配准结果。在一些实施例中，可以通过与获取第一相似度类似的方法得到第二相似度。在一些实施例中，可以通过与步骤460和步骤470相似的方法得到配准结果。在一些实施例中，配准结果可以包括每一个骨骼的配准结果。
在完成多骨骼整体刚性配准后，可以得到配准多骨骼3D影像。然后可以对配准多骨骼3D影像进行自动骨骼分割，得到每一骨骼的3D影像，并对每一骨骼的3D影像进行投影重建，得到每一骨骼的DRR影像。如图10所示，在步骤1040后，可以基于通过步骤1050对配准多骨骼3D影像进行自动骨骼分割，得到每一骨骼的3D影像，即骨骼1、骨骼2、…、骨骼N，其中，N为大于或等于2的正整数；然后对骨骼1、骨骼2、…、骨骼N分别进行DRR投影，得到对应的投影图像DRR1、DRR2、…、DRRN。每一骨骼的DRR影像的投影位姿和2D影像1030的拍摄位姿一致。
在一些实施例中，可以直接对原始的多骨骼3D影像(即未进行整体刚性配准的多骨骼3D影像)直接进行自动骨骼分割，得到每一骨骼的3D影像。然后可以基于整体刚性配准结果对每一骨骼的3D影像进行位姿调整，再对调整后的每一骨骼的3D影像进行投影重建，得到每一骨骼的DRR影像。
在一些实施例中，可以基于每一骨骼的DRR影像和每一骨骼的2D影像像进行精配准，即将每一骨骼的投影图像与每一骨骼的2D影像进行相似度的比对，实现对每一骨骼的刚性配准，以得到每一骨骼的配准结果。如图10所示，可以将投影图像DRR1、DRR2、…、DRRN与2D影像1030中对应骨骼的2D图像分别进行刚性配准，得到对应的骨骼1配准结果、骨骼2配准结果、…、骨骼N配准结果。在一些实施例中，每个骨骼的配准结果可以指每个骨骼对应的目标转换矩阵(也称为配准矩阵)，即从三维影像坐标系到术中手术空间坐标系的转换矩阵。获取每一骨骼的配准矩阵作为配准结果，能够完整体现每一骨骼 的变换情况，配准精度更高，效果更好。
在一些实施例中，相似度的预设阈值(例如，第一预设阈值、第二预设阈值等)可以根据实际需求进行确定，具体的，可以根据配准部位进行确定。例如，采用上述配准方法对盆骨和股骨进行配准，且采用梯度差值作为相似度时，第一预设阈值可设置为0.5，第二预设阈值可设置为0.88，第三预设阈值可设置为0.2，第四预设阈值可设置为0.35。在一些实施例中，由于配准的部位不同以及选用的相似度函数不同，对应的预设阈值也可以不相同。例如，第一预设阈值的范围可以为0.4～0.8，第二预设阈值的范围可以为0.7～1.0，第三预设阈值的范围可以为0.2～0.4，第四预设阈值的范围可以为0.3～0.5。
本说明书一些实施例中，通过获取3D影像以及2D影像；将3D影像基于第一空间变换参数进行空间变换，得到第一配准3D影像，并获取第一配准3D影像与2D影像的第一相似度；若第一相似度符合第一预设阈值，则基于第一配准3D影像获取每一骨骼的单骨骼3D影像，基于2D影像获取每一骨骼的单骨骼2D影像；将单骨骼3D影像基于第二空间变换参数进行空间变换，得到每一骨骼的第二配准3D影像，并获取第二配准3D影像与对应的单骨骼2D影像的第二相似度；若第二相似度符合第二预设阈值，则基于第二配准3D影像获取配准结果的方式，采用多骨骼整体刚性配准结合单个骨骼刚性配准的方法，提高了配准算法的精度和成功率。另外，上述多骨骼影像配准方法无需人工交互，能够全自动完成影像配准，节约了人力成本。
图11是根据本说明书一些实施例所示的配准系统的结构示意图。
在一些实施例中，如图11所示的配准系统可以包括医学影像设备1110、处理器1120和导航设备1130。
在一些实施例中，医学影像设备1110可以用于在术中拍摄目标对象(例如，人体等)的至少一张张二维影像，和/或在术前拍摄目标对象的三维影像，并将拍摄的二维影像和/或三维影像发送至处理器1120。在一些实施例中，医学影像设备1110可以包括二维影像设备和三维影像设备等中的至少一种。
在一些实施例中，处理器1120可以用于获取目标对象在手术前拍摄的三维影像和手术中拍摄的至少一张二维影像，并对所获取影像进行2D-3D配准。例如，处理器1120可以执行本说明书图3-10所描述的配准方法。
在一些实施例中，导航设备1130可以用于基于手术指导信息将手术器械导航至目标手术区域。其中，目标手术区域与目标部位对应，例如，脊柱上的某个脊椎等。例如，导航设备1130可以基于配准结果(例如，目标转化矩阵)将手术路径信息转化成手术指导信息，以将手术器械导航至目标手术区域。在一些实施例中，导航设备30可以为NDI导航系统或其他手术导航系统。
本说明书实施例可能带来的有益效果包括但不限于：(1)通过对现有的2D-3D配准方法进行改进，通过自动粗配准、自动定位并提取二维影像中待配准解剖结构区域、只对待配准解剖结构进行DRR投影的方式，使得用户无需对术前、术中影像进行交互操作，大幅减少了交互过程，提升了用户体验和算法的临床可行性，实现了只需少量简单交互的半自动2D-3D配准和某些配准场景下的全自动2D-3D配准；(2)通过基于三维影像进行二维影像的分割，利用三维影像的清晰度的优势克服了二维影像成像清晰度低的缺点，且排除了二维影像中手术器件等的干扰，从而提高了分割准确度，保证了分割效果；(3)通过以上手段的综合运用，实现了二维影像中目标影像到手术空间坐标系中的精确映射，从而使得手术效果得到了保证。需要说明的是，不同实施例可能产生的有益效果不同，在不同的实施例里，可能产生的有益效果可以是以上任意一种或几种的组合，也可以是其他任何可能获得的有益效果。
上文已对基本概念做了描述，显然，对于本领域技术人员来说，上述详细披露仅仅作为示例，而并不构成对本说明书的限定。虽然此处并没有明确说明，本领域技术人员可能会对本说明书进行各种修改、改进和修正。该类修改、改进和修正在本说明书中被建议，所以该类修改、改进、修正仍属于本说明书示范实施例的精神和范围。
同时，本说明书使用了特定词语来描述本说明书的实施例。如“一个实施例”、“一实施例”、和/或“一些实施例”意指与本说明书至少一个实施例相关的某一特征、结构或特点。因此，应强调并注意的是，本说明书中在不同位置两次或多次提及的“一实施例”或“一个实施例”或“一个替代性实施例”并不一定是指同一实施例。此外，本说明书的一个或多个实施例中的某些特征、结构或特点可以进行适当的组合。
此外，除非权利要求中明确说明，本说明书所述处理元素和序列的顺序、数字字母的使用、或其他名称的使用，并非用于限定本说明书流程和方法的顺序。尽管上述披露中通过各种示例讨论了一些目前认为有用的发明实施例，但应当理解的是，该类细节仅起到说明的目的，附加的权利要求并不仅限于披露的实施例，相反，权利要求旨在覆盖所有符合本说明书实施例实质和范围的修正和等价组合。例如，虽然以上所描述的系统组件可以通过硬件设备实现，但是也可以只通过软件的解决方案得以实现，如在现有的服务器或移动设备上安装所描述的系统。
同理，应当注意的是，为了简化本说明书披露的表述，从而帮助对一个或多个发明实施例的理解， 前文对本说明书实施例的描述中，有时会将多种特征归并至一个实施例、附图或对其的描述中。但是，这种披露方法并不意味着本说明书对象所需要的特征比权利要求中提及的特征多。实际上，实施例的特征要少于上述披露的单个实施例的全部特征。
一些实施例中使用了描述成分、属性数量的数字，应当理解的是，此类用于实施例描述的数字，在一些示例中使用了修饰词“大约”、“近似”或“大体上”来修饰。除非另外说明，“大约”、“近似”或“大体上”表明所述数字允许有±20％的变化。相应地，在一些实施例中，说明书和权利要求中使用的数值参数均为近似值，该近似值根据个别实施例所需特点可以发生改变。在一些实施例中，数值参数应考虑规定的有效数位并采用一般位数保留的方法。尽管本说明书一些实施例中用于确认其范围广度的数值域和参数为近似值，在具体实施例中，此类数值的设定在可行范围内尽可能精确。
针对本说明书引用的每个专利、专利申请、专利申请公开物和其他材料，如文章、书籍、说明书、出版物、文档等，特此将其全部内容并入本说明书作为参考。与本说明书内容不一致或产生冲突的申请历史文件除外，对本说明书权利要求最广范围有限制的文件(当前或之后附加于本说明书中的)也除外。需要说明的是，如果本说明书附属材料中的描述、定义、和/或术语的使用与本说明书所述内容有不一致或冲突的地方，以本说明书的描述、定义和/或术语的使用为准。
最后，应当理解的是，本说明书中所述实施例仅用以说明本说明书实施例的原则。其他的变形也可能属于本说明书的范围。因此，作为示例而非限制，本说明书实施例的替代配置可视为与本说明书的教导一致。相应地，本说明书的实施例不仅限于本说明书明确介绍和描述的实施例。

Claims (23)

  1. 一种配准方法，由至少一个处理器执行，所述方法包括：
    获取目标对象在手术前拍摄的三维影像和手术中拍摄的至少一张二维影像；
    基于所述至少一张二维影像，对所述三维影像进行位姿变换以得到配准三维影像，以及所述三维影像对应的三维影像坐标系和所述手术对应的手术空间坐标系之间的第一转换矩阵；
    在所述至少一张二维影像中确定所述目标对象的目标部位对应的二维目标影像，在所述配准三维影像中确定所述目标部位对应的三维目标影像；以及
    基于所述三维目标影像和每张二维影像中的所述二维目标影像，对所述配准三维影像进行位姿变换，以优化所述第一转换矩阵，得到目标转换矩阵。
  2. 如权利要求1所述的方法，所述基于所述至少一张二维影像，对所述三维影像进行位姿变换以得到配准三维影像，以及所述三维影像对应的三维影像坐标系和所述手术对应的手术空间坐标系之间的第一转换矩阵，包括：
    基于预设倍数对所述三维影像和所述至少一张二维影像进行下采样，得到下采样三维影像和至少一张下采样二维影像；
    基于预设步长调整所述下采样三维影像的位姿，得到调整后的下采样三维影像；
    对所述每一张二维影像，基于所述二维影像的拍摄位姿对所述调整后的下采样三维影像进行投影，得到对应的第一投影影像；
    响应于至少一张所述第一投影影像和所述至少一张下采样二维影像满足第一预设条件，则根据所述预设倍数对调整后的下采样三维影像进行上采样，得到所述配准三维影像；以及
    基于从所述三维影像到所述配准三维影像的位姿变换过程确定所述第一转换矩阵。
  3. 如权利要求2所述的方法，进一步包括：
    基于至少一张所述第一投影影像和所述至少一张下采样二维影像确定第一相似度；以及
    响应于所述第一相似度大于相似度阈值，确定所述至少一张第一投影影像和所述至少一张下采样二维影像满足第一预设条件。
  4. 如权利要求2所述的方法，其特征在于，所述对所述每一张二维影像，基于所述二维影像的拍摄位姿对所述调整后的下采样三维影像进行投影，得到对应的第一投影影像之后，还包括：
    响应于至少一张所述第一投影影像和所述至少一张下采样二维影像不满足第一预设条件，则按照所述预设步长调整所述调整后的下采样三维影像的位姿，以重复上述得到与每一张二维影像对应的第一投影影像的过程，直至至少一张所述第一投影影像和所述至少一张下采样二维影像满足所述第一预设条件。
  5. 如权利要求2所述的方法，其特征在于，所述响应于至少一张所述第一投影影像和所述至少一张下采样二维影像满足第一预设条件，则根据所述预设倍数对调整后的下采样三维影像进行上采样，得到所述配准三维影像，包括：
    响应于至少一张所述第一投影影像和所述至少一张下采样二维影像满足所述第一预设条件，且所述预设倍数不满足所述第二预设条件，则根据所述预设倍数对所述调整后的下采样三维影像进行上采样，以更新所述三维影像；
    降低所述预设倍数，基于降低后的所述预设倍数对所述更新后的三维影像和所述至少一张二维影像进行下采样，以重复上述得到与每一张二维影像对应的第一投影影像的过程，直至至少一张所述第一投影影像和所述至少一张下采样二维影像满足所述第一预设条件，且所述预设倍数满足所述第二预设条件。
  6. 如权利要求5所述的方法，进一步包括：
    响应于所述预设倍数等于阈值，确定至少一张所述第一投影影像和所述至少一张下采样二维影像满足所述第二预设条件。
  7. 如权利要求2所述的方法，所述基于从所述三维影像到所述配准三维影像的位姿变换过程确定所述第一转换矩阵包括：
    获取二维成像设备坐标系和所述手术空间坐标系之间的第二转换矩阵；
    基于从所述三维影像到所述配准三维影像的位姿变换过程，得到所述三维影像坐标系和所述二维成像设备坐标系之间的第三转换矩阵；以及
    基于所述第二转换矩阵和所述第三转换矩阵得到所述第一转换矩阵。
  8. 如权利要求1所述的方法，所述在所述至少一张二维影像中确定所述目标对象的目标部位对应的二维目标影像，在所述配准三维影像中确定所述目标部位对应的三维目标影像，包括：
    在所述配准三维影像中确定对应所述目标部位的所述三维目标影像；
    对所述至少一张二维影像中的每一张，
    获取所述二维影像对应的二维影像坐标系和所述手术空间坐标系之间的第四转换矩阵；以及
    基于所述三维目标影像、所述第一转换矩阵和所述第四转换矩阵，在所述二维影像中确定所述二维目标影像。
  9. 如权利要求8所述的方法，所述基于所述三维目标影像、所述第一转换矩阵和所述第四转换矩阵，在所述二维影像中确定所述二维目标影像包括：
    基于所述三维目标影像，确定所述目标部位的代表点在所述三维影像坐标系中的三维坐标和所述目标部位的尺寸参数；
    基于所述三维坐标、所述第一转换矩阵和所述第四转换矩阵确定所述代表点在所述二维影像坐标系中的二维坐标；以及
    基于所述二维坐标和所述尺寸参数，在所述二维影像中确定所述二维目标影像。
  10. 如权利要求8所述的方法，所述获取所述二维影像对应的二维影像坐标系和所述手术空间坐标系之间的第四转换矩阵包括：
    获取二维成像设备坐标系和所述手术空间坐标系之间的第二转换矩阵；
    获取所述二维成像设备坐标系和所述二维影像坐标系之间的第五转换矩阵；以及
    基于所述第二转换矩阵和所述第五转换矩阵得到所述第四转换矩阵。
  11. 如权利要求1所述的方法，所述基于所述三维目标影像和每张二维影像中的所述二维目标影像，对所述配准三维影像进行位姿变换，以优化所述第一转换矩阵，得到目标转换矩阵包括：
    对所述每一张二维影像，基于所述二维影像的拍摄位姿对所述三维目标影像进行投影，得到对应的第二投影影像；
    基于至少一张所述第二投影影像和所述至少一张二维目标影像确定第二相似度；以及
    基于所述第二相似度对所述配准三维影像进行位姿调整，以优化所述第一转换矩阵，得到所述目标转换矩阵。
  12. 一种配准系统，包括影像获取模块、位姿变换模块、目标影像确定模块和目标矩阵确定模块；
    所述影像获取模块用于获取目标对象在手术前拍摄的三维影像和手术中拍摄的至少一张二维影像；
    所述位姿变换模块用于基于所述至少一张二维影像，对所述三维影像进行位姿变换以得到配准三维影像，以及所述三维影像对应的三维影像坐标系和所述手术对应的手术空间坐标系之间的第一转换矩阵；
    所述目标影像确定模块用于在所述至少一张二维影像中确定所述目标对象的目标部位对应的二维目标影像，在所述配准三维影像中确定所述目标部位对应的三维目标影像；以及
    所述目标矩阵确定模块用于基于所述三维目标影像和每张二维影像中的所述二维目标影像，对所述配准三维影像进行位姿变换，以优化所述第一转换矩阵，得到目标转换矩阵。
  13. 如权利要求12所述的系统，所述位姿变换模块用于：
    基于预设倍数对所述三维影像和所述至少一张二维影像进行下采样，得到下采样三维影像和至少一张下采样二维影像；
    基于预设步长调整所述下采样三维影像的位姿，得到调整后的下采样三维影像；
    对所述每一张二维影像，基于所述二维影像的拍摄位姿对所述调整后的下采样三维影像进行投影，得到对应的第一投影影像；
    响应于至少一张所述第一投影影像和所述至少一张下采样二维影像满足第一预设条件，则根据所述预设倍数对调整后的下采样三维影像进行上采样，得到所述配准三维影像；以及
    基于从所述三维影像到所述配准三维影像的位姿变换过程确定所述第一转换矩阵。
  14. 如权利要求13所述的系统，所述位姿变换模块还用于：
    基于至少一张所述第一投影影像和所述至少一张下采样二维影像确定第一相似度；以及
    响应于所述第一相似度大于相似度阈值，确定所述至少一张第一投影影像和所述至少一张下采样二维影像满足第一预设条件。
  15. 如权利要求13所述的系统，所述位姿变换模块还用于：
    响应于至少一张所述第一投影影像和所述至少一张下采样二维影像不满足第一预设条件，则按照所述预设步长调整所述调整后的下采样三维影像的位姿，以重复上述得到与每一张二维影像对应的第一投影影像的过程，直至至少一张所述第一投影影像和所述至少一张下采样二维影像满足所述第一预设条件。
  16. 如权利要求13所述的系统，所述位姿变换模块用于：
    响应于至少一张所述第一投影影像和所述至少一张下采样二维影像满足所述第一预设条件，且所述预设倍数不满足所述第二预设条件，则根据所述预设倍数对所述调整后的下采样三维影像进行上采样，以更新所述三维影像；
    降低所述预设倍数，基于降低后的所述预设倍数对所述更新后的三维影像和所述至少一张二维影像进行下采样，以重复上述得到与每一张二维影像对应的第一投影影像的过程，直至至少一张所述第一投影影像和所述至少一张下采样二维影像满足所述第一预设条件，且所述预设倍数满足所述第二预设条件。
  17. 如权利要求16所述的系统，所述位姿变换模块还用于：
    响应于所述预设倍数等于阈值，确定至少一张所述第一投影影像和所述至少一张下采样二维影像满足所述第二预设条件。
  18. 如权利要求13所述的系统，所述位姿变换模块用于：
    获取二维成像设备坐标系和所述手术空间坐标系之间的第二转换矩阵；
    基于从所述三维影像到所述配准三维影像的位姿变换过程，得到所述三维影像坐标系和所述二维成像设备坐标系之间的第三转换矩阵；以及
    基于所述第二转换矩阵和所述第三转换矩阵得到所述第一转换矩阵。
  19. 如权利要求12所述的系统，所述目标影像确定模块用于：
    在所述配准三维影像中确定对应所述目标部位的所述三维目标影像；
    对所述至少一张二维影像中的每一张，
    获取所述二维影像对应的二维影像坐标系和所述手术空间坐标系之间的第四转换矩阵；以及
    基于所述三维目标影像、所述第一转换矩阵和所述第四转换矩阵，在所述二维影像中确定所述二维目标影像。
  20. 如权利要求19所述的系统，所述目标影像确定模块用于：
    基于所述三维目标影像，确定所述目标部位的代表点在所述三维影像坐标系中的三维坐标和所述目标部位的尺寸参数；
    基于所述三维坐标、所述第一转换矩阵和所述第四转换矩阵确定所述代表点在所述二维影像坐标系中的二维坐标；以及
    基于所述二维坐标和所述尺寸参数，在所述二维影像中确定所述二维目标影像。
  21. 如权利要求19所述的系统，所述目标影像确定模块用于：
    获取二维成像设备坐标系和所述手术空间坐标系之间的第二转换矩阵；
    获取所述二维成像设备坐标系和所述二维影像坐标系之间的第五转换矩阵；以及
    基于所述第二转换矩阵和所述第五转换矩阵得到所述第四转换矩阵。
  22. 如权利要求12所述的系统，所述目标矩阵确定模块用于：
    对所述每一张二维影像，基于所述二维影像的拍摄位姿对所述三维目标影像进行投影，得到对应的第二投影影像；
    基于至少一张所述第二投影影像和所述至少一张二维目标影像确定第二相似度；以及
    基于所述第二相似度对所述配准三维影像进行位姿调整，以优化所述第一转换矩阵，得到所述目标转换矩阵。
  23. 一种计算机可读存储介质，所述存储介质存储计算机指令，当计算机读取存储介质中的计算机指令后，计算机执行如权利要求1～11任一项所述的方法。
PCT/CN2022/108552 2021-07-30 2022-07-28 一种配准方法和系统 Ceased WO2023006021A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP22848626.2A EP4365838B1 (en) 2021-07-30 2022-07-28 Registration method and system
US18/427,758 US20240221190A1 (en) 2021-07-30 2024-01-30 Methods and systems for registration

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202110875674.6 2021-07-30
CN202110875674.6A CN113570648B (zh) 2021-07-30 2021-07-30 多骨骼影像配准方法、电子装置以及医学导航系统
CN202210147197.6A CN114529594B (zh) 2022-02-17 2022-02-17 配准方法、系统、计算机设备和存储介质
CN202210147197.6 2022-02-17

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/427,758 Continuation US20240221190A1 (en) 2021-07-30 2024-01-30 Methods and systems for registration

Publications (1)

Publication Number Publication Date
WO2023006021A1 true WO2023006021A1 (zh) 2023-02-02

Family

ID=85087497

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/108552 Ceased WO2023006021A1 (zh) 2021-07-30 2022-07-28 一种配准方法和系统

Country Status (3)

Country Link
US (1) US20240221190A1 (zh)
EP (1) EP4365838B1 (zh)
WO (1) WO2023006021A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116128936A (zh) * 2023-02-15 2023-05-16 北京纳通医用机器人科技有限公司 配准方法、装置、设备及存储介质
CN116630206A (zh) * 2023-07-20 2023-08-22 杭州安劼医学科技有限公司 快速配准的定位方法及系统
CN117017487A (zh) * 2023-10-09 2023-11-10 杭州键嘉医疗科技股份有限公司 一种脊柱配准方法、装置、设备及存储介质
WO2025046505A1 (en) * 2023-09-01 2025-03-06 Medtronic Navigation, Inc. Systems and methods for patient registration using 2d image planes

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110122226A1 (en) * 2009-03-16 2011-05-26 Siemens Corporation System and method for robust 2d-3d image registration
US20140334709A1 (en) * 2013-05-07 2014-11-13 The Johns Hopkins University 3d-2d image registration for medical imaging
CN109325971A (zh) * 2018-08-02 2019-02-12 沈阳东软医疗系统有限公司 图像配准方法及装置
CN111429491A (zh) * 2020-03-11 2020-07-17 上海嘉奥信息科技发展有限公司 脊柱术前三维影像与术中二维影像配准的方法与系统
CN112233155A (zh) * 2020-12-07 2021-01-15 南京佗道医疗科技有限公司 一种2d-3d影像配准算法
CN112308765A (zh) * 2020-10-13 2021-02-02 杭州三坛医疗科技有限公司 投影参数的确定方法及装置
CN113570648A (zh) * 2021-07-30 2021-10-29 武汉联影智融医疗科技有限公司 多骨骼影像配准方法、电子装置以及医学导航系统
CN114529594A (zh) * 2022-02-17 2022-05-24 武汉联影智融医疗科技有限公司 配准方法、系统、计算机设备和存储介质

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7570791B2 (en) * 2003-04-25 2009-08-04 Medtronic Navigation, Inc. Method and apparatus for performing 2D to 3D registration
KR101753312B1 (ko) * 2010-09-17 2017-07-03 삼성전자주식회사 뎁스 영상 생성 장치 및 방법
US9524552B2 (en) * 2011-08-03 2016-12-20 The Regents Of The University Of California 2D/3D registration of a digital mouse atlas with X-ray projection images and optical camera photos
GB201506842D0 (en) * 2015-04-22 2015-06-03 Ucl Business Plc And Schooling Steven Locally rigid vessel based registration for laparoscopic liver surgery
US10262424B2 (en) * 2015-12-18 2019-04-16 The Johns Hopkins University Method for deformable 3D-2D registration using multiple locally rigid registrations
US10102640B2 (en) * 2016-11-29 2018-10-16 Optinav Sp. Z O.O. Registering three-dimensional image data of an imaged object with a set of two-dimensional projection images of the object
GB201805299D0 (en) * 2018-03-29 2018-05-16 Cydar Ltd Deformation correction

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110122226A1 (en) * 2009-03-16 2011-05-26 Siemens Corporation System and method for robust 2d-3d image registration
US20140334709A1 (en) * 2013-05-07 2014-11-13 The Johns Hopkins University 3d-2d image registration for medical imaging
CN109325971A (zh) * 2018-08-02 2019-02-12 沈阳东软医疗系统有限公司 图像配准方法及装置
CN111429491A (zh) * 2020-03-11 2020-07-17 上海嘉奥信息科技发展有限公司 脊柱术前三维影像与术中二维影像配准的方法与系统
CN112308765A (zh) * 2020-10-13 2021-02-02 杭州三坛医疗科技有限公司 投影参数的确定方法及装置
CN112233155A (zh) * 2020-12-07 2021-01-15 南京佗道医疗科技有限公司 一种2d-3d影像配准算法
CN113570648A (zh) * 2021-07-30 2021-10-29 武汉联影智融医疗科技有限公司 多骨骼影像配准方法、电子装置以及医学导航系统
CN114529594A (zh) * 2022-02-17 2022-05-24 武汉联影智融医疗科技有限公司 配准方法、系统、计算机设备和存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4365838A4 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116128936A (zh) * 2023-02-15 2023-05-16 北京纳通医用机器人科技有限公司 配准方法、装置、设备及存储介质
CN116630206A (zh) * 2023-07-20 2023-08-22 杭州安劼医学科技有限公司 快速配准的定位方法及系统
CN116630206B (zh) * 2023-07-20 2023-10-03 杭州安劼医学科技有限公司 快速配准的定位方法及系统
WO2025046505A1 (en) * 2023-09-01 2025-03-06 Medtronic Navigation, Inc. Systems and methods for patient registration using 2d image planes
CN117017487A (zh) * 2023-10-09 2023-11-10 杭州键嘉医疗科技股份有限公司 一种脊柱配准方法、装置、设备及存储介质
CN117017487B (zh) * 2023-10-09 2024-01-05 杭州键嘉医疗科技股份有限公司 一种脊柱配准方法、装置、设备及存储介质

Also Published As

Publication number Publication date
EP4365838B1 (en) 2026-04-22
US20240221190A1 (en) 2024-07-04
EP4365838A1 (en) 2024-05-08
EP4365838A4 (en) 2024-10-02

Similar Documents

Publication Publication Date Title
TWI836492B (zh) 註冊二維影像資料組與感興趣部位的三維影像資料組的方法及導航系統
US20240185509A1 (en) 3d reconstruction of anatomical images
WO2023006021A1 (zh) 一种配准方法和系统
US11628012B2 (en) Patient positioning using a skeleton model
CN115526929A (zh) 基于图像的配准方法和装置
CN109152566B (zh) 校正超声融合成像系统中的探头引起的变形
US11669982B2 (en) Soft tissue stereo-tracking
CN113570648B (zh) 多骨骼影像配准方法、电子装置以及医学导航系统
CN111179373B (zh) 医学图像去骨模型构建方法与骨信息去除方法
WO2023089566A1 (en) Method and navigation system for registering two-dimensional image data set with three-dimensional image data set of body of interest
US20250017667A1 (en) Determining a consensus plane for imaging a medical device
Chen et al. An optimization-based baseline for rigid 2d/3d registration applied to spine surgical navigation using cma-es
CN116549108A (zh) 基于影像的目标点位置确定方法、装置、设备及存储介质
WO2024250611A1 (zh) 图像配准方法及装置、电子设备和存储介质
CN118557286A (zh) 一种膝关节置换导航方法、装置、电子设备及存储介质
JP2024541438A (ja) 二次元画像データセットと関心部位の三次元画像データセットを登録する方法及びナビゲーションシステム
WO2025026523A1 (en) Self-validating elastic fusion
CN119722754A (zh) 图像配准方法、装置及基于图像配准的骨科手术导航系统
CN119423790A (zh) 医学成像方法、装置和医学成像设备
CN120266165A (zh) 用于使用先验患者数据进行体积重建的系统和方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22848626

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2022848626

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2022848626

Country of ref document: EP

Effective date: 20240129

NENP Non-entry into the national phase

Ref country code: DE

WWG Wipo information: grant in national office

Ref document number: 2022848626

Country of ref document: EP