WO2024009132A1 - Amélioration de procédures d'échographie endoscopique avec intelligence de poursuite et visuelle - Google Patents

Amélioration de procédures d'échographie endoscopique avec intelligence de poursuite et visuelle Download PDF

Info

Publication number
WO2024009132A1
WO2024009132A1 PCT/IB2022/056270 IB2022056270W WO2024009132A1 WO 2024009132 A1 WO2024009132 A1 WO 2024009132A1 IB 2022056270 W IB2022056270 W IB 2022056270W WO 2024009132 A1 WO2024009132 A1 WO 2024009132A1
Authority
WO
WIPO (PCT)
Prior art keywords
probe
orientation
model
visual output
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/IB2022/056270
Other languages
English (en)
Inventor
David Menashe
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Gi Compass Ltd
Original Assignee
Gi Compass Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Gi Compass Ltd filed Critical Gi Compass Ltd
Priority to PCT/IB2022/056270 priority Critical patent/WO2024009132A1/fr
Publication of WO2024009132A1 publication Critical patent/WO2024009132A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/12—Diagnosis using ultrasonic, sonic or infrasonic waves in body cavities or body tracts, e.g. by using catheters
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/42—Details of probe positioning or probe attachment to the patient
    • A61B8/4245—Details of probe positioning or probe attachment to the patient involving determining the position of the probe, e.g. with respect to an external reference frame or to the patient
    • A61B8/4263—Details of probe positioning or probe attachment to the patient involving determining the position of the probe, e.g. with respect to an external reference frame or to the patient using sensors not mounted on the probe, e.g. mounted on an external reference frame
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • A61B2034/101—Computer-aided simulation of surgical operations
    • A61B2034/105—Modelling of the patient, e.g. for ligaments or bones
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2048—Tracking techniques using an accelerometer or inertia sensor
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2051—Electromagnetic tracking systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/48—Diagnostic techniques
    • A61B8/483—Diagnostic techniques involving the acquisition of a 3D volume of data

Definitions

  • This disclosure relates to medical imaging.
  • the disclosure relates to a novel system and method for improving endoscopic ultrasound procedures by adding a tracking element to the ultrasound probe and using the resulting information to provide the physician with real-time analysis based on computer vision and artificial intelligence. This facilitates more widespread adoption of endoscopic ultrasound procedures, allows for a faster and simpler procedure, and improves the overall success rate.
  • Endoscopic Ultrasound is used extensively in the diagnosis of pathologies related to the digestive system, such as lesions in the pancreas, biliary system, liver and gastrointestinal (GI) tract lining.
  • a typical EUS system consists of an endoscope equipped with an Ultrasound (US) probe at the distal end of the scope.
  • US Ultrasound
  • the US probe When inserted through the mouth of the patient into the GI tract, particularly the stomach or duodenum, the US probe can be used to provide sharp US images of organs adjacent to the GI tract, thus allowing the physician to identify and diagnose pathologies related to these organs.
  • the US images can also be used to guide biopsies of suspicious lesions.
  • biopsies are known as fine needle aspiration (FNA) and Fine needle biopsies (FNB)
  • EUS is a powerful diagnostic tool, it requires that the user has many years of experience in order to correctly obtain and interpret the US images and derive relevant diagnostic information. Even then, the procedure can require a significant amount of time (an hour or more) and can often result in inaccurate diagnosis. This is mainly due to the fact that it is difficult to know the exact location and/or orientation of the US probe within the GI tract, information which is crucial for correctly interpreting the resulting US image (a task which is often challenging even when this information is available).
  • CT computer tomography
  • MRI magnetic resonance imaging
  • FIG 1 shows schematically a typical known EUS system numbered 100.
  • EUS system 100 comprises an US probe 102 which is located at the distal end of an endoscope (not shown).
  • the endoscope typically includes a video camera and one or more working channels through which tools, needles, and other diagnostic probes may be inserted into the subject's GI tract.
  • US probe 102 is connected to an US control and processing unit 104 which controls the US signals emitted by the probe and processes the return signals received by the probe in order to produce a stream of US images 106, which are then displayed on a display unit 108.
  • US image stream 106 is updated at a video frame rate in the range of 25-50 frames per second (fps) such that a smooth US video is seen by the user on display unit 108.
  • the US video may also be stored on a permanent memory system such as a hard disk drive (not shown) for future reference and/or processing.
  • the user guides the endoscope through the GI tract of the subject, with the aid of the video camera embedded in the endoscope, to one of a number of positions, also known as “stations”, which are typically used for US imaging of the digestive system.
  • stations also known as “stations”
  • the user switches on US probe 102 and views the US image stream 106 on display unit 108.
  • an experienced user can interpret the US image and identify various organs and vessels shown in the image.
  • the user may also slightly manipulate the endoscope (e.g. by inserting it further or extracting it a little, and by changing the probe direction and/or orientation using the endoscope controls) while viewing and interpreting the real-time US image stream.
  • Performing a EUS procedure as described above requires considerable skill and extensive experience in order to correctly guide and position the endoscope and to interpret the US image. While the video camera allows the user to approximately position the EUS probe at the desired location, the precise position and orientation of the probe can only be inferred by correctly interpreting the resulting US image. Based on this interpretation, the user may then fine-tune the position and orientation of the probe to obtain US images with relevant diagnostic information. This process may require a number of iterations before obtaining the desired US images, and in some cases may fail altogether to provide the desired images. In other cases, misinterpretation of the US image may result in erroneous diagnosis. Furthermore, since the US image is 2D with a relatively restricted field of view, the user may need to mentally reconstruct a 3D image of the region of interest based on the stream of 2D US images. Such mental reconstruction also requires significant experience and skill.
  • the feed-back provided to the user may include, but is not limited to, one or more of the following elements: • Combining all the 2D US images collected during the procedure into a common 3D volume, and allowing the user to view the volume and virtually navigate through it without physically moving the US probe;
  • an enhanced endoscopic ultrasound (EUS) system comprising: a US probe located at a distal end of the endoscope; a US control and processing unit operatively coupled with the US probe and configured to output an US image stream; a tracking sensor with a known position and orientation relative to the US probe; a tracking control and processing unit operatively coupled with the tracking sensor and configured to calculate and output the position and orientation of the US probe at any given moment; a visual intelligence unit configured to accept as input and process the US image stream and the position and orientation of the US probe at any given moment into a visual output; and a display unit configured to display the visual output.
  • EUS enhanced endoscopic ultrasound
  • the visual intelligence unit is further configured to accept as input a preprocedure model.
  • the visual output includes a current position and orientation of the US probe shown in relation to the pre-procedure model.
  • the visual output includes labels and/or marks of borders and/or one or more key points of selected anatomical elements and/or pathologies overlaid on a US image of the US image stream, and derived from the pre-procedure model.
  • the visual output includes instructions to a user on guidance of the US probe to visualize a target pathology identified in the pre-procedure model.
  • a method for enhancing EUS procedures in a system configured to output an US image stream and a position and orientation of a US probe at any given moment, comprising: processing the US image stream and position and orientation of the US probe at any given moment into a visual output; and displaying the visual output.
  • any given moment refers to a continuously updated output, i.e. a corresponding position and orientation of the US probe is known for each image in the image stream.
  • the tracking sensor is an electromagnetic (EM) tracking sensor.
  • EM electromagnetic
  • the visual output includes a current US probe position and orientation shown in relation to a plurality of previous positions and orientations within a common 3D volume.
  • the visual output includes labels and/or marks of borders and/or one or more key points of selected anatomical elements and/or pathologies overlaid on a US image of the US image stream.
  • the visual output includes a 3D model of one or more selected anatomical elements and/or pathologies. In some examples, the visual output includes a current position and orientation of the US probe shown in relation to the 3D model.
  • the processing includes processing the US image stream and position and orientation of the US probe at any given moment together with a pre-procedure model into the visual output.
  • the visual output includes a current position and orientation of the US probe shown in relation to the pre-procedure model.
  • the visual output includes labels and/or marks of borders and/or one or more key points of selected anatomical elements and/or pathologies overlaid on a US image of the US image stream, and derived from the pre-procedure model.
  • the visual output includes instructions to a user on guidance of the US probe to visualize a target pathology identified in the pre-procedure model.
  • FIG. 1 shows a schematic representation of a typical known EUS system
  • FIG. 2 shows a schematic representation of one embodiment of the proposed system for enhancing EUS procedures
  • FIG. 3 shows a schematic representation of one embodiment of an US probe with tracking capabilities
  • FIG. 4A shows one embodiment of a flow chart of an algorithm implemented in the visual intelligence unit of the system
  • FIG. 4B shows an example view of a 3D common volume
  • FIG. 5 shows an example of detection and identification of liver and vessels in a 2D US frame
  • FIG. 6 shows another example of detection and identification of liver and vessels in a 2D US frame
  • FIG. 7 shows an example of a 3D model of the biliary system near the papila, created from segmentation of a sequence of US images
  • FIG. 8 shows an example segmentation of a lesion in an US image, and a resulting 3D model created from segmentation of a sequence of US images
  • FIG. 9 shows a schematic representation of another embodiment of the proposed system for enhancing EUS procedures.
  • FIG. 10 shows one embodiment of a flowchart for creating a pre-procedure 3D model
  • FIG. 11 shows an example of an abdominal CT slice with segmentation of selected organs
  • FIG. 12 shows an example of a pre-procedure model built from a segmented CT scan
  • FIG. 13 shows another embodiment of a flow chart of an algorithm implemented in the visual intelligence unit of the system
  • FIG. 14A shows an example view of the US probe position and orientation in relation to a pre-procedure model
  • FIG. 14B shows an example view of a combined 3D model including a 3D common volume and a part of a pre procedure model
  • FIG. 15 shows an example of a segmentation of the pancreas in an US image created with the aid of registration of the US probe to a pre-procedure model
  • FIG. 16 shows an example of a segmentation of a target lesion in an US image created with the aid of registration of the US probe to a pre-procedure model
  • FIG. 17 shows a flowchart of an embodiment of a method for guiding the user to position and orient the US probe so as to optimally image the target lesion
  • FIG. 18 shows a schematic representation of another embodiment of the proposed system for enhancing EUS procedures
  • FIG. 19 shows exemplary video images that can be used to help pinpoint the position of the US probe during the EUS procedure.
  • FIG. 2 shows a schematic representation 200 of a system for enhancing EUS procedures and addressing the problems listed above, as disclosed herein.
  • the system includes a tracking sensor 202 connected to a tracking control and processing unit 210.
  • Tracking sensor 202 is configured to have a known position and orientation relative to US probe 102, such that measuring the position and orientation of tracking sensor 202 allows the calculation of the position and orientation of US probe 102.
  • Tracking control and processing unit 210 is configured to receive inputs from tracking sensor 202, and to compute and output an instant (at each moment) position and orientation 212 of US probe 102.
  • tracking processing unit 210 is operationally coupled to additional elements (not shown) in order to compute output 212 US image stream 106 together with US probe position and orientation 212 are provided as input to visual intelligence unit processing unit 214.
  • Visual intelligence unit 214 uses computer vision (CV) and artificial intelligence (Al) capabilities to process the input and to provide real time visual feed-back to the user performing the EUS procedure, which is then displayed on display unit 108.
  • visual intelligence unit 214 may receive additional input (not shown) from the user designed to configure the content of the real time visual feed-back.
  • US control and processing unit 104, tracking control and processing unit 210, and visual intelligence unit 214 are separate physical units. In other embodiments, two or all of these units may be combined into a single physical unit. It should be apparent to those skilled in the art that different physical configurations of units 104, 210 and 212 are possible without loss of generality of the disclosure.
  • the tracking technology used in the system described in FIG. 2 may be based on electro-magnetic (EM) tracking technology as known in the art.
  • FIG. 3 shows a possible implementation of this technology for the current disclosure.
  • a field generator (FG) 302 generates a spatially varying EM field 304 which is sensed by tracking sensor 202.
  • FG 302 is positioned such that varying EM field 304 covers a volume that includes the patient's digestive system in which US probe 102 and tracking sensor 202 are inserted for the purpose of carrying out the EUS procedure.
  • FG 302 and tracking sensor 202 are connected to tracking control and processing unit 210, which is designed to calculate the position and orientation of tracking sensor 202 and thus the US probe position and orientation 212 at any given moment.
  • tracking sensor 202 is connected to tracking control and processing unit 210 via wires that run through the endoscope.
  • one or more additional sensors 306 may be connected to the patient's body and used as reference positions and/or to compensate for patient movement during the EUS procedure.
  • tracking sensor 202 may be a six degree of freedom (6DOF) sensor such that complete translational and rotational information for US probe 102 can be computed by tracking control and processing unit 210 at any given moment.
  • tracking sensor 202 may be permanently embedded at the distal end of the endoscope in close and fixed proximity to US probe 102. In other embodiments, tracking sensor 202 is not embedded in the endoscope, but instead inserted through the working channel of the endoscope to a position near US probe 102, and locked in this position until it is no longer needed.
  • a calibration step is implemented following insertion and locking of tracking sensor 202 to determine the position and orientation of the sensor relative to US probe 102.
  • the calibration step may involve placing the endoscope with US probe 102 and tracking sensor 202 in one or more predetermined positions and orientations, thus allowing tracking control and processing unit 210 to calculate the position and orientation of tracking sensor 202 relative to US probe 102
  • the calibration step may involve inserting the endoscope into a calibration jig designed to produce a predetermined pattern in an US image, and analyzing the resulting US image to identify the pattern and thus determine the relation between US probe 102 and tracking sensor 202
  • tracking sensor 102 is disposable and is disposed of once extracted from the working channel of the endoscope
  • tracking technologies may be used in conjunction with the system shown in FIG. 2.
  • inertial measurement units known in the art and including magnetometers and/or gyroscopes and/or accelerometers may be included in tracking sensor 202.
  • a fiber-optics based sensor may be embedded in the endoscope such that the position and orientation of US probe 102 can be determined, as for example described in US Patent application US20110172519A1. It should be apparent to those skilled in the art that different tracking technologies may be used without loss of generality of the disclosure.
  • FIG. 4A shows a flow chart 400 of a method implemented on visual intelligence unit 214.
  • patient data is received.
  • such data is automatically extracted from a centralized database.
  • the patient data is manually entered by the user via a keyboard or other input device.
  • a combination of automatic extraction and manual input is utilized.
  • the patient data may include only informative data such as identity number, name, age and gender.
  • the data may include additional clinical data such as physical characteristics (height, weight, body mass index) and/or relevant medical history.
  • the patient data may include a pre-procedure scan such as a CT or MRI.
  • the patient data may include a pre-procedure scan enhanced with additional information, such as segmentation of organs and vessels and pathologies.
  • a new frame is acquired from US image stream 106, together with the corresponding position and orientation 212 of the US probe.
  • the US transducer in US probe 202 may be disabled so that a blank or null frame is acquired from US image stream 106 and only the position and orientation 212 of the US probe contains meaningful information. This may occur for example when the US probe is being moved from one EUS station to the other.
  • a frame from the video camera of the endoscope may be acquired.
  • a frame from the video camera is acquired in addition to the US frame, regardless of whether or not the US transducer is disabled.
  • the position and orientation 212 of the US probe is used to register the 2D US frame to a common 3D volume containing all previously acquired 2D US frames.
  • the settings of US probe 102 are read from US control and processing unit 104 in order to determine the precise shape of the US fan representing the 2D cross section covered by the US frame.
  • the settings of US probe 102 are determined by identifying markings and/or text produced by US control and processing unit 104 on each of the US images in US image stream 106.
  • step 406 includes compensation of patient movement during the procedure by also tracking the position of one or more additional sensors, such as for example 306 in the case of EM tracking, attached to the patient's body, and using this additional information to derive the correct location in the common 3D volume independent of the patient movement.
  • additional sensors such as for example 306 in the case of EM tracking
  • step 406 includes compensation of patient movement during the procedure by also tracking the position of one or more additional sensors, such as for example 306 in the case of EM tracking, attached to the patient's body, and using this additional information to derive the correct location in the common 3D volume independent of the patient movement.
  • additional sensors such as for example 306 in the case of EM tracking
  • the 3D model of the esophagus may be enhanced by measuring the radius of the esophagus using the endoscopic video camera stream in conjunction with structure from motion technology known to the art.
  • a view of the 3D common volume is created and displayed (step 410 of FIG. 4A) to assist the user in the EUS examination.
  • 3D meshes of different elements of the 3D common volume are created using, for example, the known marching cubes algorithm.
  • the different meshes are color coded to enhance the visualization.
  • an additional smoothing step such as the known laplacian smoothing algorithm, is applied to the 3D meshes to improve their visual quality.
  • the 3D common volume is viewed as a point cloud rather than a series of meshes.
  • different points in the point cloud can be color coded to enhance the visualization.
  • a combination of meshes and point clouds may be used to view the 3D common volume.
  • Allowing the user to view the 3D common volume from different angles, positions and zoom levels, using standard 3D visualization techniques known to the art, provides valuable feedback during the examination. For example, when screening a patient for various pathologies, such as cancer of the pancreas, viewing the entire area covered by the US scan during the examination can highlight whether there are gaps representing areas of the anatomy not scanned, and which should be scanned to complete the screening successfully.
  • showing how different 2D US frames taken at different stations during the examinations relate to each other in the 3D common volume can provide a more thorough 3D understanding of pathologies, such as a lesion, identified during the examination, and visible in the different 2D US frames.
  • the current US probe position and orientation are shown on the 3D common volume.
  • means are provided for the user to navigate within the 3D common volume in order to view previously acquired 2D US frames.
  • the user can easily and intuitively browse and recall the preceding steps of the EUS examination without having to physically reposition the US probe and reacquire a desired US image.
  • FIG. 4B shows an example mesh view 450 of a 3D common volume including a path 452 of the US probe and an area 454 covered by the US scan, i.e. all the US fans corresponding to the 2D US frames collected during the EUS examination.
  • the EUS examination included three stations, the cardia, the duodenum bulb, and the papilla.
  • step 408 detection and identification of selected organs and vessels is performed in the US image frame, with the aid of current and previous positions and orientations 212 of the US probe (also referred to below as the tracking data).
  • detection may be performed using a classical and/or deep learning algorithm known to the art, such as the known UNET segmentation model, followed by use of a second algorithm which identifies and labels the various anatomical structures detected by the detection algorithm.
  • this identification is achieved using pattern based rules and prior knowledge of the digestive system and surrounding vessels and organs. For example, if the tracking data indicates that the US probe has passed from the esophagus into the cardia, and a vessel of diameter in the range of 1.5-3 cm is seen in the US frame, then this vessel can be assumed to be the abdominal aorta.
  • a dedicated deep learning model is trained to perform the identification.
  • this model is a long short term memory (LSTM) neural network model which accepts as input the output of the detection algorithm (e.g. segmentation) for a sequence of US images in US image stream 106, together with the corresponding tracking data, and is trained to identify and label each element detected by the detection algorithm.
  • LSTM long short term memory
  • a 3D model of selected vessels and/or organs is constructed using the common volume described in step 406, together with the output of the detection algorithm for a sequence of US frames in US image stream 106.
  • This 3D model is then registered, using standard 3D registration methods known to the art, to a pre-procedure model of the digestive system and surrounding vessels and organs (see FIG. 9 below and related text), and the output of the registration is used to perform the identification, by reference to the pre-procedure model.
  • detection and identification of the anatomical structures is performed by a single classical and/or deep learning model.
  • the tracking data is first used to identify the EUS station at which the US probe is currently located and oriented.
  • a specifically trained object detection model such as the known YOLO model
  • semantic segmentation model such as UNET
  • the tracking data indicates that the US probe has passed from the stomach to the duodenum bulb
  • a model specifically trained to identify the superior mesenteric artery, the common bile duct, and the pancreatic duct in this US view is selected and provided with the current US frame as input.
  • the output of this model is the location of each of these anatomical elements, optionally together with a confidence level associated with each element.
  • the model used for simultaneous detection and identification is a variation of the LSTM model described above, where the model is provided with the raw US image stream 106 rather than the output of a separate detection model.
  • detection and/or identification may be performed manually or semi-manually by the physician performing the EUS procedure. For example, based on the tracking data the EUS station is automatically identified, and the physician is then presented with a number of typical US views visible from this station. After the physician selects the view which most closely matches the current US frame, a specific model pre-trained for this US view is then applied to the US frame in order to detect and identify the various anatomical elements visible in this view
  • detection takes the form of full segmentation of the organs and/or vessel. In other embodiments, detection takes the form of locating the center or other specific key points of organs and/or vessels. In yet other embodiments, detection takes the form of a bounding box surrounding the organs and/or vessels. In some embodiments, detection is performed on a single 2D US frame. In other embodiments, detection is performed on a sequence of US frames from US image stream 106 after registration to a 3D common volume as described in step 406.
  • FIG. 5 shows an example of semantic segmentation followed by identification.
  • the UNET neural network model is trained to identify vessels and liver tissue in any 2D US Image (i.e. regardless of the US view).
  • Image 502 shows an example 2D US frame
  • image 504 shows the corresponding frame with vessels (dark gray) and liver (light gray) segmented.
  • This segmentation is performed for a sequence of 2D frames, and then using the registration of the frames to a common 3D volume (step 406 in FIG. 4), a 3D model of selected vessels and/or liver tissue is constructed, as shown in image 506.
  • the tracking data shows that the US probe is located in the duodenum bulb, the two selected vessels are identified and labeled, as shown in image 508.
  • FIG. 6 shows an example of simultaneous detection and identification in a single algorithm.
  • analysis of the tracking data indicates that the US probe is located in the duodenum bulb, while analysis of the orientation of the US probe indicates that the relevant US view should show the liver, portal vein, and common bile duct.
  • a customized deep learning model specifically designed and trained to analyze this view is selected, and given image 502 as input.
  • the model is a 2D UNET Model trained to locate the center (white circles) of three anatomical features (liver, portal vein and common bile duct) seen in the US view in image 502.
  • the resulting output of this model is shown in image 604.
  • the identification and labeling of these anatomical features follows automatically since the deep learning model is specifically designed to analyze the shown view.
  • the combined output of steps 406 and 408 is used to create 3D models designed to provide specific and targeted feedback to the physician.
  • These models are constructed by using the registration of segmented US images (step 408) to a common 3D volume (step 406), common interpolation routines to interpolate regions between two successive images, followed by applications of the known marching cubes algorithm to create a 3D surface.
  • a surface smoothing algorithm such as the Laplacian smoothing algorithm may be applied to create a smoother and more realistic model.
  • a 3D model may combine contributions from US images collected at multiple EUS stations, with each set of images contributing to a different part of the model.
  • the 3D model may be combined with a view of the common volume (e.g. FIG. 4B) so that the physician can see which areas to scan in order to complete and/or enhance the model.
  • the 3D model can be combined with a view of the current US probe position and/or current US fan in order to better understand the relation between the 3D model and the current 2D US frame.
  • the user is provided with the means to view the 3D model from different angles, positions and zoom levels, using standard 3D visualization techniques known to the art.
  • FIG. 7 shows an example view 700 of a 3D model of the biliary system near the papila, including a papila 702, a common bile duct 704, and a pancreatic duct 706.
  • the model was created by automatic segmentation and identification of the common bile duct and pancreatic duct in the region of the papilla in a sequence of 2D US images taken from EUS stations just below the papilla and from the duodenal bulb.
  • Such a 3D model is especially useful during an endoscopic retrograde cholangiopancreatography procedure (ERCP), where navigation errors within the biliary system can lead to complications such as pancreatitis.
  • ERCP endoscopic retrograde cholangiopancreatography procedure
  • FIG. 8 shows an example 800 of a segmentation of a lesion in the head of the pancreas.
  • Image 802 shows a manual segmentation of the lesion, performed by the user, in one example 2D US image. This manual segmentation is performed on a number of US images representing different cross sections of the lesion. Then, using the 3D common volume, in a manner similar to that described above in relation to FIG. 7, a 3D model of the lesion is then created. Once the 3D model is created, it is displayed together with the position and orientation of the US fan at any given moment, as shown in image 804. This allows the physician performing the EUS procedure to better characterize the lesion under study, and also to better prepare for a biopsy in the case of FNA or FNB.
  • FIG. 8 shows an example 800 of a segmentation of a lesion in the head of the pancreas.
  • Image 802 shows a manual segmentation of the lesion, performed by the user, in one example 2D US image. This manual segmentation is performed on a number
  • step 410 updates display unit 108 with information derived from the previous steps that provides real time visual feed-back to the user performing the EUS procedure.
  • this visual feedback includes an annotated version of the 2D US frame, as for example shown in images 504, 508, and 604.
  • this visual feedback includes a 3D anatomical model derived from multiple US frames, as shown in images 506, and 700.
  • this visual feedback includes a 3D model derived from multiple US frames together with the location of the US fan in the current frame, as for example shown in image 804.
  • the visual feedback may include a representation of the 3D common volume with the path of the US probe shown with the volume, the areas covered by all previous US images, and the location of the current US image, as for example shown in FIG. 4B. It should be apparent to one skilled in the art that any visual representation of the information derived in prior steps of the flow chart (including steps not shown in FIG. 4A) may be included in the update of display unit 108.
  • FIG. 9 shows a schematic representation 900 of another embodiment of the proposed system for enhancing EUS procedures disclosed herein.
  • a pre-procedure model 916 of the digestive system and surrounding organs and vessels is provided as input to visual intelligence unit 214 in addition to the US image stream 106 and the position and orientation of the US probe 212.
  • the pre-procedure model is a generic model adapted to the patient's characteristics, such as gender, height, and BMI.
  • the pre-procedure model is a generic model adapted to the patient's anatomy using data gathered from the tracking data.
  • the pre-procedure model is derived from segmentation of relevant anatomical structures in a pre-procedure 3D CT or MRI scan.
  • the segmentation is performed automatically.
  • the segmentation is performed manually by a radiologist or physician.
  • the segmentation is performed semi- automatically together with manual input from a radiologist or physician.
  • the segmentation of a pre-procedure scan may include segmentation of a region of interest (ROI), such as a lesion or tumor or other potential pathology identified as suspicious, and which is a target for investigation during the EUS procedure.
  • ROI region of interest
  • FIG. 10 shows a flow chart 1000 of an exemplary method embodiment used to generate a pre-procedure model 916.
  • a 3D scan such as a CT or MRI is received as input.
  • segmentation is performed for selected organs and vessels relevant to the EUS procedure.
  • the segmented organs and vessels may include, but are not limited to, the esophagus, stomach, duodenum, pancreas, spleen, liver, gallbladder, bile ducts and surrounding veins and arteries such as the abdominal aorta, the inferior vena cava (IVC), the portal venous system and mesenteric veins and arteries.
  • IVC inferior vena cava
  • a physician or radiologist may select which organs and vessels to segment based on the specific EUS examination to be performed.
  • the segmentation may be performed automatically, for example by a deep learning model such as the UNET model.
  • the segmentation may be performed manually by the radiologist or physician.
  • semi-automatic segmentation may be performed with guidance from a radiologist or physician.
  • automatic segmentation may be performed on individual 2D slices. In other embodiments, automatic segmentation may be performed in 3D on all slices simultaneously.
  • a 3D mesh is created for each segmented organ or vessel using, for example, the marching cubes algorithm.
  • a single mesh may be created for multiple organs or vessels that are connected one to the other.
  • a single mesh could be created for all the bile ducts and the gallbladder.
  • the 3D meshes created in step 1006 are smoothed using, for example, a known Laplacian smoothing algorithm.
  • individual 3D models are optionally compared to generic versions of these models with the purpose of identifying and correcting anomalies in the underlying segmentation (step 1004).
  • the generic model is first registered to the 3D model taking into account reasonable deformations (such as resizing) and using standard known 3D registration techniques. Once the two models are registered, the difference between them is analyzed and anomalies are identified. For example, if the model represents a blood vessel, then differences that depart significantly from cylindrical symmetry are marked as anomalies and corrected. In step 1012, all 3D models are combined into a single 3D model using the base coordinate system of the 3D scan received as input in step 1002.
  • the resulting 3D model is deformed to take into account a possible difference between the orientation of the patient for the 3D scan used as input in step 1002 and the orientation of the patient during the EUS procedure.
  • this deformation may be performed using a generic deformation map derived from anatomical knowledge.
  • the generic deformation map may be augmented using anatomical landmarks derived from tracking the US probe, as for example described in relation to step 406 of FIG. 4A.
  • this deformation may be performed using a deep learning model trained to accept as input a 3D model and output a customized deformation map for the input model.
  • combinations of the above methods are employed to derive the deformation map.
  • FIG. 11 demonstrates segmentation of a 2D slice of a CT scan.
  • Image 1100 shows a 2D slice of an abdominal CT scan.
  • Curve 1102 (white) denotes the borders of the pancreas in this slice.
  • Curve 1104 (black) denotes the borders of the spleen in this slice
  • FIG. 12 shows an example view 1200 of a 3D model of selected organs, vessels and ducts relevant to a typical EUS procedure, created from a segmented CT scan as described in FIG. 10.
  • the model includes a gallbladder 1202, a pancreas 1204, a spleen 1206, a common bile duct 1208, a splenic vein 1210, and a portal vein 1212.
  • FIG. 13 shows a flow chart 1300 of another method implemented on visual intelligence unit 214.
  • Step 1302 is similar to step 402 in FIG. 4A, with the specific requirement that a pre-procedure model 916 is received as part of the patient data.
  • step 1304 registration of the US probe position and orientation 212 to the pre-procedure model 916 is initialized or updated. In some embodiments, the registration is initialized by calculating the position and orientation of the US probe with respect to one or more additional sensors, as for example 306 in FIG. 3, connected to the patient's body, which themselves have been registered to the pre-procedure model.
  • the additional sensors may be connected to surface anatomy landmarks such as the umbilicus, the tip of the xiphoid, and/or the anterior superior iliac spine, which have also been identified manually or automatically in a pre-procedure CT used to construct pre-procedure model 916
  • information derived from the tracking data may be used to initialize the registration.
  • the point at which the probe moves from the esophagus to the stomach may be identified, and together with the direction of movement from the esophagus to the stomach, the registration is initialized.
  • Identification of the point between the esophagus and stomach can, for example, be achieved manually via input from the user, automatically by following the trajectory of the US probe in the esophagus, or automatically by using computer vision to analyze the video feed from the endoscope camera.
  • the user may be requested to perform an US scan of a particular anatomical element, and the resulting segmentation and 3D model of this anatomical element may be used to perform the registration to the 3D model using known standard 3D registration techniques. For example, as the user enters the cardia from the esophagus, he/she may be requested to perform a US scan of the branching point of the celiac trunk from the abdominal aorta. Once the scan is complete, the abdominal aorta and celiac trunk are segmented (either manually by the user or automatically using known segmentation methods), and the 3D model of the branching point is constructed and used to initialize the registration to the pre-procedure model. In yet other embodiments, initialization of the registration may be achieved by using a combination of the above methods.
  • the registration can be updated and fine-tuned in subsequent iterations of the algorithm, using information derived from previous iterations of the algorithms.
  • a 3D segmentation of selected organs and/or vessels derived from previous US images is used to fine tune the registration using standard 3D registration techniques known to the art.
  • selected anatomical landmarks such as the confluence between the splenic vein, portal vein and superior mesenteric vein, are identified in the US images and used to update the registration.
  • additional anatomical landmarks are identified by tracking the path of the US probe, for example, the pyloric orifice connecting the stomach to the duodenum.
  • additional anatomical landmarks are identified by using computer vision to analyze the video feed from the endoscope camera.
  • a combination of one or more of the above methods is used to update the registration.
  • the registration between pre-procedure model 916 and US probe position and orientation 212 is utilized to create and display (in step 410 of FIG. 13) a view of a combined 3D model which includes pre-procedure model 916, the position of the US probe, and the US fan representing the 2D cross section covered by the current US frame in US image stream 106.
  • FIG. 14A shows an example of such a view 1400, with the position of the US probe 1402 and the US fan 1404 shown in relation to an example of pre-procedure model 916.
  • the shape and size of the US fan are determined according to the methods described in relation to step 406 in FIG. 4A.
  • the user is able to zoom, rotate and pan the combined 3D model using standard known 3D visualization techniques to create different views of the model.
  • the ability to clearly visualize in 3D the location and orientation of the US probe and the US fan relative to the patient's anatomy facilitates the interpretation and analysis of the US image, and thus enhances the overall EUS procedure.
  • the registration between pre-procedure model 916 and US probe position and orientation 212 derived in step 1304 is utilized to create and display (in step 410 of FIG. 13) a view of a combined 3D model which includes pre-procedure model 916, and the 3D common volume created in step 406.
  • a selected part of interest of pre-procedure model 916 is included in the combined 3D model.
  • the selected part of interest is a particular organ, or part thereof, which the physician would like to screen for pathologies during the EUS procedure.
  • the user is able to zoom, rotate and pan the combined 3D model using standard known 3D visualization techniques, to create different views of the model and hence to gain further insight.
  • FIG. 14B shows an example view 1450 of such a combined 3D model including the 3D common volume with path 452 of the US probe and the area covered by the US scan 454, i.e. all the 2D US frames collected during the EUS examination (as in FIG. 4B).
  • the pancreas 1452 taken from pre-procedure model 916 is also visible, allowing the user to see which parts of the pancreas 1454 have been covered by the US scan. This is particularly useful during a screening examination of Pancreas, where it is important to cover all the pancreas in order to ensure there are no suspicious pathologies.
  • the registration derived between the US probe and the pre-procedure model in step 1304 is used in step 408 for the detection and identification of organs and vessels, as for example described above in the context of FIG. 5 and FIG. 6.
  • step 1306 information found in the pre-procedure model is mapped to the 2D US image.
  • this information may include the location and/or boundaries of anatomical elements or landmarks that are part of the pre-procedure model, and which were not detected and identified in step 408.
  • this information may include the location and/or borders of a target lesion identified in the pre-procedure scan
  • FIG. 15 shows an example of a segmentation of the pancreas in a US image 1500 created with the aid of registration of the US image to a pre-procedure CT scan as described in step 1304 of FIG. 13. Since the pancreas was already segmented in the CT scan, the registration of the US image to the CT scan allows the segmentation to be automatically transferred to the US image, as shown by the overlaid white line 1502. An additional label 1504 may also be overlaid on the US image, with the resulting modified image displayed in step 410 of FIG. 13.
  • FIG. 16 shows an example of a segmentation of a target lesion in a US image 1600 created with the aid of registration of the US image to a pre-procedure CT scan as described in step 1304 of FIG. 13. Since the target lesion was already segmented in the CT scan, the registration of the US image to the CT scan allows the segmentation to be automatically transferred to the US image, as shown by the overlaid white line 1602. The resulting modified image can be displayed in step 410 of FIG. 13.
  • FIG. 17 shows an embodiment of a flowchart 1700 for guiding the user to position and orient the US probe so as to optimally image a target pathology (e.g. lesion) identified in pre-procedure model 916.
  • the optimal position and orientation of the EUS probe for imaging the target lesion is determined.
  • the best EUS station to image the lesion from is first determined using a database of lesion positions and corresponding EUS stations.
  • the user may be prompted to manually enter the best EUS station from a predetermined selection.
  • a pre-procedure model 916 may be used to locate the position on the wall of the stomach or duodenum closest to the lesion, and then the best orientation of the EUS probe at that position is determined in order to obtain an optimal crosssection of the lesion.
  • a deep learning model may be trained to accept as input a pre-procedure model 916 including the target lesion and to output the optimal position and orientation of the US probe.
  • step 1704 the current position and orientation 212 of the US probe is read.
  • step 1706 the current and desired position and orientation of the US probe are compared to determine whether the US probe position and orientation are correct. If not, step 1708 is executed where the user is instructed on how to move the US probe to reach the desired position and orientation. In some embodiments, this step may require keeping track of all previous positions of the US probe to map the 3D endoscope topology in the body, and to provide the user with exact instruction on how to manipulate the endoscope controls (for example, insertion, extraction, left/right angulation, up/down angulation, rotation). In other embodiments, the current and desired US probe position and orientation may be shown on a 3D model such as 1400 to guide the user.
  • step 1710 is executed where the user is provided with a set of optimal US settings such as gain, frequency and depth, with which to best view the target lesion. In some embodiments, these settings may be automatically adjusted using US control and processing unit 104. Finally, in step 1712 the lesion position is overlaid on the US image, as for example shown in image 1600, and the user is notified that the correct US probe position and orientation has been achieved.
  • FIG. 18 shows a schematic representation 1800 of another embodiment of a proposed system for enhancing EUS procedures.
  • a video camera 1802 located at the distal end of the endoscope in close proximity to US probe 102 and tracking sensor 202, is connected to a video control and processing unit 1804 which controls the video camera settings (such as focus and zoom) and processes the video camera output signal.
  • Video control and processing unit 1804 outputs a video stream 1806, which is used as input to visual intelligence unit 214, in addition to the US image stream 106 and the position and orientation of the US probe 212.
  • visual intelligence unit 214 uses video stream 1806 to provide enhanced visual feedback to the user.
  • visual intelligence unit 214 may also receive as input pre-procedure model 916 (not shown) in addition to video stream 1806, US image stream 106 and position and orientation of the US probe 212.
  • visual intelligence unit 214 may use individual images from video stream 1806 to help in pinpointing the location of US probe 102 relative to the patient's anatomy. This information can then be used to facilitate detection and identification step 408 in FIG. 4A, and/or to facilitate registration step 1306 in FIG. 13.
  • FIG. 19 shows examples 1900 of such individual images.
  • Image 1902 shows the diaphragm separating the esophagus from the stomach.
  • Image 1904 is a typical image of the major papilla in the duodenum, which is a well-known station for EUS examinations. By automatically identifying the papilla in video steam 1806, it is possible to determine that US probe 102 is located at this station.
  • visual intelligence unit 214 uses video stream 1806, together with position and orientation of US probe 212 to construct 3D models of the esophagus and/or the stomach and/or the duodenum, or parts thereof, using structure from motion methods known to the art. These 3D models are then used in step 1306 of FIG. 13 to initialize and/or update the registration to pre-procedure model 916.
  • the information derived by visual intelligence unit 214 from video stream 1806 is provided as direct feedback to the user during the examination in order to assist the user in navigation. For example, if a particular point of interest, such as the papilla, is identified in the video stream, this can be displayed to the user so that he/she knows that they are located at this particular EUS station within the duodenum.
  • video control and processing unit 1804 is a separate physical unit. In other embodiments, video control and processing unit 1804 is combined in a single physical unit with one or more of US control and processing unit 104, tracking control and processing unit 210, and visual intelligence unit 214.
  • Some stages (steps) of the aforementioned method(s) may also be implemented in a computer program for running on a computer system, at least including code portions for performing steps of the relevant method when run on a programmable apparatus, such as a computer system or enabling a programmable apparatus to perform functions of a device or system according to the disclosure.
  • Such methods may also be implemented in a computer program for running on the computer system, at least including code portions that make a computer execute the steps of a method according to the disclosure.
  • a computer program is a list of instructions such as a particular application program and/or an operating system.
  • the computer program may for instance include one or more of: a subroutine, a function, a procedure, a method, an implementation, an executable application, an applet, a servlet, a source code, code, a shared library/dynamic load library and/or other sequence of instructions designed for execution on a computer system.
  • the computer program may be stored internally on a non-transitory computer readable medium. All or some of the computer program may be provided on computer readable media permanently, removably or remotely coupled to an information processing system.
  • the computer readable media may include, for example and without limitation, any number of the following: magnetic storage media including disk and tape storage media; optical storage media such as compact disk media (e g., CD-ROM, CD-R, etc.) and digital video disk storage media; nonvolatile memory storage media including semiconductor-based memory units such as FLASH memory, EEPROM, EPROM, ROM; ferromagnetic digital memories; MRAM; volatile storage media including registers, buffers or caches, mam memory, RAM, etc.
  • a computer process typically includes an executing (running) program or portion of a program, current program values and state information, and the resources used by the operating system to manage the execution of the process.
  • An operating system is the software that manages the sharing of the resources of a computer and provides programmers with an interface used to access those resources.
  • An operating system processes system data and user input, and responds by allocating and managing tasks and internal system resources as a service to users and programs of the system.
  • the computer system may for instance include at least one processing unit, associated memory and a number of input/output (I/O) devices.
  • I/O input/output
  • the computer system processes information according to the computer program and produces resultant output information via I/O devices.

Landscapes

  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

La présente invention concerne des systèmes et des procédés d'échographie endoscopique (EUS) améliorés et des procédés pour améliorer les procédures EUS. Dans un système EUS configuré pour délivrer un flux d'images échographiques et une position et une orientation d'une sonde échographique à un moment donné quelconque, un procédé comprend le traitement du flux d'images échographiques et la position et l'orientation de la sonde échographique à un moment donné quelconque dans une sortie visuelle, et l'affichage de la sortie visuelle.
PCT/IB2022/056270 2022-07-07 2022-07-07 Amélioration de procédures d'échographie endoscopique avec intelligence de poursuite et visuelle Ceased WO2024009132A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/IB2022/056270 WO2024009132A1 (fr) 2022-07-07 2022-07-07 Amélioration de procédures d'échographie endoscopique avec intelligence de poursuite et visuelle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/IB2022/056270 WO2024009132A1 (fr) 2022-07-07 2022-07-07 Amélioration de procédures d'échographie endoscopique avec intelligence de poursuite et visuelle

Publications (1)

Publication Number Publication Date
WO2024009132A1 true WO2024009132A1 (fr) 2024-01-11

Family

ID=89454450

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2022/056270 Ceased WO2024009132A1 (fr) 2022-07-07 2022-07-07 Amélioration de procédures d'échographie endoscopique avec intelligence de poursuite et visuelle

Country Status (1)

Country Link
WO (1) WO2024009132A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2026015292A (ja) * 2024-07-18 2026-01-29 オリンパス・ヴィンター・ウント・イベ・ゲゼルシャフト・ミット・ベシュレンクテル・ハフツング ３ｄデータ分析方法、臨床決定支援システム、３ｄデータ分析プログラム、およびコンピュータ可読媒体

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160183767A1 (en) * 2014-12-30 2016-06-30 Siemens Medical Solutions Usa, Inc. Integrated tracking system for inter-patient imaging
US20180042680A1 (en) * 2005-06-06 2018-02-15 Intuitive Surgical Operations, Inc. Interactive user interfaces for minimally invasive telesurgical systems

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180042680A1 (en) * 2005-06-06 2018-02-15 Intuitive Surgical Operations, Inc. Interactive user interfaces for minimally invasive telesurgical systems
US20160183767A1 (en) * 2014-12-30 2016-06-30 Siemens Medical Solutions Usa, Inc. Integrated tracking system for inter-patient imaging

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2026015292A (ja) * 2024-07-18 2026-01-29 オリンパス・ヴィンター・ウント・イベ・ゲゼルシャフト・ミット・ベシュレンクテル・ハフツング ３ｄデータ分析方法、臨床決定支援システム、３ｄデータ分析プログラム、およびコンピュータ可読媒体

Similar Documents

Publication Publication Date Title
US8509877B2 (en) Endoscope insertion support system and endoscope insertion support method
JP6509906B2 (ja) 医療機器の作動方法
JP4653542B2 (ja) 画像処理装置
JP4676021B2 (ja) 診断支援装置、診断支援プログラムおよび診断支援方法
JP4388958B2 (ja) 内視鏡的パスプランニングのための方法及びシステム
JP6080248B2 (ja) ３次元画像表示装置および方法並びにプログラム
US8099155B2 (en) Method for assisting with percutaneous interventions
CN100413458C (zh) 插入支持系统
JP6824125B2 (ja) 医用撮像装置及び画像処理方法
JP6254053B2 (ja) 内視鏡画像診断支援装置、システムおよびプログラム、並びに内視鏡画像診断支援装置の作動方法
US20060195033A1 (en) Insertion support system for specifying a location of interest as an arbitrary region and also appropriately setting a navigation leading to the specified region
WO2007129493A1 (fr) Dispositif permettant d'observation d'une image médicale
JP2016189812A (ja) 内視鏡撮影部位特定装置、方法、プログラム
JP7759436B2 (ja) 医療用ナビゲーションのためのシステムおよび方法
JP2006246941A (ja) 画像処理装置及び管走行トラッキング方法
CN113177945A (zh) 用于将分割图链接到体数据的系统和方法
US20140334706A1 (en) Ultrasound diagnostic apparatus and contour extraction method
CN119053278A (zh) 用于精确的内窥镜外科手术的两阶段器械引导
US20210082164A1 (en) Endoscopic surgery support apparatus, endoscopic surgery support method, and endoscopic surgery support system
US9123163B2 (en) Medical image display apparatus, method and program
WO2024009132A1 (fr) Amélioration de procédures d'échographie endoscopique avec intelligence de poursuite et visuelle
CN119894460A (zh) 用于淋巴结评估的系统和方法
US20250281022A1 (en) Endoscopy support device, endoscopy support method, and recording medium
JP2007014483A (ja) 医用診断装置及び診断支援装置
CN115861168A (zh) 具有附加对象的2d医学图像数据的自动分析

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22950124

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 07/05/2025)

122 Ep: pct application non-entry in european phase

Ref document number: 22950124

Country of ref document: EP

Kind code of ref document: A1