WO2024200545A1 - Dispositif chirurgical de positionnement spatial - Google Patents

Dispositif chirurgical de positionnement spatial Download PDF

Info

Publication number
WO2024200545A1
WO2024200545A1 PCT/EP2024/058305 EP2024058305W WO2024200545A1 WO 2024200545 A1 WO2024200545 A1 WO 2024200545A1 EP 2024058305 W EP2024058305 W EP 2024058305W WO 2024200545 A1 WO2024200545 A1 WO 2024200545A1
Authority
WO
WIPO (PCT)
Prior art keywords
tissue
surgical instrument
light
algorithm
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/EP2024/058305
Other languages
English (en)
Inventor
Zhuoqi CHENG
René Lynge ERIKSEN
Thiusius Rajeeth Savarimuthu
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Syddansk Universitet
Original Assignee
Syddansk Universitet
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Syddansk Universitet filed Critical Syddansk Universitet
Publication of WO2024200545A1 publication Critical patent/WO2024200545A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods
    • A61B17/34—Trocars; Puncturing needles
    • A61B17/3403—Needle locating or guiding means
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0082—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence adapted for particular medical purposes
    • A61B5/0084—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence adapted for particular medical purposes for introduction into the body, e.g. by catheters
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/06—Devices, other than using radiation, for detecting or locating foreign bodies ; Determining position of diagnostic devices within or on the body of the patient
    • A61B5/061—Determining position of a probe within the body employing means separate from the probe, e.g. sensing internal probe position employing impedance electrodes on the surface of the body
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods
    • A61B2017/00017—Electrical control of surgical instruments
    • A61B2017/00115—Electrical control of surgical instruments with audible or visual output
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods
    • A61B2017/00681—Aspects not otherwise provided for
    • A61B2017/00707—Dummies, phantoms; Devices simulating patient or parts of patient
    • A61B2017/00716—Dummies, phantoms; Devices simulating patient or parts of patient simulating physical properties
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods
    • A61B2017/00681—Aspects not otherwise provided for
    • A61B2017/00725—Calibration or performance testing
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2051—Electromagnetic tracking systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2065—Tracking using image or pattern recognition
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/39—Markers, e.g. radio-opaque or breast lesions markers
    • A61B2090/3937—Visible markers
    • A61B2090/3945—Active visible markers, e.g. light emitting diodes
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6846—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be brought in contact with an internal body part, i.e. invasive
    • A61B5/6847—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be brought in contact with an internal body part, i.e. invasive mounted on an invasive device
    • A61B5/6848—Needles
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M25/00—Catheters; Hollow probes
    • A61M25/01—Introducing, guiding, advancing, emplacing or holding catheters
    • A61M25/0105—Steering means as part of the catheter or advancing means; Markers for positioning
    • A61M2025/0166—Sensors, electrodes or the like for guiding the catheter to a target zone, e.g. image guided or magnetically guided
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61M—DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY; DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY; DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR
    • A61M5/00—Devices for bringing media into the body in a subcutaneous, intra-vascular or intramuscular way; Accessories therefor, e.g. filling or cleaning devices, arm-rests
    • A61M5/42—Devices for bringing media into the body in a subcutaneous, intra-vascular or intramuscular way; Accessories therefor, e.g. filling or cleaning devices, arm-rests having means for desensitising skin, for protruding skin to facilitate piercing, or for locating point where body is to be pierced
    • A61M5/427—Locating point where body is to be pierced, e.g. vein location means using ultrasonic waves, injection site templates

Definitions

  • the present invention relates to a device, system and method adapted to detect and provide spatial orientation and position of a surgical instrument in tissue.
  • the invention aims to address the challenge of e.g. needle tip tracking during a deep percutaneous insertion.
  • Such operation is often requested in healthcare such as deep vein/artery intervention, deep brain stimulation (electrodes insertion) and biopsy.
  • the location and orientation of the needle tip is estimated empirically by the operator, and thus, it is hard to guarantee hitting a targeted region especially in a deep tissue region.
  • significant deflection of needle and tissue rupture occurs during the insertion, making the tracking task even more difficult.
  • the above-described object and several other objects are intended to be obtained in a first aspect of the invention by providing a computer implemented method of determining the spatial position and orientation of a surgical instrument, such as a cannula or catheter, when the surgical instrument is positioned within tissue of a subject, the computer implemented method comprising:
  • the invention is particularly, but not exclusively, advantageous for obtaining spatial information relating to the inserting of a surgical instrument, such as a cannula, percutaneously of a patient.
  • the present invention relates to a computer implemented method of determining the spatial position and orientation of a surgical instrument within tissue, the computer implemented method comprising: -emitting light from a portion of the surgical instrument, -providing a light detection device outside of said tissue, -detecting light scatter from within the tissue, with the light detection device, -providing an algorithm adapted to determine the spatial position and orientation of the instrument, relative to a reference point, into said tissue, based on the light scattered from within the tissue, -providing the position and orientation of the surgical instrument to a user.
  • the computer implemented method according to the invention does not construe surgical steps, such as performing any control or movements to the surgical instrument, but provides spatial information as to the position and orientation of said surgical instrument to a user; and furthermore may provide a signal when detecting a change of light scatter related to a change in tissue type at or near the surgical instrument.
  • the invention provides an advantageous method for providing a physician information, to faster, easier and with less inconvenience to a patient to insert e.g. a needle or cannula into a vein or an artery of a patient.
  • the surgical instrument is comprised of an outer cannular, and an inner needle, and wherein the inner needle is adapted with light emitting means.
  • the inner needle is removed while the outer cannula is kept in place at said target, and a secondary surgical device can be positioned within said outer cannula.
  • the surgical instrument is comprised of a first and at least a second lumen, and wherein one lumen is suitable for emitting light and the at least second lumen is suitable for e.g. the insertion of a peripheral device, such as a cannula, needle or other relevant device.
  • a peripheral device such as a cannula, needle or other relevant device.
  • the invention is further advantageous for providing a method of aiding a physician in reaching e.g. a certain part of percutaneous tissue for e.g. tissue biopsy extraction or e.g. deep vessel insertion.
  • tissue biopsy extraction or e.g. deep vessel insertion e.g. tissue biopsy extraction or e.g. deep vessel insertion.
  • the present invention is not limited to entry points of the skin of a patient, but would also be advantageous for guiding a surgical instrument into tissue through e.g. the oral, nasal cavity, vaginal cavity or rectal cavity, wherein the surgical instrument penetrates the tissue from e.g. the oral cavity.
  • guiding is to be construed as continuously providing spatial information relating to the surgical instrument, to the user.
  • the invention may be advantageous for aiding dentists, ophthalmologist and veterinarians, where surgical instruments may be inserted into tissue of a patient. Even further, the invention may be advantageous for aiding a physician or nurse in reaching a vein of very young patients, such as an infant or small child, to reduce pain or discomfort to said young patient.
  • spatial position and orientation is to be understood, as the use of information provided to a user relating to the part of the surgical instrument which is not in view of the user due to said instrument being inserted into and beyond a tissue surface, such as the skin of a patient.
  • the surgical instrument may comprise one or more lumens and wherein, in some embodiments, at least one of said lumens can be used for e.g. technical features of the invention, such as for containing a light source, optical fibre, processing means or other relevant parts of the invention as disclosed; and wherein at least one other of said one or more lumens may be used for e.g. extraction of or injection of a substance.
  • a lumen of the instrument may be suitable for the injection or extraction of a fluid.
  • a lumen of the instrument may be suitable for the insertion or extraction of a solid or semi-solid member, such as for extraction of a biopsy or insertion of an implant.
  • a lumen of the instrument may be used as a catheter for insertion into a tissue or a lumen of the patient.
  • the light emission is directed from a tip end of the surgical instrument, such as in a conical or fan-like direction from said tip end, towards tissue substantially in front of an insertion direction of said tip end.
  • the light is emitted e.g. 90°, 180° or 360° around a periphery of the surgical instrument.
  • a plurality of light emitting devices is adapted to the instrument, to enable the detection of light both around a periphery of and directly in front of said instrument. It is to be understood that light emitted from each of the plurality of light emission sources may be distinct or similar.
  • some of the light emitted is emitted at a first wavelength or with a temporal pattern and/or spatial pattern, and light emitted from e.g. second and/or third sources may be transmitted or provided at different wavelengths or with differing temporal patterns or spatial patterns.
  • the spatial pattern provided from one or more light sources may be a tailored output phase profile adapted to create astigmatic light beams. This embodiment is particularly advantageous for detecting the light scatter within tissue.
  • the spatial pattern provided from one or more light sources may be higher order guided modes.
  • the light provided at the surgical instrument is a lens, in other embodiments, the light is provided from an optic fiber.
  • the spatial pattern provided from one or more light sources may be higher order guided modes from an optical fiber such as doughnut beams.
  • the spatial pattern provided from one or more light sources may be non-diffracting Bessel beams.
  • the above embodiments may be combined, and provides the advantage of enabling better detection of the position, orientation and puncture detection of the surgical instrument, based on light scattered, wherein the one or more lights sources is adapted with a spatial pattern.
  • the surgical instrument may be adapted with a plurality of light sources, and wherein one or more may be adapted with a lens and one or more may be adapted with a temporal pattern.
  • the above embodiments are particularly advantageous , as using at least two, or a plurality of light beams with different focus points in the tissue, can be discriminated by modulating each beam with a distinct modulation frequency, thus the origin of the beam is known by proper demodulation.
  • the light emitted may be a combination of one or more white light, spectrums of light, polarized light, coherent light, temporally patterned light and/or lights emitted at different wavelengths.
  • light detection is to be understood as a means of detecting light, such as a photo-sensor, photodiode(s), Complementary metal-oxide-semiconductor (CMOS), charge-coupled devices (CCD's) or a camera, photo-voltaic sensor (array) or other suitable device.
  • CMOS Complementary metal-oxide-semiconductor
  • CCD's charge-coupled devices
  • camera photo-voltaic sensor (array) or other suitable device.
  • light scatter is to be understood as the physical phenomenon that occurs when light interacts with turbid materials or particles in biological tissues, causing it to be absorbed, depolarized, change direction and scatter in multiple directions.
  • the present invention is adapted to utilize said light scatter, when detected, to define the spatial position and orientation of the surgical instrument, based on algorithms or mathematical models adapted to calculate the spatial information based on said detected light and/or light scatter.
  • entry point is to be understood as the point at which the surgical instrument is entering the tissue of the subject, through the skin, i.e. an analogue spatial reference which is visible to the physician during the procedure.
  • spatial information relates to e.g. a longitudinal axis of the surgical instrument, in an x-y-z plane within said tissue, and the position of a tip-end or other relevant spatial information relevant to a user during a procedure.
  • reference point is to be understood as a defined and static spatial coordinate relative to the point at which light is emitted from the surgical instrument.
  • the reference point is the light detection device.
  • the algorithm is a machine learning algorithm, the method further comprising: -providing said machine learning algorithm with a training set adapted to train the machine learning algorithm to determining the position and orientation of the surgical instrument emitting light within a tissue, the training set being based on predetermined spatial positions, orientations and light scatter patterns of light emitted from tissue.
  • This embodiment is particularly advantageous for providing a method, wherein the algorithm is enabled to be trained on verified training sets, ensuring a more precise determination of spatial position of the surgical instrument.
  • the learning algorithm may be selected from one or more of a convolutional neural network or fundamental image processing in combination with multi-layer perceptron (MLP).
  • MLP multi-layer perceptron
  • the algorithm may be another type of Artificial Intelligence algorithm.
  • Al Artificial Intelligence
  • Machine Learning uses algorithms trained on data sets to create models that enable machines, such as the present invention, to perform tasks without explicit instructions.
  • these tasks include categorizing images and analyzing data.
  • a specific type of machine learning model is a Neural Network. These models make decisions in a manner similar to the human brain, using processes that mimic the way biological neurons work together to identify phenomena, weigh options, and arrive at conclusions. Every neural network consists of layers of nodes, or artificial neurons— an input layer, one or more hidden layers, and an output layer. Each node connects to others, and has its own associated weight and threshold.
  • a specialized type of neural network is a Convolutional Neural Network (CNN).
  • CNNs excel in processing structured grid data, such as images. They are composed of one or more convolutional layers, often followed by pooling layers, then one or more fully connected layers.
  • the convolutional layer uses a set of learnable filters, which are small spatially but extend through the full depth of the input volume. Their application to the input results in activation maps that give the responses of that filter at different spatial positions.
  • CNNs are specifically designed for tasks like image recognition, which makes them a specialized tool within the machine learning toolkit, and particularly advantageous within the present invention.
  • the method further comprises detecting a first tissue based on the light scatter and detecting a second tissue based on a change in the light scatter; providing the user with information that a second tissue has been detected.
  • This embodiment is particularly advantageous for providing the user with information related to whether the tissue structure has changed, thus providing a verification that e.g. a venous tissue has been reached by the user.
  • the embodiment may be able to verify e.g. a puncture of a vein or artery performed by the user, based on a change in light scatter.
  • the method further comprises: -providing light scatter information relating to tissue structures, such as blood vessels, nerves or bone, adjacent to the light emitted from the surgical instrument and providing the user with spatial tissue structure information relative to a current position of the surgical instrument.
  • tissue structures such as blood vessels, nerves or bone
  • tissue structure information is to be understood as the change in scatter pattern and absorption of light between different densities and other properties of different tissues, which in turn can be utilized by the invention to provide further information to the user.
  • the invention is enabled to distinguish between muscle and a tumor based on detected light scatter and provide said information to the user.
  • the method further comprises: -providing verification to the user relating to the surgical instrument entering a tissue structure, such as a blood vessel, within the tissue, based on the detected light scatter from within said tissue structure.
  • the computer implemented method does not involve the surgical step of moving the surgical instrument within tissue, but to convey spatial information relating to the surgical instrument to a user.
  • This embodiment is particularly advantageous for providing a verification of e.g. a tip of the surgical instrument having reached a target, such as a blood vessel.
  • a target such as a blood vessel.
  • the target may be any relevant physiological target, such as a tumor, blood vessel, a specific region or other.
  • the surgical instrument such as a needle tip punctures and enters specific tissues such as blood vessels
  • a high amount of emitted light at specific wavelengths is suddenly absorbed by the tissue, causing a sudden change of scattering pattern detected by the light detection device.
  • the absorption rates of different tissues are different.
  • the absorption coefficients of HbO2, Hb, skin, subcutaneous tissues and muscles are 6, 5, 0.8, 1.07 and 0.32 respectively (unit: cm ). This indicates a significant decrease of intensity of the scattering image which can be detected when the needle tip punctures the blood vessel, thus enabling for verification to a user reaching a specific target such as said blood vessel.
  • the method further comprises mapping tissue structures adjacent to the instrument, based on the light scatter and providing a spatial overview of the tissue to the user.
  • adjacent is to be understood as e.g. mapping tissue between 1 and 10 cm from the point at which light is emitted from the surgical instrument.
  • the invention is enabled to map further in one spatial direction than in another spatial direction.
  • the mapping can be performed distally from a longitudinal axis of the surgical instrument from between 0.1 to 20 cm from a tip end of the surgical instrument.
  • the method further comprises: -receiving an input from the user relating to a selected target within the tissue, -guiding the user to translate the surgical device to said target based on detected light scatter, and when the surgical instrument is positioned at or within said target, -verifying the position of the surgical instrument at or within said target to the user.
  • the computer implemented method does not involve the surgical step of moving the surgical instrument within tissue, but to convey spatial information relating to the surgical instrument to a user.
  • the method further comprises: -receiving an input from the user relating to a selected target within the tissue, -providing a current position and orientation of the surgical instrument to the user; and when the surgical instrument is positioned at or within said target, -verifying the position of the surgical instrument at or within said target to the user.
  • the computer implemented method does not involve surgical steps, such as moving a surgical instrument, but to continuously provide spatial information relating to the surgical instrument, to a user, such as a medical professional.
  • This embodiment is particularly advantageous for aiding a physician in e.g. reaching a blood vessel fast, reducing the discomfort applied to a patient during the procedure.
  • the first method is based on single image input and the second method is based on image sequences.
  • a neural network such as convolutional Neural Network (CNN) or ResNet is used and trained, to identify a puncture event directly based on the acquired image.
  • CNN convolutional Neural Network
  • ResNet ResNet
  • the second method uses a time sequence of images or a video clip as input for the Neural Network in order to determine whether the surgical instrument tip enters a different material, i.e. tissue.
  • the Network architecture may be a CNN + LSTM or 3D CNN.
  • the method further comprises applying a graphics processing units (GPU's) to train and perform the machine learning algorithm.
  • GPU's graphics processing units
  • the GPUs may be used for training and performing the machine learning algorithm.
  • the parallel processing capabilities of GPUs accelerate the training of neural networks. Further, the processing time can be reduced to sub-millisecond when the algorithm is performed on a computer comprising graphical processing units (GPUs). This observation suggests that the proposed system is sufficiently fast for real-time tracking of the needle tip, even when implemented on a portable setup.
  • the invention relates to a spatial positioning and orientation system for a surgical instrument, the system comprising:
  • -an instrument adapted to emit light from a portion of said instrument within tissue, when said portion of the instrument is positioned within tissue
  • -a processor adapted to execute an algorithm, said algorithm determining a spatial position and orientation of the surgical instrument within the tissue, based on light scattered from within the tissue and detected by the light detection device, wherein the system is configured to provide the spatial position and orientation of the surgical instrument to a user through the user interface, while said user operates the surgical instrument within tissue of a subject.
  • the light detection device is a camera positioned at least above an entry point of the tissue, the entry point being the point of entry of the surgical device into the tissue of the subject, the system further comprising a distance measuring device adapted to measure a distance between the entry point and said camera, the device adapted to calibrate the measured light scatter from the tissue to the camera. It is to be understood that at least above is to be understood as e.g. between 5 to 100 cm from said entry point.
  • the light detection device is a sensory array, preferably a patch with an array of sensors, adapted to be adhered to skin of a subject, near an entry point, into tissue of said subject, the sensory array comprising photo-sensors adapted to detect light scatter from within said tissue.
  • near is to be understood as e.g. between 1 to 50 cm from the entry point, depending on where the surgical instrument is to be inserted into the tissue.
  • This embodiment is particularly advantageous, as the use of a photo sensitive I photo detector patch/array compared to a traditional photon counting sensor (CCD or CMOS) allows faster sampling rates kHz-MHz, thus amplitude modulation of the light signal is possible, allowing for lock-in amplification, improving signal to noise ratio.
  • the surgical instrument comprises one or more light sources, such as an LED or an array or LED's, the light source(s) positioned at or near a distal tip of the surgical instrument.
  • the surgical instrument is adapted with a lumen comprising one or more optical fibres adapted to emit light at or near a distal tip of the surgical instrument.
  • the surgical instrument comprises a second lumen adapted for physiological purpose, such as for injecting or extracting matter to/from the patient.
  • the system comprises a plurality of light sources which is adapted to emit light at different wavelengths and/or or at different temporal intervals, to generate a specific pattern of light emitted.
  • the light sources are adapted e.g. at a tip-end of the surgical instrument.
  • the light sources are provided at various locations along a length and periphery of the surgical instrument.
  • said light sources are positioned at length of said surgical instrument, which is within the patient's tissue during normal operation of the surgical instrument.
  • the system is a decision support system and wherein the system is adapted to guide the user to reach a target of the tissue with the surgical instrument, said target being specified by said user through the user interface.
  • the user interface comprises visible or audible output adapted to guide the user to operate the surgical instrument within tissue.
  • This embodiment may be particularly advantageous for teaching purposes, wherein decision support provided through auditory or visual stimulus may increase the proficiency of a user, in turn reducing distress and discomfort to patients.
  • the surgical instrument is adapted as a bioimpedance sensor, to provide further information relating to tissue of the subject.
  • EBI Electrical bioimpedance
  • the electrical bioimpedance can be measured based on one electrode configuration or multiple electrodes configurations. An external ground electrode may also be required to be attached to the body.
  • the measured impedance value can indicate the likelihood of blood contact at the needle tip, since blood is more conductive compared to skin and fat.
  • the venipuncture can be detected through combining two sensing modalities, the proposed optical method and EBI.
  • the sensor fusion methods can be one of the following, but not limited to, Bayesian fusion, covariance intersection, fuzzy logic or decision tree.
  • an electrical bioimpedance measurement verify that the instrument is placed in an artery or a vein.
  • the instrument is expected to be placed in an artery or a vein due to the method of determining a spatial position and orientation of the surgical instrument, then by measuring the electrical bioimpedance, it can be verified that the instrument indeed is placed in an artery or a vein as the electrical bioimpedance can determine the kind of tissue the instrument is in.
  • the combination of multiple sensor modalities can be used by the algorithms to detect both a spatial position and orientation of the surgical instrument as well as verifying whether the surgical instrument is positioned within an artery or vein; or other tissue where electrical conductivity can be used to determine tissue type.
  • the user interface is a peripheral device, such as a tablet or a computer wherein the peripheral device is adapted to receive data wirelessly from the spatial positioning and orientation system.
  • the surgical instrument is adapted with a lens, said lens adapted to focus the light emitted from the surgical instrument.
  • the surgical instrument is adapted with a collimator, said collimator adapted to collimate the light emitted from the surgical instrument.
  • the surgical instrument is adapted with a lens and a collimator, to provide a specific light beam profile.
  • the processor applies graphics processing units (GPU's) to train and perform a machine learning algorithm.
  • the processing time can be reduced to sub-millisecond when the algorithm is performed on a computer comprising graphical processing units (GPUs), suggesting that the method is sufficiently fast for real-time tracking of the needle tip, even when implemented on a portable setup.
  • GPUs graphical processing units
  • the sensory array comprises at least 8x8 sensors, preferably at least 16x16 sensors, even more preferred at least 32x32 sensors.
  • the sensory array comprises between 32x32 and 128x128 sensors.
  • the sensory array comprises between 16x16 and 512x512 sensors or such as between 32x32 and 256x256 sensors.
  • the invention relates to a surgical instrument, preferably a cannula, comprising:
  • processor adapted to receive and process a signal received from the light detection device
  • a wireless transmitter adapted to transmit processed data from the processor to an associated peripheral device, such as a computer, and -optionally an energy storage device, such as a battery, wherein the surgical instrument is configured to detect light scatter from within tissue, when at least the distal tip of the surgical device is inserted into said tissue and provide information relating to tissue structures adjacent to the light source, to a user through said associated peripheral device.
  • the surgical instrument may comprise all the technical features in a single, hand-held instrument.
  • the surgical instrument is substantially sealed, enabling for easy sanitation between use.
  • the invention relates to use of a system or device according to the second or third aspect of the invention for injecting a substance or obtaining a biological sample from a patient.
  • the invention relates to a method of training an algorithm to determine the spatial position and orientation of the surgical instrument according to the system of the second aspect or the surgical instrument of the third aspect of the invention, the training method comprising:
  • the algorithm with a training set adapted to train the algorithm to determine the position and orientation of the surgical instrument emitting light within a tissue, the training set being based on predetermined spatial positions, orientations and light scatter patterns of light emitted from tissue.
  • the training set comprises at least a first set of images obtained from outside of the tissue, and a data set comprising a measured position and orientation of the surgical instrument within the tissue.
  • algorithm is further trained to detect the transition of position of the surgical instrument, from a first tissue to a second tissue.
  • the training set further comprises images obtained from outside of the tissue, wherein the surgical instrument emits light within the first tissue, and images wherein the surgical instrument emits light within the second tissue, training the algorithm to detect a change in tissue.
  • the algorithm is selected from one or more of a convolutional neural network or fundamental image processing in combination with multi-layer perceptron.
  • the first, second, third, fourth and fifth aspect of the present invention may each be combined with any of the other aspects.
  • Fig. 1 shows an illustration of the system and method according to an embodiment of the invention.
  • Fig. 2 shows an illustration of the system and method according to another embodiment of the invention.
  • Fig. 3 shows another illustration of the system and method according to an embodiment of the invention.
  • Fig. 4 shows an illustration of the surgical instrument, according to an embodiment of the invention.
  • Fig. 5 shows an illustration of the system and method according to yet another embodiment of the invention.
  • Fig. 6 shows another illustration of the system and method according to yet another embodiment of the invention.
  • Fig. 7A and Fig. 7B show a chart of a proof-of-concept simulation study.
  • Fig. 8A to 8E show the scattering pattern detected by a light detection device, from a tissue surface.
  • Fig. 9 shows an illustration of the surgical instrument SI, according to another embodiment of the invention.
  • Fig. 10 shows a flow-chart of a method according to an embodiment of the invention.
  • Fig. 11 shows a simplified model that treats the skin and adipose tissues together as a homogeneous material.
  • Fig. 12 shows a visual representation of the model architecture.
  • Fig. 13 shows L2 norm errors in different insertion depths Z*.
  • Fig. 14 shows that the error in the near-surface area is not higher than that in the subsurface region.
  • Fig. 15 shows examples of imaging from two different depths for both the bacon and fresh pork phantoms.
  • Fig. 16 shows a graph, the graph representing a performance study perform in relation to different image resolutions.
  • Fig. 17 shows a confusion matrix, related to a study of puncture detection accuracy by use of the algorithm according to an embodiment of the invention.
  • Fig. 1 shows an illustration of the system 1 and method according to an embodiment of the invention.
  • Fig. 1 shows an arm of a subject SUB, with a surgical instrument SI inserted into tissue of the subject SUB.
  • the surgical instrument SI is adapted with light emission means at a tip end, providing a light from within the tissue of the subject SUB, the light illustrated by light scatter LSC detected by the light detection device LDD.
  • the light detection device LDD is depicted as a camera, but may be another suitable photo sensor adapted to detect light scatter LS.
  • the distance d, between the light detection device LDD and the entry point EP of the surgical instrument SI is known, either by measuring the distance manually or by providing a distance measurement device (not shown) enabled to provide the distance to the system 1.
  • the distance dl is measured perpendicular from the surface of the tissue to the light detection device or a distance measurement device.
  • Fig. 2 shows an illustration of the system 1 and method according to another embodiment of the invention.
  • Fig. 2 shows an arm of a subject SUB, with a surgical instrument SI inserted into tissue of the subject SUB.
  • the surgical instrument SI is in optical connection to a light source LS, to provide a light from within the tissue of the subject SUB, the light illustrated by light scatter LSC detected by the light detection device LDD.
  • the light detection device LDD is depicted as a camera, but may be another suitable photo sensor adapted to detect light scatter LS.
  • the distance d, between the light detection device LDD and the entry point EP of the surgical instrument SI is known, either by measuring the distance manually or by providing a distance measurement device (not shown) enabled to provide the distance to the system 1.
  • the distance dl is measured perpendicular from the surface of the tissue to the light detection device or a distance measurement device.
  • Fig. 3 shows another illustration of the system 1 and method according to an embodiment of the invention.
  • Fig. 3 shows tissue TIS represented by a 3D segment, with a surgical instrument SI inserted into the tissue TIS.
  • the horizontal upper surface of the tissue TIS represents an outer layer of tissue, from which the light detection device LDD can measure light scatter LSC.
  • the surgical instrument SI is adapted to emit light from within the tissue TIS, the light illustrated by light scatter LSC detected by the light detection device LDD.
  • the distance d, between the light detection device LDD and the entry point EP of the surgical instrument SI is known, either by measuring the distance manually or by providing a distance measurement device (not shown) enabled to provide the distance to the system 1.
  • the light detection device LDD obtains an image IMG of the light scatter, in which an algorithm of the system 1 is adapted to convert the detected light scatter LSC into X, Y and Z coordinates of the tip end P of the surgical instrument SI within the tissue, relative to the entry point EP; and the spatial orientation of the longitudinal axis L_A of the surgical instrument SI.
  • the distance dl is measured perpendicular from the surface of the tissue to the light detection device or a distance measurement device.
  • Fig. 4 shows an illustration of the surgical instrument SI, according to an embodiment of the invention.
  • Fig. 4 shows the tip end TE of the surgical instrument SI, the surgical instrument comprising a plurality of lumens LU within the outer periphery OP of the surgical instrument SI.
  • one or more of the plurality of lumens LU may contain at least one light source or a plurality of light sources and/or optic fibres, the optic fibres adapted to emit light from a light source (not shown) in optic connection with said optical fibres.
  • the light source at the needle tip may be programmable by light transferred through a multi-core optic fiber.
  • the scattering pattern can be different and used for further tracking/detection purposes.
  • the proposed invention may be combined with other sensing methods via sensor fusion for a detection task.
  • the needle tube itself can be used as an electrode for electrical bioimpedance sensing.
  • the measured bioimpedance signal can be fused with the optical method in a tissue detection task.
  • Fig. 5 shows an illustration of the system 1 and method according to yet another embodiment of the invention.
  • Fig. 5 shows a light detection device LDD, positioned on the skin of a subject SUB.
  • the light detection device LDD is a patch adhered to the skin, the patch comprising an array or matrix of photo sensors as shown by the white rectangles of the light detection device LDD.
  • the light detection device LDD is adapted to detect light scatter from within tissue of the subject SUB, and wherein the light is emitted from a light source LS in optical connection with the surgical instrument SI.
  • Fig. 5 may be used for generating a closed- loop control in a robotic system.
  • Such robotic system can achieve automatic needle insertion to a pre-defined spatial location, based on spatial information provided by the system and method.
  • the level of melanin of epidermis may affect the accuracy of the proposed method.
  • This obstacle may be solved by e.g. capturing the skin colour of the specific subject, at the beginning of the procedure and input this value to the algorithm.
  • Fig. 6 shows another illustration of the system 1 and method according to yet another embodiment of the invention.
  • Fig. 5 shows the surgical instrument SI being inserted into a blood vessel BV within tissue TIS.
  • the surgical instrument SI is optically connected to a first and second light source LS, LS*, each of the light sources LS, LS* emitting light at different wavelengths WL, WL*, such as at 680 nm and 850 nm respectively.
  • the first and second light sources LS, LS* is operated by a switch SW.
  • the first and second light sources LS, LS* is connected to the surgical instrument SI through an optical combiner, so as to enable for both light sources LS, LS* to emit light through a single optical fibre (not shown) of the surgical instrument SI.
  • each of the first and second light source LS, LS* may be individually connected to respective optical fibres of the surgical instrument.
  • a single light source may be adapted to emit light at different wavelengths.
  • Light scattered from the blood vessel and tissue is detected by the light detection device LDD, generating first and second images IMG, IMG*, showing a more intense light scatter in the first image IMG, relative to the light scatter of the second image IMG*, as light scatter using a first wavelength WL, such as 680 nm scatters more in blood, than when using a second wavelength WL*, such as 850 nm.
  • the proposed invention can be used for tissue identification through applying a lighting spectrum.
  • An example is shown in Fig. 6, in which a switch SW controls two lighting sources LS, LS* of 660 nm and 850 nm respectively, to generate the scatter image. Since the absorption rates of different tissues are different, the generated scatter images by two sources can be compared. Specifically, the differential of scatter image between two sources can be used as input of a machine learning algorithm. As an example, HbR absorbs more light of 660 nm than HbO2, while their absorption rates are similar for light sources of 850 nm. Therefore, if little change of the scatter image in terms of intensity and size is found between two lighting sources, the needle may be determined to have punctured into a vein. If the scatter image is bigger with a 660nm source compared to that with an 850nm source, it is more likely to have punctured an artery.
  • Fig. 7A and Fig. 7B show a chart according to a proof-of-concept simulation study.
  • the invention can be used for indicating the excitation of non-homogeneous anatomic features near the emitted light, such as emitted from a needle bevel NB.
  • a simulation study is conducted as a proof of concept. Finite Element methods are used to simulate the scatting pattern.
  • the left subfigures of Fig. 7A and Fig. 7B are the setup and physical elements of the study, showing depth from a tissue surface TS in mm in both x and y spatial direction where y is depth; and the right subfigures of Fig. 7A and Fig. 7B are the radiation intensity on the surface, as measured from the light detection device.
  • the simulation is simplified as a 2D model and light is emitted from the needle bevel NB.
  • Fig. 8A to 8E show the scattering pattern detected by the light detection device, from a tissue surface TS, of, shape and intensity distribution, as it varies through 8A to 8E, from a change in depth and orientation of a beveled needle tip inside a phantom, the needle tip emitting light.
  • Method 1 fundamental image processing + multi-layer perceptron (MLP)
  • the illumination from the needle tip generates a scattering image on the tissue surface, which is captured by an external imaging sensor.
  • the image is firstly processed through threshholding or multiple threshholding and provides a series of binary images. From the binary images, information of the black circle can be retrieved including centre position (x,y), size S, moment Mx and My. Then, a dataset can be generated with [x, y, S, Mx, My] as input; and the needle tip spatial position [X, Y, Z, Ox, Oy, Oz] as output is generated. It is to be understood that the output is a vector after normalization, and a Sigmoid function is used as the activation function of the output layer.
  • An MLP neural network can be trained for the above regression.
  • the CNN is generally capable of extracting image features owing to the integrated convolution layer.
  • the input is the grey scale image from the imaging sensor and the output is the tip spatial position.
  • the basic CNN architecture passes the input grey scale image to a convolution layer and a pooling layer for feature extraction.
  • the outputs from the pooling layer link to a fully connected network for determining the final output.
  • Fig. 9 shows an illustration of the surgical instrument SI, according to another embodiment of the invention.
  • Fig. 9A shows a trimetric view of the tip end TE of the surgical instrument SI.
  • Fig. 9B shows a cross section of the surgical instrument SI comprising two lumens, wherein the first lumen LUI is adapted for surgical operations, such as for injecting or extracting a fluid.
  • the second lumen LU2 is adapted with a light emission device LED, in this embodiment an optic fibre.
  • Fig. 9C shows a cross section of the surgical instrument SI comprising two lumens, wherein the first lumen LUI is adapted for surgical operations, such as for injecting or extracting a fluid.
  • the second lumen LU2 is adapted with a light emission device LED, in this embodiment an optic fibre, and wherein a lens has been adapted within the second lumen to focus the light emitted from the optic fibre.
  • Fig. 10 is a flow-chart of a method according to an embodiment of the invention, the method comprising the following steps:
  • Deep needle insertion is a medical procedure that allows medical professionals inserting a needle from the skin and reach targets located beneath the surface, such as tumors, nerves, or blood vessels. It plays a crucial role in enabling precise and targeted medical interventions such as administering medication, obtaining samples, or carrying out therapeutic procedures. Therefore, deep needle insertion is widely utilized across a diverse range of medical disciplines, including deep vessel interventions, biopsies, brachytherapy, interventional radiology, anaesthesia, and pain management.
  • needle guidance technology can also incorporate embedded sensors, particularly for the task of needle tip tracking (NTT).
  • NTT needle tip tracking
  • the NTT can be achieved by leveraging the accurate model of the needle deflection.
  • Shape sensing techniques such as strain gauges and Fiber Bragg Gratings (FBG)
  • FBG Fiber Bragg Gratings
  • the FBG technology while offering accurate needle shape reconstruction in real-time, is sensitive to temperature variations and can be relatively expensive to implement.
  • the objective is to present a novel sensing method for tracking the position of a needle tip accurately and in real time.
  • the proposed method incorporates an optical fiber within the lumen of the needle to emit light at the needle tip.
  • an Al-empowered image processing algorithm is employed to analyse the images and estimates the spatial position of the needle tip.
  • the primary focus of this research is to utilize this technique for guiding needle insertion into deep vessels in the groin region, with the ultimate aim of enhancing patient care through improved effectiveness and efficiency.
  • the proposed NTT approach presents a pioneering advancement, and holds significant potential for extension to various other applications such as vessel puncture detection.
  • guiding is to be construed as providing spatial information to a user in relation to the surgical instrument.
  • the invention may be rephrased to a method for aiding surgical tools, such as needles, insertion to a non-homogeneous tissue area inside the body or organ.
  • the target tissue area may be in an irregular shape (tumor), but also can be in a tubular structure (vessels).
  • PDT is a treatment that uses special drugs, sometimes called photosensitizing agents, along with light to kill cancer cells.
  • the drugs only work after they have been activated by certain kinds of light. However, light cannot travel very far through body tissues. This limits its application to treating large cancers or cancers that have grown deeply into the skin or other organs.
  • the invention can be applied for introducing optic needle to the target area and providing lighting during this treatment procedure.
  • the needle bevel face can point downwards, or specifically, towards the vessel, instead of facing upwards.
  • the imaging is able to indicate the proximity of vessel, as well as increase the sensitivity of puncture detection. This is because more light can be absorbed by the blood right after the venipuncture with this trick.
  • one of the primary applications for the proposed technology is to facilitate the guidance of needle insertion into the femoral artery in the groin area.
  • the artery is ensconced within layers of adipose tissue and connective tissue beneath the skin.
  • the distance between the skin surface and the femoral artery in the groin area is subject to variation, particularly in relation to the patient's body mass index (BMI).
  • BMI body mass index
  • the femoral artery usually resides 2 to 3 cm below the skin.
  • the artery can be positioned as deep as 4 cm.
  • Fig. 11 shows a simplified model that treats the skin and adipose tissues together as a homogeneous material, assuming a uniform optical property for both. Given that the bevel tip of the needle consistently points upward during insertion, once the needle tip penetrates the tissue, the light emitted from it travels through the tissue and generates a scattering image on the skin's surface. Moreover, we assume that the tissue surface is flat.
  • the Bouguer-Beer-Lambert law an exponential function, governs the intensity of the light as it travels through the tissue, causing it to attenuate.
  • the intensity at point Pi can be calculated as follows: where di is the distance between the light source Po and Pi, RF and pt represent the diffuse Fresnel reflectance and the attenuation coefficient of tissue respectively.
  • the current system set the distance between the tissue surface and the camera's imaging plane, h, as a fixed value.
  • the generated scattering imaging on the tissue surface has a unique mapping to the needle tip relative position inside the tissue. Since establishing the correlation between the needle tip position and the produced scattering image is complex, an Al based method is developed.
  • the system incorporates a custom-designed needle which is created by inserting an optic fiber (M137L02, Thorlabs Inc., US) into the lumen of a 22G needle and then securing the two components with glue.
  • the Thorlabs 880 nm LED is selected as the light source, because the near-infrared (NIR) spectrum has relatively lower absorption rate in tissue compared to visible light, enabling the light to penetrate thicker tissue. Additionally, the selected light source has a power output of about 0.6 mW, which provides sufficient intensity while maintaining safety.
  • the control and data acquisition system is based on a portable PC (NUC10i7), which manages a micro-controller (Arduino UNO) for toggling the light source on and off.
  • a camera (ASI178MM, ZWO Co. Ltd., China) is positioned 100 mm above the tissue surface and interfaced with the PC to capture the scattering image.
  • a TrakSTAR electromagnet (EM) tracking system (NDI Inc., Canada) is employed to track the spatial position of the needle tip, serving as the ground-truth values for supervised learning.
  • the EM sensor system comprises two main components: a coil and an EM sensor.
  • the coil is securely fastened onto the platform, while the EM sensor is affixed to the needle hub. Please note that the EM sensor is only used for acquiring needle tip position to feed the network training, and is not necessary for the actual deployment.
  • the software system is constructed using the Robot Operation System (ROS) framework, which operates on the NUC PC. Additionally, a system with a Threadripper 5965WX CPU and an NVIDIA RTX 6000 Ada Generation GPU is used for training the Al algorithm which is developed based on PyTorch with PyTorch Lightning. After the Al model has been fully trained, it is tested on the NUC PC to assess its efficacy and performance. We choose to implement the final system on the NUC PC for consideration of system portability.
  • ROS Robot Operation System
  • the calibration procedure comprises two essential steps.
  • the intrinsic parameters of the camera should be acquired in order to generate a distortion-free imaging. This is done by utilizing a chessboard pattern to ensure accurate mapping between real-world coordinates and their corresponding image coordinates. OpenCV lib is employed for calculating these required camera parameters.
  • a coordinate transformation is performed to establish a mapping between the coordinate of the needle tip ⁇ N ⁇ in relation to the global coordinate on the platform ⁇ G ⁇ . This task is accomplished through the EM sensor, whose relative position in relation to the coil T ⁇ a
  • EM can be obtained directly.
  • a pivot calibration is done by manually controlling the needle tip to the origin of ⁇ G ⁇ multiple times and at varying orientations.
  • TTM the relation between the EM sensor and the needle tip
  • the needle tip is navigated to 9 predefined points on the platform, arranged in 3 rows and 3 columns and spaced 30 mm apart. The center of these 9 points is the origin of ⁇ G ⁇ .
  • the least square method is used to fit the collected points, and we can calculate the transformation matrix between the coil frame and the global frame as T oil .
  • the parameter fitting procedure also calculate the residual which corresponds to the Euclidean distance reprojection error of 0.64 mm.
  • the position of the needle tip in the global coordinate system can be calculated as follows: G __ T G n Coil T EM X JV — ⁇ Coil * L EM '
  • the user presses a button on the keyboard and the system automatically save the residual image (downscaled to 400x400) and the current needle tip position with respect to the global coordinate.
  • each image is divided by the maximum pixel value to enhance convergence for the model.
  • the images are initially stored in a uintl6 format, with a pixel value range spanning from 0 to 65535. Consequently, the images are normalized to a range of [0, 1].
  • the 3D positional information corresponds to a spatial region within the global coordinate system.
  • the positional data in the X and Y axes are further normalized to the range of [-1, 1], while the Z-axis data is normalized to the range of [0, 1].
  • Quaternion is used in this invention to represent orientation of the needle tip. Due to the unit norm constraint, the direct probabilistic modelling of quaternion trajectories becomes intractable. For this, we use the method for converting quaternion into Euclidean space. An auxiliary quaternion qo is introduced, and a 4- D quaternion qi can be transformed to a 3-D vector by calculating the logarithm of orientation difference as follows: if u 7 ⁇ 0 if otherwise.
  • the present invention utilizes a Convolutional Neural Network (CNN) model for processing input images.
  • the CNN architecture chosen for this purpose consists of five convolutional blocks, each employing a varying number of filters: 8, 16, 32, 64, and 128.
  • Each block comprises two convolutional layers that employ Rectified Linear Unit (ReLU) activation function.
  • a max pooling layer is employed to down sample the feature map.
  • the final feature map dimensions are 128X4X4.
  • a fully connected layer with 417 neurons, activated by ReLU is incorporated.
  • the output layer of the model consists of two fully connected layers with linear activation, which yield predictions for both needle position and orientation.
  • a visual representation of the model architecture can be seen in Fig. 12.
  • the experimental design aims to validate the proposed method in guiding needle insertion during the femoral artery intravenous task. Also, the test is conducted to evaluate the proposed method's potential in other medical applications that involve more intricate anatomical structures.
  • the evaluation metrics include positional accuracy, orientation accuracy and processing time.
  • the orientation accuracy is quantified by the absolute value
  • the optic needle was manually inserted into the phantom in various depths and orientations for generating datasets. Specifically, the data were collected within a box with a length of 60 mm, a width of 40 mm and a depth of 45 mm. The insertion angle was kept with the needle bevel tip facing up towards the camera. The pitch cp was between 30° and 70° with a yaw 0 from -45° to 45°. The data collection process was conducted with even sampling across the defined space.
  • the second set of experiments involved the use of porcine tissue phantoms to create a realistic evaluation environment.
  • two types of phantoms were obtained from the market: a piece of bacon and a piece of fresh pork slab.
  • the size of both phantoms were big enough for ensuring coverage of the camera's field of view.
  • the bacon phantom featured a 2 mm thick layer of skin and a substantial layer of fat, closely simulating the actual anatomical characteristics of the human groin area.
  • the total thickness of the bacon phantom was measured to be 34.9 ⁇ 2.2 mm.
  • As for the fresh pork phantom a more complicated anatomic structure was observed with intersecting layers of fat and muscle beneath the skin surface.
  • the overall thickness of the fresh pork phantom was found to be 47.8 ⁇ 3.2 mm. This complex anatomical structure allows us to assess the capabilities of the proposed technology in handling more intricate medical applications.
  • a Bayesian hyperparameter tuning approach was employed with the training and validation split from the rubber phantom.
  • the ranges of the explored parameter values can be found in Table 1.
  • a learning rate scheduler was employed if the hyperparameter was true. This scheduler would reduce the current learning rate with a factor of 0.1 if the validation loss of the model did not demonstrate improvement within a span of 30 epochs and terminates the training if no improvement is seen within 100 epochs.
  • MSE Mean Squared Error
  • the following parameter settings were selected from 668 runs: the AdamW optimizer, with a batch size 32.
  • the initial filter count for the convolutional blocks was set to 8 with 5 blocks, while the fully connected layer had a size of 517 neurons with one layer.
  • the learning rate was set to 0.00013 with a learning rate scheduler.
  • the orientation component of the loss was weight with 1.174. Results based on the rubber phantom
  • Table 2 presents the computed position accuracy in the X, Y, and Z components, as well as the L2 norm, based on the test set. For each component, the mean error, standard deviation (STD) and 90 percentile (90%ile) are reported. To mitigate the influence of random weight initialization, the models were trained independently ten times, and the aggregated results were considered for analysis.
  • results in Table 2 indicate that small differences of the position accuracy in the X, Y, and Z component. Additionally, results using 100 ms exposure time are found considerably better compared to the results using 20 ms exposure time. Moreover, the L2 norm errors in different insertion depths Z* are plotted in Fig. 13. The insertion depth Z* is obtained by subtracting the needle tip's Z value, relative to the global coordinate, from the height of the phantom. Relatively higher errors are observed from the first 5 mm of depth, while
  • Table 2 The mean error (mean), standard deviation (STD) and 90 percentile (90%ile) in mm comparing the CNN output and ground-truth on the rubber phantom with 20 and 100 ms exposure time based on the test set.
  • Table 3 The orientation error obtained from the rubber phantom experiment.
  • results from 5 to 40 mm are found relatively consistent. Also, higher positional accuracy of the system with 100 ms exposure time is found compared to that with 20 ms exposure time.
  • , are presented in table 3. Results of different exposure time, namely 20 ms and 100 ms, are shown together for comparison.
  • the positional tracking error based on the bacon and fresh porcine phantoms are summarized in Table 4. Specifically, the errors in the X, Y, and Z directions, as well as their corresponding L2 norm are presented. The observed errors in the X, Y, and Z component exhibits similarities: approximately 1.0 mm for the bacon phantom and 1.6 mm for the fresh pork phantom. In fact, the tracking accuracy (L2 norm) is higher for the bacon phantom compared to the fresh pork phantom, with values of 2.0 ⁇ 1.2 mm and 3.2 ⁇ 3.1 mm respectively.
  • the L2 norm accuracy in different insertion depths Z* are shown in Figure 8.
  • the errors observed in the results obtained from the bacon phantom are relatively smaller when compared to those from the fresh pork phantom. This is coherent to the results reported in Table 4.
  • the results collected from the bacon phantom demonstrate consistency across different insertion depths. In contrast, an increasing trend is evident in the results acquired from the fresh pork phantom.
  • the needle tip orientation tracking errors are provided in Table 5. Slightly higher accuracy in tip orientation estimation is found on the bacon phantom (0.16 ⁇ 0.1) compared to the fresh pork phantom (0.19 ⁇ 0.1).
  • the proposed system and the CNN based tracking algorithm was implemented and tested on a portable PC, NUC (Intel i7 CPU).
  • NUC Intel i7 CPU
  • the processing time was found to be 20.2 ⁇ 0.8 ms.
  • the processing time can be further reduced to sub-millisecond when the same algorithm was tested on a computer with graphical processing units (GPUs). This observation suggests that the proposed system is sufficiently fast for real-time tracking of the needle tip, even when implemented on a portable setup. Discussion
  • Table 4 The mean error (mean), standard deviation (STD) and 90 percentile (90%ile) in mm comparing the CNN output and ground-truth on both the bacon phantom (Bacon) and the fresh pork phantom (Fresh) based on the test set.
  • the L2 norm errors derived from the bacon phantom remained within 3.6 mm, whereas the errors from the fresh pork phantom are generally higher, mostly within 5.4 mm.
  • the bacon phantom more closely resembles the anatomy of the human groin area, where mostly consists of fat. Therefore, the achievement of a 3.6 mm error demonstrates the efficacy of the proposed technology, considering that the radius of the femoral artery typically ranges from 4 to 5 mm.
  • the proposed method is considered sufficient for this application since only the tracking accuracy in X and Y direction are more critical to target the needle to the artery.
  • the results obtained from the fresh pork phantom exhibit reasonable performance up to an insertion depth of 30 mm.
  • the accuracy of needle orientation tracking is measured by the 11 ⁇
  • the results demonstrate that the orientation accuracy achieved is commendable. While it may not be critical for this specific application, the ability to track needle orientation holds significant importance in other medical applications, such as needle steering.
  • the power of the light source has not been optimized.
  • an ultra-low power light source (0.6 mW) is chosen.
  • small manual errors may still involve, affecting the consistency of the transformation matrix between the needle tip and the EM sensor. While these errors are challenging to completely eliminate, multiple measures were taken to minimize their impact.
  • the above positional accuracy in each case must be assessed in relation to the system's calibration. Namely, the Euclidean distance reprojection error, 0.64 mm, should be taken into account when evaluating the system's performance.
  • virtual reality technique can be integrated in the future for guiding the whole insertion process.
  • This invention presents a cutting-edge technology aiming to achieve precise and efficient needle tip tracking during the insertion to a deep subsurface location.
  • the system is developed through combining optical imaging and Al-based algorithms.
  • Experimental evaluations conducted on rubber and porcine tissue phantoms validate the system's capability for accurate and real-time tracking of the needle tip during the insertion procedure.
  • the results highlight the potential of the system to enhance the safety and efficiency of femoral artery insertion procedures, and demonstrate its versatility for different medical applications.
  • Fig. 16 shows a graph, the graph representing a performance study perform in relation to different image resolutions.
  • CNN Convolutional Neural Network
  • the researchers compared the performance of the CNN using original images with those resized to different dimensions.
  • the original images were of size 400x400 pixels, while the resized versions were 8, 16, 32, and 64 pixels.
  • the performance is evaluated as the error in mm between the needle tracking sensor (ground truth) and the model estimate for the 3 coordinates x,y,z.
  • Two sets of samples were utilized in the study: one for training, consisting of nearly 800 images, and another for testing, comprising over 200 images.
  • results are shown in the following figure, with performance, as difference in mm compared between true measured and determined by the CNN, shown on y- axis; and resolution in pixels shown on the x-axis.
  • the performance improves from 8x8 pixels to 32x32 pixels. Beyond this point, however, additional information does not yield further improvement in model performance, leading to a plateau effect.
  • an optimal array of sensors for a patch is between 32x32 and 128x128 sensors.
  • Fig. 17 shows a confusion matrix CONF MAT., related to puncture detection accuracy, based on a study performed on tissue (fat from a pig) and fake blood respectively, using the algorithm according to the invention, to detect a change between a first and second tissue.
  • T. indicates true value
  • PRED. indicates predicted by the algorithm.
  • the algorithm provides a high accuracy in determining a change in tissue, from at least between fat and blood.
  • a confusion matrix is a tool for evaluating the performance of a machine learning model, especially in classification tasks.
  • TP True Positives
  • the Fl score is the harmonic mean of precision and recall.
  • the invention relates to a system and method for providing the depth and orientation of a surgical instrument inserted into biological tissue.
  • the system comprises a surgical instrument adapted to emit light into the biological tissue and a light detection device outside said tissue, and wherein an algorithm is adapted to provide said depth and orientation information, based on light scatter detected by the light detection device; and a peripheral user interface adapted to receive and display said information to a user if the surgical instrument, such as a health care person.
  • the invention is advantageous for aiding e.g. a nurse in reaching a small blood vessel of e.g. an infant or small child with a cannula.
  • the invention is advantageous for training health care personnel to improve their needle insertion technique, such as with a training phantom and provide decision support during needle insertion.
  • E1-E20 the invention may relate to:
  • a computer implemented method of determining the spatial position and orientation of a surgical instrument, such as a cannula or catheter, when the surgical instrument is positioned within tissue of a subject comprising:
  • E2 The computer implemented method according to embodiment 1, wherein the algorithm is a machine learning algorithm, the method further comprising: -providing said machine learning algorithm with a training set adapted to train the machine learning algorithm to determining the position and orientation of the surgical instrument emitting light within a tissue, the training set being based on predetermined spatial positions, orientations and light scatter patterns of light emitted from tissue.
  • the algorithm is a machine learning algorithm
  • the method further comprising: -providing said machine learning algorithm with a training set adapted to train the machine learning algorithm to determining the position and orientation of the surgical instrument emitting light within a tissue, the training set being based on predetermined spatial positions, orientations and light scatter patterns of light emitted from tissue.
  • tissue structures such as blood vessels, nerves or bone
  • the method further comprising: -providing verification to the user relating to the surgical instrument entering a tissue structure, such as a blood vessel, within the tissue, based on the detected light scatter from within said tissue structure.
  • E5. The computer implemented method according to any of the preceding embodiments, the method further comprising mapping tissue structures adjacent to the instrument, based on the light scatter and providing a spatial overview of the tissue to the user.
  • a spatial positioning and orientation system for a surgical instrument comprising:
  • -an instrument adapted to emit light from a portion of said instrument within tissue, when said portion of the instrument is positioned within tissue
  • -a processor adapted to execute an algorithm, said algorithm determining a spatial position and orientation of the surgical instrument within the tissue, based on light scattered from within the tissue and detected by the light detection device, wherein the system is configured to provide the spatial position and orientation of the surgical instrument to a user through the user interface, while said user operates the surgical instrument within tissue of a subject.
  • the light detection device is a camera positioned at least above an entry point of the tissue, the entry point being the point of entry of the surgical device into the tissue of the subject, the system further comprising a distance measuring device adapted to measure a distance between an outer surface of the tissue and said camera, the device adapted to calibrate the measured light scatter from the tissue to the camera.
  • the light detection device is a sensory array, preferably a patch with an array of sensors, adapted to be adhered to skin of a subject adjacent to an entry point into tissue of said subject, the sensory array comprising photo-sensors adapted to detect light scatter from within said tissue.
  • the surgical instrument comprising one or more light sources, such as an LED or an array of LED's, the light source(s) positioned at or near a distal tip of the surgical instrument.
  • E12 The system according to any of embodiments E8, E9 or E10, the surgical instrument being adapted with a lumen comprising one or more optical fibres adapted to emit light at or near a distal tip of the surgical instrument.
  • E15 The system according to any of embodiments E8 to E14, the user interface comprising visible or audible output adapted to guide the user to operate the surgical instrument within tissue.
  • E16 The system according to any of embodiments E8 to E15, wherein the processor applies graphics processing units (GPU's) to train and perform a machine learning algorithm.
  • GPU's graphics processing units
  • a surgical instrument preferably cannula, comprising:
  • processor adapted to receive and process a signal received from the light detection device
  • a wireless transmitter adapted to transmit processed data from the processor to an associated peripheral device, such as a computer, and -optionally an energy storage device, such as a battery, wherein the surgical instrument is configured to detect light scatter from within tissue, when at least the distal tip of the surgical device is inserted into said tissue and provide information relating to tissue structures adjacent to the light source, to a user through said associated peripheral device.
  • E19 The surgical instrument according to E18, further comprising a bioimpedance sensor adapted to measure changes in electrical conductivity.
  • E20 The surgical instrument according to E19, wherein the bioimpedance sensor is integrated into or near the distal tip of the instrument.
  • the algorithm with a training set adapted to train the algorithm to determine the position and orientation of the surgical instrument emitting light within a tissue, the training set being based on predetermined spatial positions, orientations and light scatter patterns of light emitted from tissue.
  • E27 The method of training an algorithm according to E26, wherein the training set comprises at least a first set of images obtained from outside of the tissue, and a data set comprising a measured position and orientation of the surgical instrument within the tissue.
  • E35 The computer implemented method according to any of the embodiments El to E7 or E21 to E23, wherein the emitted light is adapted with a spatial pattern and temporal pattern.
  • E36 The computer implemented method according to any of the embodiments El to E7 or E21 to E23 or E33 to E35 further comprising

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Pathology (AREA)
  • Physics & Mathematics (AREA)
  • Biophysics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Human Computer Interaction (AREA)
  • Robotics (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

L'invention concerne un système et un procédé destinés à fournir la profondeur et l'orientation d'un instrument chirurgical inséré dans un tissu biologique. Le système comprend un instrument chirurgical conçu pour émettre de la lumière dans le tissu biologique et un dispositif de détection de lumière à l'extérieur dudit tissu et un algorithme étant conçu pour fournir lesdites informations de profondeur et d'orientation, sur la base d'une diffusion de lumière détectée par le dispositif de détection de lumière; et une interface utilisateur périphérique conçue pour recevoir et afficher lesdites informations à un utilisateur de l'instrument chirurgical, tel qu'une personne de soins de santé. En particulier, l'invention est avantageuse pour une insertion dans un vaisseau profond ou pour aider, par exemple une infirmière, à atteindre un petit vaisseau sanguin, par exemple, d'un nourrisson ou d'un petit enfant à l'aide d'une canule. En outre, l'invention est avantageuse pour l'entraînement de personnel de soins de santé afin d'améliorer leur technique d'insertion d'aiguille, par exemple avec un fantôme d'entraînement et de fournir un support de décision pendant l'insertion d'aiguille.
PCT/EP2024/058305 2023-03-29 2024-03-27 Dispositif chirurgical de positionnement spatial Ceased WO2024200545A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
EP23165112.6 2023-03-29
EP23165112 2023-03-29
EP23203066.8 2023-10-11
EP23203066 2023-10-11

Publications (1)

Publication Number Publication Date
WO2024200545A1 true WO2024200545A1 (fr) 2024-10-03

Family

ID=90368677

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2024/058305 Ceased WO2024200545A1 (fr) 2023-03-29 2024-03-27 Dispositif chirurgical de positionnement spatial

Country Status (1)

Country Link
WO (1) WO2024200545A1 (fr)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070293748A1 (en) * 2004-06-24 2007-12-20 Redsense Medical Ab Means and Method for Detection of Blood Leakage from Wounds
US20080039715A1 (en) * 2004-11-04 2008-02-14 Wilson David F Three-dimensional optical guidance for catheter placement
US20080097378A1 (en) * 2006-08-02 2008-04-24 Zuckerman Stephen D Optical device for needle placement into a joint
WO2009019707A1 (fr) * 2007-08-08 2009-02-12 Impediguide Ltd. Procédé et dispositif d'identification de tissu
US20140243656A1 (en) * 2011-06-30 2014-08-28 Kai Kronström Arrangement For Defining A Location Within An Organism And method For Manufacturing A Mandrin To be Accommodated In a Needle
US20170259013A1 (en) * 2012-10-30 2017-09-14 Elwha Llc Systems and Methods for Generating an Injection Guide
US20200337781A1 (en) * 2019-04-24 2020-10-29 Board Of Regents, The University Of Texas System Systems and methods for locating an inserted catheter tip

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070293748A1 (en) * 2004-06-24 2007-12-20 Redsense Medical Ab Means and Method for Detection of Blood Leakage from Wounds
US20080039715A1 (en) * 2004-11-04 2008-02-14 Wilson David F Three-dimensional optical guidance for catheter placement
US20080097378A1 (en) * 2006-08-02 2008-04-24 Zuckerman Stephen D Optical device for needle placement into a joint
WO2009019707A1 (fr) * 2007-08-08 2009-02-12 Impediguide Ltd. Procédé et dispositif d'identification de tissu
US20140243656A1 (en) * 2011-06-30 2014-08-28 Kai Kronström Arrangement For Defining A Location Within An Organism And method For Manufacturing A Mandrin To be Accommodated In a Needle
US20170259013A1 (en) * 2012-10-30 2017-09-14 Elwha Llc Systems and Methods for Generating an Injection Guide
US20200337781A1 (en) * 2019-04-24 2020-10-29 Board Of Regents, The University Of Texas System Systems and methods for locating an inserted catheter tip

Similar Documents

Publication Publication Date Title
JP6568478B2 (ja) 低侵襲治療のための計画、誘導およびシミュレーションシステムおよび方法
Gillette et al. Recent developments in canine locomotor analysis: a review
JP2024074910A (ja) 手術ロボット外科手術のための人工知能
CN105491952B (zh) 探头定位
Tsuge et al. Preoperative vascular mapping for anterolateral thigh flap surgeries: a clinical trial of photoacoustic tomography imaging
US7865236B2 (en) Active electrode, bio-impedance based, tissue discrimination system and methods of use
KR102245150B1 (ko) 초음파 발생 시스템 및 그의 제어 방법
CN113017669A (zh) 医学图像处理的装置、方法、计算机可读程序以及移动对象跟踪装置和放射治疗系统
KR20150019311A (ko) 환자와 3차원 의료영상의 비침습 정합 시스템 및 방법
US20240268784A1 (en) Ultrasound probe
CN116824173A (zh) 医学图像的处理方法、设备及存储介质
CN120544792A (zh) 一种基于数字孪生的人机协作式高精度针灸方法及系统
US20240000511A1 (en) Visually positioned surgery
CN115171464B (zh) 一种可视化穿刺模具
Cheng et al. Needle tip tracking based on optical imaging and AI
KR20210126410A (ko) 종양을 추적하기 위한 움직임 자동 평가 장치 및 이를 이용한 방사선 치료 시스템
CN111481268A (zh) 颅底卵圆孔自动定位及引导系统
Ese et al. Influence of 12-bit and 16-bit CT values of metals on dose calculation in radiotherapy using PRIMO, a Monte Carlo code for clinical linear accelerators
CN103648424B (zh) 能量施加规划设备
Årsvold et al. Lymph node detection using robot assisted electrical impedance scanning and an artificial neural network
WO2019153983A1 (fr) Système d'imagerie thermique de pointe de scalpel chirurgical et procédé associé
CN119138834A (zh) 一种基于内窥成像传感器的神经刺激装置
CN118216983A (zh) 经皮穿刺方法、装置、系统、电子设备及存储介质
CN209490085U (zh) 肿瘤治疗仪
CN112308817A (zh) 对结构的自动定位

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24712872

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 24712872

Country of ref document: EP

Kind code of ref document: A1