WO2017100139A1 - Système de biopsie optique par imagerie suivie - Google Patents

Système de biopsie optique par imagerie suivie Download PDF

Info

Publication number
WO2017100139A1
WO2017100139A1 PCT/US2016/065014 US2016065014W WO2017100139A1 WO 2017100139 A1 WO2017100139 A1 WO 2017100139A1 US 2016065014 W US2016065014 W US 2016065014W WO 2017100139 A1 WO2017100139 A1 WO 2017100139A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
data
optical probe
optical
images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2016/065014
Other languages
English (en)
Inventor
Mark Pierce
Robert Toth
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Publication of WO2017100139A1 publication Critical patent/WO2017100139A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B10/00—Instruments for taking body samples for diagnostic purposes; Other methods or instruments for diagnosis, e.g. for vaccination diagnosis, sex determination or ovulation-period determination; Throat striking implements
    • A61B10/02—Instruments for taking cell samples or for biopsy
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/06—Devices, other than using radiation, for detecting or locating foreign bodies ; Determining position of diagnostic devices within or on the body of the patient
    • A61B5/065—Determining position of the probe employing exclusively positioning means located on or in the probe, e.g. using position sensors arranged on the probe
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/012—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor characterised by internal passages or accessories therefor
    • A61B1/0125—Endoscope within endoscope
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/012—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor characterised by internal passages or accessories therefor
    • A61B1/018—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor characterised by internal passages or accessories therefor for receiving instruments
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/042—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances characterised by a proximal camera, e.g. a CCD camera
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A61B1/07—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements using light-conductive means, e.g. optical fibres
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B10/00—Instruments for taking body samples for diagnostic purposes; Other methods or instruments for diagnosis, e.g. for vaccination diagnosis, sex determination or ovulation-period determination; Throat striking implements
    • A61B10/02—Instruments for taking cell samples or for biopsy
    • A61B10/04—Endoscopic instruments, e.g. catheter-type instruments

Definitions

  • This document relates generally to optical probe-based assessment of biological tissues.
  • a key prognostic indicator is the status of tissue at the surgical margin, with positive margins universally associated with poor outcomes.
  • the challenge facing the surgeon is then to balance the use of wide margins to ensure complete removal of tumor, against preserving as much healthy tissue as possible to retain cosmetic appearance and function. This challenge is particularly acute with the shift towards more conservative procedures, such as lumpectomy versus mastectomy in breast cancer.
  • Figure is a block diagram illustrating components of an optical biopsy system, according to an embodiment.
  • Figure 2 is a block diagram illustrating components of an optical biopsy system using optical sectioning, according to an embodiment.
  • Figure 3 is an example three dimensional model generated by an optical biopsy system, according to an embodiment.
  • Figure 4 is an example two dimensional model generated from a three dimensional model by an optical biopsy system, according to an embodiment.
  • Figure 6 is an example user interface generated by an optical biopsy system, according to an embodiment.
  • Figure 7 is a flow chart depicting processes performed by an optical biopsy system, according to an embodiment.
  • Figure 8 is a block diagram depicting a computer system of an optical biopsy system, according to an embodiment.
  • systems described herein may include an optical probe, a tracking system to track the position of the optical probe, and a system for optical sectioning to improve diagnostic performance of the optical probe.
  • Non-invasive fiber-optic imaging and/or spectroscopy probes may provide clinicians with the ability to assess tissue status in real-time. This information can be used to help the user decide which site(s) to take a biopsy from, whether a biopsy is even necessary, or where a resection margin should be during surgery.
  • Current optical biopsy imaging (OBI) probes including epi-fluorescence, scanning confocal, and multiphoton variations) only provide discrete images from each site examined.
  • Marking or tagging optical measurement sites for subsequent biopsy may include creating a dimple on the tissue by pressing the probe against the mucosa or using a cauterizing laser to physically mark the site.
  • Embodiments of the optical biopsy systems described herein may integrate fiber-optic OBI hardware with a tracking device to acquire both optical imaging information as well as the precise 3D location of the imaging probe.
  • Various probe-based optical imaging and spectroscopy devices may provide microscopic level information about target portions of a subject. These optical probes have demonstrated good diagnostic accuracy in laboratory and pilot clinical studies but face practical challenges in gaining widespread clinical acceptance.
  • One challenge is cost. Some confocal platforms are expensive due to the need for a rapidly scanned beam from a laser source. Multi-photon systems may also be cost-prohibitive as an optical probe.
  • a second challenge which is common to these devices relates to how they are used in practice. For instance, when performing a biopsy, a medical practitioner may collect biopsy specimens from precisely documented anatomical locations. The specimens could then be sent to a pathology lab for processing and interpretation by a trained pathologist.
  • optical probes and other diagnostic instruments that provide a real-time ability to explore a tissue field prior to committing to biopsy, raises new challenges for integration within the clinical workflow.
  • a medical practitioner must obtain clinical data from optical probes while documenting individual sites, interpret the images received from the optical probe, and return to specific sites where an image was taken by an optical probe.
  • a practitioner may wish to use multiple images in a region to gather sufficient information to make a diagnosis or map a region of diagnosis.
  • the systems described herein incorporate an optical probe to provide detailed microscopic level information with a tracking device to track the location of images and optical sectioning technology.
  • the optical sectioning technology may improve the images provided by the optical probe by blocking out of focus light from the image.
  • an optical biopsy system may integrate an in-vivo Optical Biopsy Imaging ("OBI") with electromagnetic (EM) tracking hardware, and three dimensional (“3D”) data registration systems.
  • a tracking system other than an EM tracking system may be used to determine a position of the optical probe.
  • the optical biopsy system may allow users to examine tissue at the cellular scale in living patients and log the spatial location(s) of imaged sites. This may enable the user to precisely return to areas of interest (e.g., high-risk sites for biopsy collection) based on the logged spatial location(s).
  • the logged spatial location(s) may also be stitched together into a single large area image for improved diagnostic ability or guidance on surgical margin status.
  • the spatial locations corresponding to images may be mapped into a three dimensional space for viewing or analysis.
  • an optical probe may be implemented as a handheld probe for use during minimally invasive procedures, open surgeries, or dermatologic
  • the location of the optical probe may be tracked using a tracking device.
  • the optical biopsy system may then execute 3D tracking and image integration systems for data processing/display and may allow a clinician to return to previous potential biopsy locations for biopsy/treatment.
  • the optical biopsy system may be used in a variety of applications.
  • the present solution can be used in Gastrointestinal ("GI"), urinary, reproductive, respiratory, skin and other tissue/organ systems.
  • the handheld probe may also be used in conjunction with an endoscope, laparoscope, or the like to perform respective procedures.
  • a tracking device may be integrated with an optical probe and provide improved imaging with optical sectioning. Some embodiments may further accurately map an image from the optical probe to a 3D location in a virtual space. An optical biopsy system may then provide a view of the 3D space with captured images. In some embodiments, an optical biopsy system may generate 2D views of the captured images from a perspective in the 3D space. In some embodiments, the optical biopsy system may stitch together images from the 3D space to generate a 2D view of a portion of the 3D space. In some embodiments, a practitioner may use a 2D or 3D model generated by the system to return the optical probe, or another instrument, to a previously logged position that is identified as an area of interest.
  • the tracking system may include an EM tracker that tracks the position of an EM device attached to the optical probe.
  • EM tracker that tracks the position of an EM device attached to the optical probe.
  • other tracking devices and systems may be used. Any known or to be known tracker can be used herein which provides position and/or orientation values.
  • a mechanical tracker can be used in addition to or alternative to an EM tracker.
  • the tracking devices may provide tracking with 6 degrees of freedom (e.g., an x value, a y value, a z value, a roll value, a pitch value, and a yaw value).
  • a tracking device may provide fewer degrees of position or orientation values if they can be inferred by other means.
  • the position of the tracker relative to the subject may provide sufficient data without additional orientation data.
  • more than one tracker may be used to provide position and/or orientation values when combined.
  • one tracker may provide position data and a second tracker (potentially of a different kind) may provide orientation data.
  • two trackers providing position data may use their relative positions to determine orientation data.
  • the data from multiple trackers may be combined (for example by averaging) in order to reduce variability or other sources of inaccurate tracking information from a single tracker.
  • tracking values from a single tracker may be combined (for example by a weighted average of the data from a preceding time interval) as an alternative method for reducing variability or other sources of inaccuracies.
  • Various trackers may include EM trackers, gyroscopes, accelero meters, potentiometers, proximity sensors, or the like.
  • an optical biopsy system as described herein may be used as a handheld imaging probe, for applications in an open area of tissue (such as the skin) or surgical field.
  • an optical biopsy system may also be used in minimally- invasive procedures.
  • the handheld imaging probe may be designed to fit through a standard endoscope or laparoscope accessory channel.
  • a practitioner may move the probe to a new location.
  • the tracking system may record the location of the tracker as the images are collected from the optical probe.
  • the optical biopsy system may then use the position of the tracker to determine a position of the optical probe as an image is taken.
  • a position in 3D space of an image received from an optical probe may be determined.
  • the optical biopsy system may generate a 3D map of the captured images using the position and image data.
  • the 3D map may be generated without using overlapping portions of the optical biopsy data to determine the positions.
  • an optical biopsy system as described herein may include hardware and processing to provide optical sectioning. Using optical sectioning, an optical biopsy system may block out of focus light from distorting the images received by the optical probe.
  • a light source may provide structured illumination to a target area of an optical probe. The light received by the optical probe may then be processed by the optical biopsy system to block out of focus light received by the optical probe from distorting the received images.
  • optical sectioning may be performed by confocal methods, nonlinear optical methods, coherent illumination methods, coherence gating methods, or other approaches.
  • the optical biopsy system may include a wide-field imaging device.
  • the wide-field image device may provide wide-field images or video.
  • the wide-field device may be a camera that provides a view of an area including the areas viewed by the optical probes.
  • the wide-field images may be used by the optical biopsy system to generate wide field images that include images from the optical probe.
  • the position of a tracker relative to the wide-field imaging device may be used to determine a location in the field of view of the wide-field imaging device of the optical probe.
  • the images received from the optical probe may then be superimposed onto the images received from the wide-field imaging device. The view may therefore be improved to include a wide-field view of a patient with additional data received from an optical probe.
  • Figure 1 depicts an optical biopsy system 100, according to an embodiment.
  • the optical biopsy system may include a biopsy computing system 1 10 in communication with a tracking system 120 and an optical probe 130.
  • the tracking system 120 may track the position of a tracker 125.
  • the optical probe 130 may provide imaging data.
  • a light source 135 may provide light that is captured by the optical probe 130 when reflected (or fluorescent light generated by the light from the light source).
  • the optical computing system 1 10 may receive input streams from the optical probe 130 and the tracking system 120.
  • the optical biopsy system 100 may also communicate with a wide-field imaging device 140 that provides wide-field imaging.
  • the optical biopsy system 100 may include fewer or additional components.
  • the optical biopsy system 100 may not have a wide-field imaging device 140, or may have more than one optical probe 130, light source 135, or tracker 125.
  • the light source 135 may provide light in a manner to improve the images received by the optical probe 130.
  • the light source 135 may provide light in a manner to enable optical sectioning. In optical sectioning the light that is generated from locations that are out of focus from a plane of interest to the optical probe 130 may be blocked out.
  • the optical probe 130 may generate a 2D image of the tissue of a subject. The 2D image may be in a particular plane in the tissue of the subject. However, the optical probe 130 may also receive other interfering light that is reflected from other planes.
  • the light source 135 may provide light to enable optical sectioning.
  • the out of focus light may be blocked at the optical probe.
  • the light source 135 may provide structured illumination. For example, illumination that is uneven.
  • the imaging data generated by the optical probe 130 may then be processed by the biopsy computing system 1 10 to block out of focused light received by the optical probe 130 in the imaging data. Techniques for optical sectioning that may be used by the optical probe 130 and light source 135 are discussed further below with reference to Figure 2.
  • the light source 135 may be provided as part of optical probe 130.
  • the optical probe 130 is a fiber optic bundle, and the light source 135 may provide light to the fiber optic bundle that is then transmitted to the end of the fiber optic bundle.
  • the optical probe 130 may provide light as a light source 135 and provide imaging data as an optical probe 130.
  • the light source 135 may be separate and/or independent from the optical probe 130 and the tracker 125.
  • the tracking system 120 tracks the tracker 125.
  • the tracker may be an EM tracker, or any other type of tracker that provides position and/or orientation data.
  • an EM tracking system 120 may generate a magnetic field in the vicinity of the tracker 125.
  • the tracker 125 may cause changes in the magnetic field that can then be sensed by the tracking system 120 and used to generate position and/or orientation data of the tracker 125.
  • the tracking system 120 may track more than one tracker 125.
  • additional trackers may provide more accurate information, or may be used together to provide different information.
  • a first tracker may be used to track position while a second tracker is used to track an orientation, for instance.
  • the tracking system 120 may be integrated with the biopsy computing system 110, or other components of the optical biopsy system 100. In some embodiments, the tracking system 120 is stationary within the vicinity of a subject.
  • the tracking system 120 may then track the position of tracker 125 relative to the position of the tracking system 120.
  • the tracking system 120 may track the tracker 125 relative to another component.
  • the position of tracker 125 may be tracked relative to a second tracker located on the body of a patient. This may enable the location of the tracker 125 on or within a patient to be determined even if the patient moves during the procedure. For instance, if the tracker 125 is tracked relative to a component on a patient' s chest, the position of the tracker 125 may be relative to the patient' s body even if the patient sits up, rolls over, or otherwise moves.
  • the tracking system 120 may be placed on the patient' s body (e.g., on the patient' s chest) to determine a position of the tracker 125 relative to the patient' s body instead of relative to a space the patient occupies.
  • the tracking system 120 outputs position and orientation data for the tracker 125 data with six (6) degrees of freedom.
  • the six (6) degrees of freedom are described in terms of x, y, and z locations as well as yaw, pitch, and roll angles.
  • the position of the optical probe 130 may then be derived from the position and orientation of the tracker 125.
  • the tracking system 120 may output the location and orientation of the tracker 125 to the biopsy computing system 110.
  • the tracking system 120 may output the position of an image captured by the optical probe 130 to the biopsy computing system 1 10.
  • a wide-field imaging device 140 may have a tracker that determines the position of the wide-field imaging device 140 relative to the tracker 125 or the optical probe 130. Tracking the wide-field imaging device 140 may enable the biopsy computing system 1 10 to determine a perspective matrix for the imaging device (which maps 3D input locations onto 2D perspective locations to be displayed on a computer monitor). Any stored OBI image locations may also be mapped onto the 2D display relative to the perspective matrix of the wide-field imaging device 140.
  • the biopsy computing system 110 may generate 2D locations for the optical biopsy images from the position and orientation of the tracker 125, the perspective matrix of the wide-field imaging device 140, and the previously stored positions of optical biopsy images.
  • the biopsy computing system 1 10 may include a 3D mapping system 1 12, and an image processing system 1 14.
  • the biopsy computing system 1 10 may receive two input data streams.
  • a first input stream may be received from tracking system 120.
  • the second input stream may be received from the optical probe 130.
  • the input stream received from the tracking system 120 may be processed by the 3D mapping system to determine a position in a virtual 3D space of an optical probe 130 based on position and/or orientation data of the tracker 125.
  • the input stream received from the optical probe 130 may be processed by an image processing system 1 14 to generate optical biopsy images.
  • the biopsy computing system 110 may then combine the 3D mapping information with the optical biopsy images to generate a virtual 3D map of optical biopsy images.
  • the 3d mapping system 112 of the biopsy computing system 110 may determine a position of an optical probe 130 from data received from a tracking system 120.
  • the tracking system 120 outputs position and orientation data for a tracker 125.
  • the 3D mapping system 112 may use the position and orientation data of the tracker 125 to determine a position and/or orientation of the optical probe 130 or a position of imaging data provided by the optical probe 130.
  • the position of the end of optical probe 130 may be known relative to the position of the tracker 125, and the orientation of the optical probe may also be known.
  • the 3D mapping system 112 may use the received position data and orientation data to generate a position and orientation of an image.
  • the 3D mapping system 1 12 may receive x, y, and z coordinates, and yaw, pitch, and roll angles of a tracker 125. The 3D mapping system may then use the orientation data (yaw, pitch, and roll angles) to determine a position of the imaging data relative to the x, y, and z coordinates of the tracker 125. Thus, in an example, the 3D mapping system may determine that the optical scope 130 is in a position ⁇ ' , y' , z' in response to determining a position x, y, z for the tracker 125.
  • the position x' , y' , z' may be determined based on a transformation of the yaw, pitch, and roll of the tracker 125 and a set distance from the tracker 125 to the optical probe 130. For example, if the tracker 125 is rigidly attached to the optical probe 130, the distance between the two may be a set distance.
  • the position of the optical probe 130 ( ⁇ ' , y' , z' ) may be determined as being the set distance in a direction defined by the yaw, pitch, and roll of the tracker 125 from the position (x, y, z) of the tracker 125.
  • 3D mapping system 1 12 may perform other transformations to determine a location in a virtual 3D space of imaging data.
  • the 3D mapping system 1 12 may receive tracking data for more than one tracker 125.
  • the location of multiple trackers may then be used to determine a position and/or orientation of the optical probe 130.
  • position data of three trackers may be used to determine yaw, pitch, and roll of the probe.
  • the position of the optical probe 130 may be derived from the position of one of the trackers and the yaw, pitch, and roll, as discussed above.
  • two trackers may be aligned such that an optical probe 130 is known to be located a set distance along an axis formed by the position of the two trackers.
  • a location x' , y' , z' of the optical probe 130 may be determined based on a geometric transformation a set distance along the axis from a point x, y, z that is one of the trackers.
  • the 3D mapping system 112 may determine a position of optical probe 130 (and associated imaging data) in different manners.
  • the 3D mapping system 1 12 may also determine an orientation of the imaging data. For example, the direction the optical probe 130 is facing may be used to determining an orientation of imaging data provided by the optical probe.
  • the image processing system 1 14 generates images based on the imaging data received by optical probe 130.
  • generating images may include recording images received directly from the optical probe 130.
  • image processing system 1 14 may improve the quality of images before storing them for viewing by a practitioner.
  • imaging data received from an optical probe 130 may have irregularities based on the type of optical probe 130.
  • a fiber optic bundle may have distortions from the individual fibers, for instance.
  • the imaging data received by the image processing system 114 may have images that have reflectance from structured illumination of the subject.
  • the image process system 114 may perform image processing operations to generate an image by blocking out of focus light from one or more received images.
  • the image processing system 114 may process the imaging data to highlight specific cellular attributes in addition to improving quality.
  • imaging data associated with multiple wavelengths of light may be combined into a color image.
  • the biopsy computing system 1 10 may store the images generated by the image processing system 1 14 in an image data store 1 16.
  • the biopsy computing system 110 may include position and/or orientation data associated with the imaging data in image data store 1 16.
  • the stream of imaging data received by the image processing system 1 14 may be correlated with the stream of position and/or orientation data received by the 3D mapping system.
  • the biopsy images output by the image processing system 1 14 may then be associated with the corresponding output of the 3D mapping system.
  • the position and/or orientation data may be stored as metadata for the images, possibly for future retrieval.
  • the images and position and/or orientation data may be stored separately, but may be stored with an identifier indicating that the two are associated.
  • the optical probe 130 is a flexible fiber-optic bundle to deliver illumination light to the tissue and to transmit an image back to an external camera unit.
  • the fiber optic bundle may have an outer diameter of 0.45 mm to 1.70 m.
  • the optical probe 130 and tracking device 125 may be designed to fit through an auxiliary port of a standard endoscope (typically 2.8-3.2 mm). In some
  • the outer diameter for a handheld imaging probe comprising the optical probe 130 and the tracker 125 may be approximately 2 mm which may be achieved through integration of a 0.96 mm outer diameter imaging fiber with a 16-gauge (0.9 mm outer diameter) EM tracker.
  • a tracked probe with this diameter may be used in standard diameter colonoscopes and gastroscopes, and is compatible with some smaller Ear Node Throat ("ENT") scopes and bronchoscopes.
  • the outer diameter of a tracked probe may be smaller or larger than described.
  • the tracked probe may have an outer diameter of approximately 0.5mm, 1mm, 1.5mm, 2.5mm, or other diameters depending on the application.
  • Figure 2 depicts an example of optical sectioning performed by an optical biopsy system, according to an embodiment.
  • an optical biopsy system may block out of focus light from reducing the quality of light on a plane being observed by an optical probe.
  • an optical probe 230 observes a tissue specimen 210.
  • the optical probe 230 may be similar or the same as the optical probe 130 described above with reference to Figure 1.
  • the optical probe 230 may be connected to an optical biopsy system and may be coupled to a tracker to determine the position of optical probe 230.
  • the optical probe 230 is a bundle of fiber optic fibers.
  • the optical probe 230 may have as a light source, or another light source may provide incident light 240 to the tissue sample 210.
  • the incident light 240 may be of a particular frequency that promotes diagnosis in imaging data received by the optical probe 230.
  • the incident light may provide structured illumination that enables optical sectioning by the optical probe 230 or a connected image processing system.
  • the incident light may illuminate a portion 220 of the tissue specimen 210 and reflected light 250 may return from portion 220. As shown in Figure 2, the illuminated portion 220 may extend through a depth 222 of the tissue specimen 210. However, the optical probe 230 may be focused on a particular plane of the tissue specimen 210.
  • the light reflected from the illuminated portion 220 of the tissue specimen 210 that is not within the in focus area 225 may provide out of focus light to the optical probe 230.
  • the optical probe 230 may block out of focus light from generated imaging data.
  • the optical sectioning may be performed by processing of one or more images generated by the optical probe 230.
  • the optical probe 230 may take multiple images of the tissue specimen 210 while incident light 240 provides structured illumination.
  • the structured illumination For example the structured
  • illumination may provide an illumination pattern with stripes of light that are phase shifted at 0, 120, and 240 degrees.
  • the optical probe may then acquire images of a sample when the illumination pattern is at the phase shifted 0, 120, and 240 degree positions.
  • the imaging data provided by the optical probe 230 may then include multiple images.
  • An optical biopsy system, image processing system, or other system may then use the images captured under structured illumination to generate a composite image that blocks out of focus light.
  • other structured illumination patterns may be provided by the light source and additional images may be acquired by the optical probe.
  • Figure 3 depicts an example of a virtual 3D model 300 with images 310 generated from imaging data received from an optical probe.
  • the images may be generated by an optical biopsy system.
  • the images may be generated by the optical biopsy system 100 described with reference to Figure 1 above.
  • a biopsy computing system may generate the virtual 3D model 300 based on images and position and/or orientation data associated with the images.
  • the images may be retrieved from a data store that includes images and position and/or orientation data associated with the images.
  • the optical biopsy system may then generate a virtual 3D space that shows the images 310 in their corresponding coordinates and orientation planes.
  • the images 310 may be placed in the virtual 3D space as 2D images oriented in the direction they were captured from by an optical probe. In some embodiments, the images 310 may be placed in the virtual 3D space as a 2D image oriented in the direction of a perspective of a user.
  • the generated 3D space may be displayed to a medical practitioner or other user on a display screen.
  • the generated 3D model 300 may be provided as a 2D representation of the 3D space from a particular perspective.
  • the user may be able to manipulate the orientation of the 3D model.
  • a user interface may enable a user to change the position and orientation from which the 3D model is viewed.
  • a 2D plane 320 may be used to generate a 2D perspective view of the 3D model.
  • the 3D model 300 includes a number of images 310.
  • the images may be placed into 3D space based on the position and/or orientation determined by an optical biopsy system.
  • the images may be placed into the 3D model without determining positions based on overlap of the images.
  • the images 310 may be placed into a 3D model without flattening the images and aligning one image with a neighboring image to generate an overlapping mosaic of a number of different images.
  • the images 310 may be displayed following filtering, being otherwise processed, shown with various levels of opacity or colors, or the like.
  • the images 310 may be cropped. In some embodiments, only a subset of the images 310 may be displayed.
  • Figure 4 depicts an example of generating a 2D perspective view 450 from a 3D model 450 with images received from an optical probe.
  • the 3D model 410 may be generated in a similar manner as discussed above with respect to Figure 3.
  • a user interacting with the 3D model 410 may be able to manipulate the position and/or orientation of a cross section 420.
  • the user can select a 2D plane 420 through which the user can view a perspective view of the 2D model 450.
  • a biopsy computing system may then generate a 2D perspective view 450 of the 3D model 410 based on the selected plane.
  • the 3D model 410 includes images 411 , 412, 413, and 414.
  • the biopsy computing system may then determine the position of the images 41 1, 412, 413, and 414 from the perspective of a selected 2D plane 420.
  • the positions of the images 411 , 412, 413, and 414 may then be mapped from the 3D space to positions on a 2D plane.
  • the biopsy computing system may then generate a 2D perspective view 450 with the images 411 , 412, 413, and 414 shown on the 2D plane as images 451, 452, 453, and 454.
  • the images 411 , 412, 413, and 414 may be cropped, transformed, filtered, processed, displayed with various levels of opacity, or the like, for easier viewing.
  • the 2D plane 450 of the 3D model 410 may be generated to correspond to the perspective of a wide-field image capture device.
  • the images 451, 452, 453, and 453 may be superimposed onto a wide-field image to show the positions of the images relative to a wide-field image of a subject.
  • the user interface may include a user interface element that enables a user to capture an image from the optical probe.
  • the user interface element 530 is a "Capture" button that captures the current image.
  • an optical biopsy system may store the imaging data from the optical probe and a location associated with the imaging data. For example, as discussed above, position and/or orientation data of the optical probe may be generated from a tracker coupled to the optical probe
  • the operator may continue to take optical biopsies as different locations by moving the optical probe and capturing new imaging data.
  • Locations of previously stored biopsy images may be overlaid onto the wide-field images.
  • images of previous optical biopsies may be shown in a list of images 540, 542, 544.
  • the location of the images taken may be shown as an overlay in the wide-field image 510 as well.
  • an optical biopsy system may overlay indicators 550, 552, 554 corresponding to previously recorded biopsy images 540, 542, 544 respectively.
  • the indicators may have different colors, shapes, or other indications that associate the list of images with their associated indicators on the screen.
  • the location of the previously recorded biopsy images may be determined by tracking the wide-field image capture device.
  • the position of the image capture device may then be used to determine a perspective of the image capture device.
  • the perspective may then be used to generate a 2D view of capture images as discussed above with respect to Figure 4.
  • the optical biopsy system may overlay the generated 2D view on the wide-field image captured by the wide-field image capture device.
  • the user may select a user interface element 550 to terminate the session.
  • Figure 6 is an example user interface 600 showing images received from an optical probe mapped to a wide-field image of a subject. Similar to the example user interface shown in Figure 5, the user interface shown in Figure 6 includes a wide-field image 610 and an insert showing a current view of an optical probe 620. The example user interface 600 in Figure 6 is used in reference to an optical biopsy system for dermatological procedures.
  • an optical biopsy system may receive imaging data from an optical probe.
  • the optical biopsy system may be similar to that described with reference to Figure 1, and the image data may be received by the image processing system 114.
  • the optical probe may be a fiber optic bundle, or any other type of optical probe.
  • the imaging data may be raw imaging data from the optical probe.
  • the raw imaging data may include multiple signals from different components of an optical probe.
  • the raw imaging data may include imaging data taken at multiple points in time. For instance, the raw imaging data may include the output of sensors at different times that may then be used to generate a composite image.
  • the imaging data may be provided by an optical probe that blocks out of focus light to provide better clarity through optical sectioning.
  • the imaging data may include spectroscopic data or other photoelectric data from the tissue.
  • the optical biopsy system generates an image from the imaging data.
  • the image may be generated by an image processing system.
  • the image may be generated by applying optical sectioning to the imaging data.
  • an image processing system may use one or more components of imaging data to block the out of focus light received from the optical probe. This optical sectioning may improve the clarity of the generated images.
  • an image processing system may generate an image that is in a different color scheme, frequency of light, brightness, or having other
  • generating an image from the imaging data may comprise filtering one or more defects from the raw imaging data received by the optical biopsy system. In some embodiments, generating an image from the imaging data may comprise displaying the imaging data as waveforms or other numerical or graphical representations generated from the raw imaging data.
  • the optical biopsy system may also receive position and/or orientation data in block 730.
  • the position and/or orientation data may be received by a 3D mapping system as discussed above with reference to Figure 1.
  • the position and orientation data may include six degrees of position or orientation data.
  • the position and/or orientation data may be received from a tracking system that tracks a tracker or a plurality of trackers.
  • the optical biopsy system determines a position of an optical probe based on the position and/or orientation data.
  • the position of the optical probe may be determined based on a geometric transformation from the position and orientation of a tracker to a position of the optical probe.
  • the optical biopsy system may determine an orientation of the optical probe in addition to the position of the optical probe. For example, the optical biopsy system may determine that a direction that the optical probe is facing based on the yaw, pitch, and/or roll indicated in the position and/or orientation data.
  • the optical biopsy system may determine a position and/or orientation of imaging data that would be received by the optical probe.
  • the position of the imaging data may be slightly in front of the optical probe, and the imaging data may have an orientation in an opposite direction of the orientation of the optical probe.
  • the optical biopsy system may store the generated image with a determined position. For example, as the optical biopsy system receives a stream of imaging data in block 710, it may receive a corresponding stream of position and/or orientation data in block 730. The optical biopsy system may then correlate the image generated from the received imaging data with the position determined from the corresponding stream of position and/or orientation data. In some embodiments, the correlation may be determined by time- stamps associated with each stream of data. The optical biopsy system may then store the generated image and the associated position data in a data store.
  • the optical biopsy system may generate a 3D model with the generated image at the determined position.
  • the generated image may be placed in a virtual 3D space in the determined position.
  • the optical biopsy system may output the generated to a display at the determined position.
  • many images may be generated.
  • the optical biopsy system may then map each of the images to a position in the 3D space.
  • the images may then be displayed as a 3D model with each of the images in a determined position in the 3D space.
  • the optical biopsy system may provide the images as facing the perspective through which the model is viewed.
  • the images may be provided in an orientation that is associated with the image.
  • one or more of the images may overlap in 3D space.
  • the images may be flattened into 2D images.
  • the 2D images may then be compared to identify components of the images that are the same.
  • the 2D images may then be stitched together to generate a composite 2D image.
  • the overlapping images may be displayed together in the 3D space accounting for the overlap.
  • the composite 2D image may use image similarity metrics such as Normalized Cross Correlation, Mutual Information, or the like, in order to determine the optimal overlap between the components of such images.
  • Image registration algorithms may slightly reposition the raw images into the composite 2D image in order to maximize this overlap.
  • the overlapping regions between contiguous images may be blended together by an image processing system.
  • one image may be displayed atop another image (rather than blending).
  • perspective transforms such as shearing may be performed in order to maximize the image similarity metric in the composite 2D image.
  • the outer edges (or other portions) of the images may be displayed with varying levels of opacity to yield a higher quality composite image.
  • an imaging system may be employed to predict the data between images, thus filling in empty space between the images in the composite image.
  • Figure 8 depicts an example computer system 800 which can perform any one or more of the methods described herein for performing optical biopsies.
  • computer system 800 may correspond to optical biopsy system 100 or biopsy computing system 110 described with reference to Figure 1.
  • the computer system may be connected (e.g., networked) to other computer systems in a LAN, an intranet, an extranet, or the Internet.
  • the computer system 800 may operate in the capacity of a server in a client-server network environment.
  • the computer system 800 may be a personal computer (PC), a set-top box (STB), a server, a network router, switch or bridge, or any device capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that device.
  • PC personal computer
  • STB set-top box
  • server a server
  • network router switch or bridge
  • the term "computer” shall also be taken to include any collection of computers that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methods discussed herein.
  • the computer may reside in a geographically separated area (such as in a data center), providing only a set of display instructions to a second clinical device, such as a tablet or web browser.
  • the exemplary computer system 800 includes a processing device 802, a main memory 804 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM)), a static memory 806 (e.g., flash memory, static random access memory (SRAM)), and a secondary memory 816 (e.g., a data storage device), which communicate with each other via a bus 808.
  • main memory 804 e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM)
  • DRAM dynamic random access memory
  • SDRAM synchronous DRAM
  • static memory 806 e.g., flash memory, static random access memory (SRAM)
  • secondary memory 816 e.g., a data storage device
  • the processing device 802 represents one or more general-purpose processors such as a microprocessor, central processing unit, or the like.
  • the term "processing device” is used herein to refer to any combination of one or more integrated circuits and/or packages that include one or more processors (e.g., one or more processor cores). Therefore, the term processing device encompasses a single core CPU (computer processing unit), a multi-core CPU and a massively multi-core system that includes many interconnected integrated circuits, each of which may include multiple processor cores.
  • the processing device 802 may therefore include multiple processors.
  • the processing device 802 may include a complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long instruction word (VLIW) microprocessor, processor implementing other instruction sets, or processors implementing a combination of instruction sets.
  • the processing device 802 may also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, system on chip (SoC), or the like.
  • the computer system 800 may include a graphics processing unit 822, a video processing unit 828, and an audio processing unit 832. In some instances, the graphics processing unit 822 may be used to perform image processing instead of, or in conjunction with, the general-purpose CPU 802.
  • the computer system 800 may further include a network interface device 808.
  • the computer system 800 also may include a video display unit 810 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device 812 (e.g., a keyboard), a cursor control device 814 (e.g., a mouse), and a signal generation device 820 (e.g., a speaker).
  • the computer system 800 may include or be in communication with other input and output device.
  • the computer system 800 may receive input signals from a foot pedal, a haptic device, or the like.
  • the secondary memory 816 may include a machine-readable storage medium (or more specifically a computer-readable storage medium (CRM)) 824 on which is stored one or more sets of instructions embodying any one or more of the methodologies or functions described herein (e.g., 3D mapping system 112, image processing system 114, or the like).
  • the main memory 804 may also store, completely or partially, instructions 854 embodying any one or more of the methodologies or functions described herein (e.g., 3D mapping system 1 12, image processing system 114, or the like).
  • the processing device 802 may include instructions to implement 3D mapping system 1 12, image processing system 1 14, or the like during execution thereof by the computer system 800.
  • the main memory 804 and the processing device 802 may also include machine-readable storage media.
  • While the computer-readable storage medium 824 is shown in an exemplary implementation to be a single medium, the term “computer-readable storage medium” (or “computer-readable medium”) should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions.
  • the term “computer-readable storage medium” shall also be taken to include any medium other than a carrier wave that is capable of storing or encoding a set of instructions for execution by the machine that cause the machine to perform any one or more of the methodologies of the present disclosure.
  • the term “computer-readable storage medium” shall accordingly be taken to include, but not be limited to, non-transitory media such as solid-state memories, and optical and magnetic media.
  • modules, components and other features described herein can be implemented as discrete hardware components or integrated in the functionality of hardware components such as ASICS, FPGAs, DSPs, SoCs, or similar devices.
  • the modules can be implemented as firmware or functional circuitry within hardware devices.
  • the modules can be implemented in any combination of hardware devices and software components, or only in software.
  • Implementations of the present disclosure also relate to an apparatus for
  • This apparatus may be specially constructed for the required purposes, or it may comprise a general purpose computer system selectively programmed by a computer program stored in the computer system.
  • a computer program may be stored in a computer readable storage medium, such as, but not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs,
  • EEPROMs electrically erasable programmable read-only memory
  • magnetic disk storage media optical storage media
  • flash memory devices other type of machine-accessible storage media, or any type of media suitable for storing electronic instructions, each coupled to a computer system bus.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Pathology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biomedical Technology (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Biophysics (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)
  • Endoscopes (AREA)

Abstract

L'invention concerne un système d'imagerie comprenant une sonde optique pour générer des données d'imagerie, un système de suivi pour suivre la sonde optique, une source de lumière couplée fonctionnellement à la sonde optique, et un dispositif de traitement couplé en communication avec le dispositif de suivi. La source de lumière permet un sectionnement optique par le système d'imagerie. Le dispositif de traitement applique un sectionnement optique afin de générer une première image sur la base d'un sous-ensemble des données d'imagerie. Le dispositif de traitement détermine en outre des premières données de position et des premières données d'orientation associées à la sonde optique pendant que le sous-ensemble des données d'imagerie est généré et applique une transformation géométrique aux premières données de position et aux premières données d'orientation pour déterminer une première position dans un espace tridimensionnel virtuel. En outre, le dispositif de traitement génère un modèle tridimensionnel comprenant la première image placée à la première position dans l'espace tridimensionnel virtuel.
PCT/US2016/065014 2015-12-07 2016-12-05 Système de biopsie optique par imagerie suivie Ceased WO2017100139A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201562263809P 2015-12-07 2015-12-07
US62/263,809 2015-12-07

Publications (1)

Publication Number Publication Date
WO2017100139A1 true WO2017100139A1 (fr) 2017-06-15

Family

ID=59014088

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2016/065014 Ceased WO2017100139A1 (fr) 2015-12-07 2016-12-05 Système de biopsie optique par imagerie suivie

Country Status (1)

Country Link
WO (1) WO2017100139A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1611457B1 (fr) * 2003-03-27 2014-09-10 Koninklijke Philips N.V. Guidage de dispositifs medicaux invasifs par imagerie ultrasonique tridimensionnelle a tres large champ de vision
US20150045648A1 (en) * 2012-01-26 2015-02-12 Uc-Care Ltd. Integrated system for focused treatment and methods thereof
WO2015074018A1 (fr) * 2013-11-18 2015-05-21 Volcano Corporation Localisation d'un cathéter intraluminal

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1611457B1 (fr) * 2003-03-27 2014-09-10 Koninklijke Philips N.V. Guidage de dispositifs medicaux invasifs par imagerie ultrasonique tridimensionnelle a tres large champ de vision
US20150045648A1 (en) * 2012-01-26 2015-02-12 Uc-Care Ltd. Integrated system for focused treatment and methods thereof
WO2015074018A1 (fr) * 2013-11-18 2015-05-21 Volcano Corporation Localisation d'un cathéter intraluminal

Similar Documents

Publication Publication Date Title
US20110178395A1 (en) Imaging method and system
RU2535605C2 (ru) Повторная калибровка предварительно записанных изображений во время вмешательства с использованием игольчатого устройства
CN110638527A (zh) 基于光学相干层析增强现实的手术显微成像系统
KR20170044610A (ko) 조직 구조의 위치를 수술 중 확인하기 위한 방법 및 시스템
US8777846B2 (en) Endoscopic imaging device
Peterson et al. Feasibility of a video‐mosaicking approach to extend the field‐of‐view for reflectance confocal microscopy in the oral cavity in vivo
US20160334610A1 (en) Method and apparatus for microscopic imaging
JP2004202221A (ja) 検査システムおよび検査方法
US7162292B2 (en) Beam scanning probe system for surgery
CN101304683A (zh) 以渐进增加的分辨率观察和分析一个或多个生物样品的方法和用于该方法的设备
JP7495994B2 (ja) 医療光学系、データ処理システム、コンピュータプログラム、及び不揮発性コンピュータ可読記憶媒体
CN113670852A (zh) 检查装置及检查方法
JP2026507261A (ja) 手術顕微鏡の画像ナビゲーション方法及びそのシステム
US20230380682A1 (en) Devices, systems, and methods to measure corneal topography
WO2010061293A2 (fr) Système et procédé pour mesurer des objets vus à travers une caméra
CN119136756A (zh) 解剖结构测量
JP7653839B2 (ja) 内視鏡業務支援装置及び内視鏡業務支援システム並びに内視鏡業務支援装置の作動方法
KR20230125676A (ko) 개선된 수술용 내시경현미경의 영상 획득 시스템
JP2018029880A (ja) 医用画像処理装置および内視鏡装置
CN101779947B (zh) 一体化共聚焦显微硬质电子关节镜系统
WO2017100139A1 (fr) Système de biopsie optique par imagerie suivie
US20250040796A1 (en) Three-dimensional light-field microendoscopy with a grin lens array
CN216090895U (zh) 一种手术显微镜诊疗系统
CN118215968A (zh) 用于表征生物组织的感兴趣区域的系统
CN118799200B (zh) 一种荧光图像与视频图像的融合方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16873652

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16873652

Country of ref document: EP

Kind code of ref document: A1