WO2017210182A1 - Scape microscopy with phase modulating element and image reconstruction - Google Patents
Scape microscopy with phase modulating element and image reconstruction Download PDFInfo
- Publication number
- WO2017210182A1 WO2017210182A1 PCT/US2017/034984 US2017034984W WO2017210182A1 WO 2017210182 A1 WO2017210182 A1 WO 2017210182A1 US 2017034984 W US2017034984 W US 2017034984W WO 2017210182 A1 WO2017210182 A1 WO 2017210182A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- light
- scanning element
- objective
- excitation light
- optical components
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N21/645—Specially adapted constructive features of fluorimeters
- G01N21/6456—Spatial resolved fluorescence measurements; Imaging
- G01N21/6458—Fluorescence microscopy
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/0032—Optical details of illumination, e.g. light-sources, pinholes, beam splitters, slits, fibers
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/0036—Scanning details, e.g. scanning stages
- G02B21/0048—Scanning details, e.g. scanning stages scanning mirrors, e.g. rotating or galvanomirrors, MEMS mirrors
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/0052—Optical details of the image generation
- G02B21/0064—Optical details of the image generation multi-spectral or wavelength-selective arrangements, e.g. wavelength fan-out, chromatic profiling
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/0052—Optical details of the image generation
- G02B21/0072—Optical details of the image generation details concerning resolution or correction, including general design of CSOM objectives
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/0052—Optical details of the image generation
- G02B21/0076—Optical details of the image generation arrangements using fluorescence or luminescence
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/008—Details of detection or image processing, including general computer control
- G02B21/0084—Details of detection or image processing, including general computer control time-scale detection, e.g. strobed, ultra-fast, heterodyne detection
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/06—Means for illuminating specimens
- G02B21/08—Condensers
- G02B21/10—Condensers affording dark-field illumination
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
- G02B21/361—Optical details, e.g. image relay to the camera or image sensor
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
- G02B21/365—Control or image processing arrangements for digital or video microscopes
- G02B21/367—Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B26/00—Optical devices or arrangements for the control of light using movable or deformable optical elements
- G02B26/08—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
- G02B26/10—Scanning systems
- G02B26/105—Scanning systems with one or more pivoting mirrors or galvano-mirrors
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0075—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for altering, e.g. increasing, the depth of field or depth of focus
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/16—Microscopes adapted for ultraviolet illumination ; Fluorescence microscopes
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B2207/00—Coding scheme for general features or characteristics of optical elements and systems of subclass G02B, but not including elements and systems which would be classified in G02B6/00 and subgroups
- G02B2207/114—Two photon or multiphoton effect
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B26/00—Optical devices or arrangements for the control of light using movable or deformable optical elements
- G02B26/06—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the phase of light
Definitions
- SCAPE microscopy is a technique for high-speed 3D microscopy that uses swept, confocally aligned planar excitation.
- SCAPE imaging systems are disclosed in publication WO 2015/109323, which is incorporated herein by reference in its entirety.
- an oblique sheet of light e.g., laser light
- fluorescence from the sample is captured by a camera to produce an image.
- SCAPE can achieve resolutions rivaling techniques such as light-sheet, confocal, and two-photon microscopy, but can operate at much higher speeds.
- the excitation path includes the optical components starting at the light source and ending at the sample; and the detection path includes the optical components starting at the sample and ending at the camera.
- Some embodiments of SCAPE microscopy use two objective lenses in the detection arm placed at an angle with respect to one another for image rotation. But this configuration results in a net loss of light captured at the detector, which can adversely affect the numerical aperture, resolution, and light throughput of the imaging system.
- One aspect of the invention is directed to a first apparatus that comprises a first set of optical components having a proximal end and a distal end, the first set of optical components including an objective disposed at the distal end of the first set of optical components.
- This apparatus also comprises a second set of optical components having a proximal end and a distal end, the second set of optical components including a phase modulating element.
- This apparatus also comprises a scanning element that is disposed proximally with respect to the proximal end of the first set of optical components and proximally with respect to the proximal end of the second set of optical components.
- the scanning element is arranged to route excitation light arriving at the scanning element so that the excitation light will pass through the first set of optical components in a proximal to distal direction and project into a sample that is positioned distally beyond the objective.
- the excitation light is projected into the sample at an oblique angle, and the excitation light is projected into the sample at a position that varies depending on an orientation of the scanning element.
- the first set of optical components is arranged to route detection light from the sample in a distal to proximal direction back to the scanning element.
- the scanning element is also arranged to route the detection light so that the detection light will pass through the second set of optical components in a proximal to distal direction.
- This apparatus also comprises a camera optically positioned to capture images formed by the detection light that has passed through the second set of optical components.
- the phase modulating element induces a controlled aberration so as to homogenize point spread functions of points at, above, and below a focal plane when measured at the camera.
- the phase modulating element comprises a phase plate. In some embodiments of the first apparatus, the phase modulating element comprises a spatial light modulator. In some embodiments of the first apparatus, the phase modulating element comprises a deformable mirror.
- the objective has a pupil plane
- the phase modulating element is positioned at a plane that is conjugate to the pupil plane of the objective.
- Some of these embodiments further comprise an aperture stop positioned (a) adjacent to the phase modulating element and (b) proximally with respect to the phase modulating element.
- the detection light forms a stationary conjugate image plane between the proximal end and the distal end of the second set of optical components.
- the light detector array comprises a 2D image sensor, and the camera sequentially captures a plurality of images formed by the detection light that has passed through the second set of optical components, each of the plurality of images corresponding to a respective frame of image date.
- the apparatus further comprises a processor programmed to correct for aberration in the frames of image data.
- the processor corrects for aberration in the frames of image data by performing deconvolution of a point spread function.
- the point spread function is either a measured point spread function or a simulated point spread function.
- the light detector array comprises a linear image sensor.
- the scanning element comprises an x-y galvanometer.
- the excitation light has a wavelength such that excitation of a fluorophore in the sample requires near-simultaneous absorption of a plurality of photons.
- Some embodiments of the first apparatus further comprise a beam splitter disposed between the proximal end of the second set of optical components and the scanning element, and a source of the excitation light.
- the source of the excitation light is aimed so the excitation light is directed into the beam splitter, the beam splitter is arranged to route the excitation light towards the scanning element, and the beam splitter is arranged to route detection light arriving from the scanning element into the proximal end of the second set of optical components.
- the excitation light comprises a sheet of excitation light.
- This sheet of excitation light may be generated by at least one of (a) a cylindrical lens arranged to expand light from a light source into the sheet of excitation light; (b) an aspheric mirror arranged to expand light from a light source into the sheet of excitation light; (c) a spatial light modulator arranged to expand light from a light source into the sheet of excitation light; (d) a second scanning element arranged to expand light from a light source into the sheet of excitation light; and (e) an oscillating galvanometer mirror arranged to expand light from a light source into the sheet of excitation light.
- Some embodiments of the first apparatus further comprise a beam splitter disposed between the proximal end of the second set of optical components and the scanning element, and a source of the excitation light.
- the source of the excitation light is aimed so the excitation light is directed into the beam splitter
- the beam splitter is arranged to route the excitation light towards the scanning element
- the beam splitter is arranged to route detection light arriving from the scanning element into the proximal end of the second set of optical components
- the light detector array comprises a 2D image sensor
- the excitation light comprises a sheet of excitation light
- the camera sequentially captures a plurality of images formed by the detection light that has passed through the second set of optical components, each of the plurality of images corresponding to a respective frame of image date
- the apparatus further comprises a processor programmed to correct for aberration in the frames of image data.
- the processor corrects for aberration in the frames of image data by performing deconvolution of a point spread function.
- the objective has a pupil plane
- the phase modulating element is positioned at a plane that is conjugate to the pupil plane of the objective.
- the phase modulating element has a phase profile of
- ⁇ is a wavelength of light passing through the phase modulating element
- NA is a numerical aperture of the objective
- u and v are normalized pupil coordinates
- n is an index of refraction of an immersion medium underneath the objective
- dz is one half of a range of depths to be imaged.
- the phase modulating element has a phase profile of
- ⁇ is a wavelength of light passing through the phase modulating element
- NA is a numerical aperture of the objective
- u and v are normalized pupil coordinates
- n is an index of refraction of an immersion medium underneath the objective
- dz is one half of a range of depths to be imaged.
- Another aspect of the invention is directed to a second apparatus that comprises an objective and a plurality of optical elements disposed in a detection path arranged to receive light from the objective.
- the plurality of optical elements disposed in the detection path includes a scanning element.
- the scanning element is arranged to (a) route excitation light into the objective so as to generate a sweeping excitation beam through a forward image plane of the objective and (b) simultaneously route image light returning through the objective along the detection path to form a conjugate image.
- This apparatus further comprises a light detector array positioned to capture images of the conjugate image, and a phase modulating element disposed in the detection path between the scanning element and the light detector array.
- the phase modulating element extends a depth of field in the detection path.
- the phase modulating element extends the depth of field by inducing a controlled aberration so as to homogenize point spread functions of points at, above, and below a focal plane when measured at the light detector array.
- the aberration is corrected by an image processing algorithm. In some embodiments of the second apparatus, the aberration is corrected by deconvolution of a measured or simulated point spread function. In some embodiments of the second apparatus, the light detector array comprises a 2D image sensor. In some embodiments of the second apparatus, the light detector array comprises a linear image sensor.
- Another aspect of the invention is directed to a third apparatus that comprises a light source; a cylindrical lens or a scanner that expands light from the light source into a sheet of light; a beam splitter disposed in a path of the sheet of light; a scanning element disposed in a path of the sheet of light; a first telescope having a proximal end and a distal end, with an objective disposed at the distal end of the first telescope; and a second telescope having a proximal end and a distal end, the second telescope having an optical axis.
- the beam splitter routes the sheet of light towards the scanning element.
- the scanning element routes the sheet of light into the proximal end of the first telescope.
- the first telescope routes the sheet of light in a proximal to distal direction through the objective, accepts fluorescent light through the objective and routes the fluorescent light in a distal to proximal direction back to the scanning element.
- the scanning element routes the fluorescent light through the beam splitter and into the proximal end of the second telescope.
- the second telescope forms an image from the fluorescent light at a conjugate image plane.
- This apparatus further comprises a camera positioned on the same optical axis as the second telescope and configured to capture blurred images of the conjugate image plane; a phase modulating element disposed between the second telescope and the camera; and an image processor programmed to deblur images captured by the camera.
- the light source comprises a laser.
- the phase modulating element comprises a phase plate. In some embodiments of the third apparatus, the phase modulating element comprises at least one of a spatial light modulator and a deformable mirror. In some embodiments of the third apparatus, the image processor is programmed to deblur the images using a deconvolution algorithm.
- Another aspect of the invention is directed to a first method of imaging a sample.
- This method comprises projecting a sheet of excitation light into a sample, wherein the sheet of excitation light is projected into the sample at an oblique angle, and wherein the sheet of excitation light is projected into the sample at a position that varies depending on an orientation of a scanning element.
- This method also comprises routing detection light arriving from the sample back to the scanning element, and using the scanning element to reroute the detection light into an optical system that induces a controlled aberration so as to homogenize point spread functions of points at, above, and below a focal plane when measured at the camera.
- This method also comprises using the aberrated detection light to form a plurality of images at a plurality of times, each of the times corresponding to a different orientation of the scanning element, and capturing the plurality of images.
- Some embodiments of the first method further comprise correcting for the aberration in the plurality of images.
- the correcting step comprises performing deconvolution of a point spread function.
- the point spread function is either a measured point spread function or a simulated point spread function.
- the aberration is induced by a phase modulating element having a phase profile of
- ⁇ ( ⁇ v3 ⁇ 4 dz) - ⁇ 1/2 ⁇ * ⁇ 2 * ⁇ U ⁇ 1 ⁇ 2 2 ⁇ * z/n
- ⁇ is a wavelength of light passing through the phase modulating element
- NA is a numerical aperture of the objective
- u and v are normalized pupil coordinates
- n is an index of refraction of an immersion medium underneath the objective
- dz is one half of a range of depths to be imaged.
- FIG. 1 depicts an embodiment of a SCAPE microscope that uses a phase modulating element.
- FIG. 2A depicts a design for forming a sheet of light in which all positions on the sheet are illuminated simultaneously.
- FIG. 2B depicts a design for rapidly scanning a pencil beam of light so as to form a virtual sheet of light.
- FIGS. 3A and 3B depict, respectively, a phase function and a corresponding irradiance point spread function.
- FIGS. 4 A and 4B depict the sectioning effect of the oblique light sheet used in
- FIG. 5 depicts an embodiment of a SCAPE microscope that uses a reflective phase modulating element.
- FIG. 6 depicts an embodiment of a SCAPE microscope that uses a phase modulating element, a scanning element with two degrees of freedom, and a linear sensor.
- This application describes an alternative approach for implementing SCAPE microscopy that avoids many of the problems associated with those embodiments of SCAPE that use two objective lenses in the detection arm placed at an angle with respect to one another for image rotation.
- This alternative approach relies on a phase modulating element to extend the effective depth of field of the detection arm.
- FIG. 1 depicts a first embodiment of a SCAPE microscope that uses a phase modulating element (PME) 170.
- phase modulating elements that may be used in this embodiment include but are not limited to phase plates (e.g., cubic, logarithmic, etc.) and spatial light modulators (SLMs).
- a beam of light from a light source (e.g., laser 100 or an LED) having a wavelength within the excitation spectrum of a fluorophore of interest is passed through sheet forming optics 110.
- This sheet forming optics 110 converts the beam into a sheet of light and also dictates the geometric properties of the light that will excite the fluorescence in the sample.
- the sheet forming optics 1 10 shapes the light from the light source 100 into a true sheet of light (i.e., a sheet in which all positions on the sheet of light are illuminated simultaneously).
- a true sheet of light i.e., a sheet in which all positions on the sheet of light are illuminated simultaneously.
- a light source e.g., laser 100 or an LED
- emits light e.g., monochromatic, coherent, polarized light; or incoherent light
- an isotropic beam expander 201, 202 that uniformly magnifies the profile of the light source.
- the light passes through an anamorphic beam expander that magnifies beam from a circular shape to an elliptical shape.
- This anamorphic beam expander may be implemented, for example, using a pair of cylindrical lenses 21 1, 212 and/or matched prism pairs and/or aperture cropping with a slit, etc.
- the light is then reflected off of a sliding mirror 220 and is focused along the expanded dimension by a third cylindrical lens 230 and subsequently clipped along the nonexpanded dimension by slit shaped aperture 240.
- the slit width of this aperture 240 is used to control the sheet thickness under the objective (140, shown in FIG. 1), and by extension the depth of field of the microscope.
- the term "objective” is used in its generic sense to refer to the last optical component prior to the sample 145. It could be, for example, a simple lens element, a microscope-style objective, etc.
- the sliding mirror 220, the third cylindrical lens 230 and the aperture slit 240 may all be mechanically coupled to one another on a mount 250 that is capable of translating from side to side as a single unit, to provide adjustability.
- This adjustability may be motor-driven or manual and various means of adjustment having an equivalent effect will be evident to persons skilled in the relevant art.
- a wide variety of alternative approaches for generating a sheet of light in which all positions on the sheet are illuminated simultaneously will be readily apparent to persons skilled in the relevant arts.
- the sheet forming optics 1 10 shapes the light from the light source 100 by rapidly scanning a pencil beam of light so as to create a virtual sheet of light.
- the illumination at different positions within the sheet occurs at different instants of time.
- FIG. 2B depicts one approach for implementing this virtual sheet embodiment.
- a light source e.g., laser 100 emits light (e.g., monochromatic, coherent, polarized light), and that light is directed through an isotropic beam expander 201, 202 that uniformly magnifies the profile of the light source.
- the isotopically expanded beam is then directed onto a galvanometer mirror 260 that reorients the light and scans it into a sheet.
- the sheet of light is directed through a relay telescope 271, 272 and an aperture slit 280.
- the function of the relay telescope 271, 272 in this embodiment is to provide additional magnification if necessary and image the galvanometer mirror 260 onto a plane conjugate to the back focal plane of the objective (140, shown in FIG. 1).
- the function of the aperture slit 280 is similar to the FIG. 2A embodiment described above.
- the galvanometer mirror 260, the relay telescope 271, 272, and the aperture slit 280 may all be mechanically coupled to each other on a mount 250 that is capable of translating from side to side as a single unit to provide similar adjustability to the FIG. 2A embodiment.
- a wide variety of alternative approaches for using the scanning to generate a virtual sheet of light will be readily apparent to persons skilled in the relevant arts.
- the dichroic beam splitter 120 which separates the shorter wavelength light used for excitation of fluorescence from the longer wavelengths of light emitted by the fluorescent molecules.
- the dichroic beam splitter 120 was chosen such that the microscope was set up in long-pass configuration wherein fluorescence is transmitted through the dichroic beam splitter.
- the dichroic beam splitter 120 reflects the shorter wavelength excitation light towards the scanning element 125.
- the dichroic beam splitter 120 should be configured to reflect the longer wavelengths and pass the shorter wavelength.
- the scanning element 125 may be implemented using a planar scanning mirror with a single degree of freedom. But in alternative embodiments, a wide variety of alternative approaches for implementing scanning may be used, including but not limited to non-planer scanning mirrors, moving prisms, etc.
- the excitation light arriving from the dichroic beam splitter 120 at the scanning element 125 is reflected and scanned by the scanning element 125, after which it continues on towards a first set of optical components (e.g., scan-tube lens telescope 131, 132 and objective 140).
- the first set of optical components has a proximal end and a distal end, and the scanning element 125 routes the excitation light so that it will pass through the first set of optical components 131-140 in a proximal to distal direction.
- An objective 140 is disposed at the distal end of the first set of optical components . After passing through the first set of optical components 131 - 140, the excitation light will project into a sample 145 that is positioned distally beyond the objective 140. The excitation light is projected into the sample 145 at an oblique angle, and the position of the excitation light within the sample 145 varies depending on an orientation of the scanning element 125.
- the excitation light arrives at the back aperture of the objective 140 off-axis. As described previously by Bouchard et al (2015), this creates an oblique sheet of light 142 through the sample 145. Note that in those embodiments that employ an adjustable sliding mount (250 in FIGS. 2A and 2B), and the location of the adjustable sliding mount determines the angle of the oblique light sheet through the sample 145. It is also possible to adjust the location of the adjustable sliding mount 250 to accommodate for the various back aperture sizes of different objectives 140.
- the oblique sheet of light 142 will excite fluorescence in the sample 145.
- Fluorescent light collected from this sample is then collected by the same objective 140.
- the objective 140 and the remainder of the first set of optical components 131, 132 routes detection light from the sample in a distal to proximal direction back to the scanning element 125.
- a conjugate image of the detection light is formed between components 131 and 132. The location of this conjugate image plane changes in tandem with the laser light sweeping through the sample 145.
- the detection light arriving at the scanning element is rerouted by the scanning element 125 so that the detection light will pass through the second set of optical components 149- 170 in a proximal to distal direction.
- the second set of optical components includes an optional emission filter 149 followed by a telescope 151, 155, an aperture stop 160, and a PME 170.
- the detection light will pass through the dichroic beam splitter 120 before entering the proximal-most element the second set of optical components (i.e., lens 151).
- a conjugate image plane exists between lenses 151 and 155.
- the scanning element 125 descans the incoming detection light. As a result of the descanning, this conjugate image plane remains a stationary, despite the fact that the excitation light is scanning different sections of the sample 145 in object space.
- Light from the conjugate image plane then continues on through the distal lens
- the phase modulating element is positioned in the detection optical path at a position that is distal to the dichroic beam splitter 120.
- the PME 170 is positioned conjugate to the pupil plane of the objective 140.
- the sharpest image of the sample is formed when the camera is in focus with the primary lens's focal plane; the resolution at this plane is better than the resolution of the planes above and below and is characterized by the system's point spread function.
- the role of the PME 170 is to induce a controlled aberration into the system such that the point spread functions of points at, above, and below the focal plane (i.e., at different positions along the Z axis) when measured at the camera 190 are homogenized. This aberration degrades the point spread function of the microscope at the focal plane but retains the degraded point spread function over a wide range of depths. This, in effect, allows the microscope to image multiple planes (i.e.
- the introduction of the PME 170 thus improves the system's resolution and extends the system's depth of field.
- the general technique is known in other fields as wavefront encoding. However, here it is employed to derotate a conjugate oblique image plane.
- the PME is a cubic phase plate.
- the phase profile of the cubic phase plate may be described by the following equation: where x and y are normalized pupil coordinates and a is the tuning parameter generally set to
- phase function of the cubic phase plate can also be expressed as the sum of a set of Zernike polynomials as follows:
- a mathematical representation of a cubic phase plate was created using FRED Optical Engineering Software and simulated to be between two thin lenses.
- An irradiance spread function of an on-axis point at the focal plane of one of the lenses was determined.
- a phase function that was calculated with an alpha value of 36 and its corresponding irradiance point spread function are shown in FIGS. 3A and 3B, respectively.
- Phase plates are generally manufactured with tight tolerances (+/- 0.02 ⁇ ) and are meant to be used over a specific wavelength range. In some embodiments, this wavelength range would correspond to the emission spectra of whatever fluorescent indicator is being excited in the sample. Typical ranges for these wavelengths will be between 500 and 700 nm. In some embodiments, the system may be configured to swap in one of a plurality of different phase plates depending on the wavelength of the fluorophore that is being excited in the sample 145.
- the actual height of the phase plate may be calculated using the following equation:
- h is the height of the phase plate and n is the refractive index of the material composing the phase plate.
- phase plates specified to tighter wavelength ranges would also reduce the chromatic aberrations resulting from deviations from the design wavelength.
- Multi-color imaging can be performed in the conventional sense by placing an image splitter after the lens following the phase plate.
- phase plates By incorporating phase plates into a standard image splitter, each phase plate can be designed for a narrower spectral range.
- alternative equations may be implemented in the phase plate and/or a different class of phase plate (e.g., a logarithmic phase plate) may be used in place of the cubic phase plate described above.
- the desired phase profile can be etched into a substrate such as glass (e.g. using lithography).
- a different type of phase modulating element 170 may be used in place of the phase plate described above. Examples of such alternative phase modulating element include spatial light modulators and deformable mirrors. Note that some of these alternative phase modulating elements are programmable (e.g., SLMs). In those cases, any of the phase profiles described above can be imparted onto the programmable phase modulating element.
- a circular aperture may be placed in front of the PME 170 (i.e., on the proximal side of the PME) to modulate the numerical aperture of the microscope's detection-side optics.
- a long-pass emission filter 149 may be positioned at any appropriate location in the second set of optical components (e.g., between the dichroic beam splitter 120 and lens 151, as depicted in FIG. 1, or between lens 155 and the camera 190).
- the light After being modulated by the phase modulating element 170 (e.g., the phase plate), the light is captured using a camera 190.
- the camera is optically positioned to capture images formed by the detection light that has passed through the second set of optical components.
- the camera 190 is a single lens 192 that converges image light onto a 2D camera sensor 195.
- the illustrated single lens 192 can be replaced with a more complex zoom lens module to provide variable magnification to the system.
- image intensifiers and image splitters may also be added for improved SNR and spectral separation respectively.
- the sensor 195 of the camera 190 captures a plurality of frames, each of which represents a 2D image of the fluorescence emanating from a plane within the sample 145 at a different point in time.
- the conjugate image plane that appears between lenses 151 and 155 is not perpendicular to the axial axis of the second set of optical components, when that image reaches the sensor 195 of the camera 190, most portions of the images captured by the sensor 195 will be blurred (i.e., out of focus).
- the PME 170 alleviates this blurred condition to a significant extent by extending the depth of field in the detection path.
- This blurred condition can be improved dramatically by processing the 2D images that have been captured by the camera's sensor 195. More specifically, each of the images that are captured by the camera's sensor 195 is stored in memory (e.g., RAM, a hard drive, or an SSD), and those images can be processed by a suitably programmed processor 199 to correct for aberration in each of the images. In some embodiments, this processing corrects for aberration by performing deconvolution of a measured point spread function. In other embodiments, this this processing corrects for aberration by performing deconvolution of a simulated point spread function. Alternative approaches for correcting the aberration may also be used. The aberration-corrected images are then stored in memory.
- memory e.g., RAM, a hard drive, or an SSD
- transverse point spread function (X-Y) of a system with a cubic phase plate is depth dependent.
- image restoration may be performed by storing an empirically determined transverse point spread function at each depth in a look-up table and performing
- deconvolution with either a Weiner filter or iterative deconvolution.
- Any of a wide variety of techniques for implementing deconvolution may be used, including but not limited to linear autoregressive, ARMA, and Weiner Filters, Lucy-Richardson deconvolution - Bayesian maximum likelihood expectation maximization solution, Landweber deconvolution, Wavelet based deconvolution, and Maximum Entropy Based Deconvolution.
- the approaches disclosed in the following three publications, each of which is incorporated herein by reference, may also be used: S. Quirin at al., "Calcium imaging of neural circuits with extended depth-of-field Lightsheet Microscopy," Opt. Lett. 41, 855 (2016); 4. D. C. Andreo, et al., “Master in Photonics Fast Image Restoration in Light-Sheet Fluorescence Microscopy with Extended Depth of Field Using GPUs,” (2015); O. Olarte, et al., "Decoupled
- FIGS. 4 A and 4B show the sectioning effect of the oblique light sheet 142 used in SCAPE microscopy.
- the sheet of light 142 scans back and forth within the sample 145 along the X direction and the camera captures an image of the sheet in the Y-Z' dimension.
- the sheet in SCAPE provides sectioning along the X dimension in a manner that is similar to the way that the light sheet in SPIM light sheet systems provides sectioning in the Z dimension.
- the secondary lobes along the X dimension of the transverse (X-Y) point spread function are projected onto the Z' axis. Because the transverse profile changes slightly over the range of depths, the image acquired by the camera frame will have a spatially varying PSF.
- uniformity may be assumed over a part or all of the image. If complete uniformity is assumed, a single point spread function may be acquired/calculated within the center of the Y-Z' image and the aberrated image may be restored via non-blind deconvolution techniques.
- a point spread function may be acquired/calculated at each depth and more complex deconvolution techniques with spatially varying kernels will be used. Examples of suitable techniques for dealing with variations in the point spread function can be found in Lauer, Tod, "Deconvolution with a Spatially-Variant PSF," Astronomical Telescopes and
- phase masks such as radial quartic and logarithmic functions induce no image artifacts but face a steeper tradeoff between contrast and depth of field, as explained in M. Demenikov et al., "Image Artifacts in Hybrid Imaging Systems with a Cubic Phase Mask.,” Opt. Express 18, 8207-8212 (2010). Regardless, the implementation of such phase masks in SCAPE may follow an analogous process and would also use non-blind deconvolution techniques for image restoration.
- a sheet of excitation light 142 is projected into the sample 145 at a corresponding position, and a 2D image of the fluorescence from the illuminated plane 142 within the sample 145 is captured by the camera's sensor 195. More specifically, fluorescence from the illuminated plane 142 will be imaged onto a conjugate image plane between lenses 131 and 132, be descanned by the scanning element 125, pass through the dichroic beam splitter 120 and be imaged onto a stationary conjugate image plane between lenses 151 and 155. This conjugate image plane is stationary due to the descanning performed by the scanning element 125. This conjugate image will then be aberrated by the PME 170 as described above and be imaged onto the sensor 195 of the camera 190.
- Each frame will then be sequentially read into memory and deconvolved by the processor 199 with an empirically determined system point-spread-function to obtain a reconstructed (i.e. deblurred) image of the oblique section.
- movement of the scanning element 125 causes the sheet of excitation light 142 to move to a different location within the sample.
- the result is a stack of deblurred 2-D frames that can be assembled into a 3D volume representing a target volume within the sample 145.
- FIG. 5 depicts an alternative embodiment that is similar to the FIG. 1 embodiment, except that a reflective phase modulating element 370 is used in place of the transmissive phase modulating element (i.e., PME 170 in the FIG. 1 embodiment).
- Examples of reflective phase modulating elements 370 that may be used in these embodiments include reflective spatial light modulators (SLM) and deformable mirrors (DM), each of which is used to modulate the phase profile of the detected fluorescence.
- the reflective phase modulating element 370 is preferably positioned at a plane conjugate to the pupil plane of the primary objective 140.
- this FIG. 5 embodiment operates in the same way as the FIG. 1 embodiment described above.
- FIG. 6 depicts another alternative embodiment that has many similarities to the FIG. 1 embodiment, but also has a number of significant differences. More specifically, the FIG. 6 embodiment scans a pencil-shaped beam through the sample using a scanning element 625 with two degrees of freedom (e.g., an X-Y galvanometer scanner). In addition, the FIG. 6 embodiment uses a linear sensor 695 (i.e. a one-dimensional sensor) instead of the 2D sensor described above in connection with the FIG. 1 embodiment.
- a linear sensor 695 i.e. a one-dimensional sensor
- the FIG. 6 embodiment includes a light source (e.g., laser 100) that emits light (e.g., monochromatic, coherent, polarized light), and this light is directed through beam- shaping optics 610 to improve the characteristics of the pencil-shaped beam.
- the beam shaping optics 610 comprises an isotropic beam expander to uniformly magnify the profile of the light source, followed by a circular aperture that clips off the periphery of the beam. The size of this aperture can be used to control the beam thickness under the objective 140, and by extension the depth of field of the microscope.
- this beam splitter 120 which separates the shorter wavelength light used for excitation of fluorescence from the longer wavelengths of light emitted by the fluorescent molecules.
- the operation of this beam splitter 120 is similar to the operation described above in connection with FIG. 1.
- the beam splitter 120 directs the excitation light towards the scanning element 625.
- the scanning element 625 may be implemented using a planar scanning mirror with two degrees of freedom such as an X-Y galvanometer. But in alternative embodiments, a wide variety of alternative approaches for implementing 2-axis scanning may be used, including but not limited to non-planer scanning mirrors, moving prisms, etc.
- the excitation light arriving from the dichroic beam splitter 120 at the scanning element 625 is reflected and scanned by the scanning element 625, after which it continues on through the first set of optical components 131- 140. The operation of these components 131- 140 is similar to the operation described above in connection with FIG. 1.
- the excitation light After passing through the first set of optical components 131-140, the excitation light will project into a sample 145 that is positioned distally beyond the objective 140.
- the excitation light is projected into the sample 145 at an oblique angle, and the position of the excitation light within the sample 145 varies depending on an orientation of the scanning element 625.
- the operation of this FIG. 6 embodiment to this point is similar to the operation of the FIG. 1 embodiment discussed above, with one significant exception: instead of projecting a true or virtual sheet of excitation light into the sample 145 at an oblique angle, the FIG. 6 embodiment projects a pencil-shaped beam of excitation light 642 into the sample 145 at an oblique angle.
- the pencil-shaped beam of excitation light in this FIG. 6 embodiment can scan through the same volume within the sample that is scanned by the sheet in the FIG. 1 embodiment.
- the oblique beam of light 642 will excite fluorescence in the sample 145.
- Fluorescent light collected from this sample is then collected by the same objective 140.
- the objective 140 and the remainder of the first set of optical components 131, 132 routes detection light from the sample in a distal to proximal direction back to the scanning element 625.
- a conjugate image of the detection light is formed between components 131 and 132. The location of this conjugate image plane changes in tandem with the laser light sweeping through the sample 145.
- the detection light arriving at the scanning element is rerouted by the scanning element 625 so that the detection light will pass through the second set of optical components 149- 170 in a proximal to distal direction.
- the second set of optical components includes an optional emission filter 149 followed by a telescope 151, 155, an aperture stop 160, and a PME 170.
- the detection light will pass through the dichroic beam splitter 120 before entering the proximal-most element the second set of optical components (i.e., lens 151).
- a conjugate image plane exists between lenses 151 and 155.
- the scanning element 625 descans the incoming detection light. As a result of the descanning, this conjugate image plane remains a stationary, despite the fact that the excitation light is scanning different sections of the sample 145 in object space.
- phase modulating element is positioned in the detection optical path at a position that is distal to the dichroic beam splitter
- the PME 170 is positioned conjugate to the pupil plane of the objective 140.
- the implementation of the PME 170 in this FIG. 6 embodiment follows the same principles discussed above in connection with the FIG. 1 embodiment.
- a circular aperture 160 and/or a long -pass emission filter 149 may be included, as described above in connection with the FIG. 1 embodiment.
- the light After being modulated by the PME 170, the light is captured using a camera
- the camera is optically positioned to capture images formed by the detection light that has passed through the second set of optical components.
- the camera 690 in this embodiment is a single lens 692 that converges image light onto a one-dimensional sensor 695 (i.e., a linear array).
- fluorescence from the illuminated line 642 will be imaged onto a conjugate image plane between lenses 131 and 132, be descanned by the scanning element 625, pass through the dichroic beam splitter 120 and be imaged onto a stationary conjugate image plane between lenses 151 and 155.
- This conjugate image plane is stationary due to the descanning performed by the scanning element 625.
- This conjugate image will then be aberrated by the PME 170 as described above and be imaged onto the sensor 695 of the camera 690 as a frame of data.
- Each frame of data corresponds to a single line of the fluorescence emanating from a line 642 within the sample 145 at a different point in time.
- This line will be composed of a plurality of pixels.
- the size of each pixel in the linear sensor 695 is between 5 and 10 microns. In alternative embodiments, larger pixels are used, in which case there will be less resolution in the depth direction. In other alternative embodiments, the pixel size remains the same (i.e., 5-10 microns), but adjacent pixels are binned together (e.g., in groups of 4 or 8) to provide increased sensitivity, albeit with decreased resolution in the depth direction.
- a plurality of these 3D volumes may be captured at a plurality of different points in time (e.g., every tenth of a second) in order to depict changes in the volume being imaged (i.e., the volume within the sample) over time.
- deconvolution is not necessary. But in some embodiments, ID deconvolution may optionally be implemented to deblur each of the ID images before those images are assembled into a 3D volume. In alternative embodiments, linear filtering techniques may be used instead of ID deconvolution.
- the PME 170 of the FIG. 6 embodiment may be replaced with a reflective PME, similar to the way that the PME 170 in the FIG. 1 embodiment was replaced with a reflective PME 370 in the FIG. 3 embodiment.
- the optical components 149-170 in the detection arm and the camera 190/690 of the FIG. 1, 5, and 6 embodiments share a single optical axis.
- This configuration advantageously avoids the large losses that are inherent in many prior art SCAPE designs that use an off-axis camera.
- One complication with this design is that when the camera 190/690 and the detection arm share a single optical axis, it introduces on aberration into the image. But this aberration is corrected algorithmically as explained above.
- the conjugate image plane is oblique with respect to the optical axis
- the camera 190/690 aligned along the optical axis, as opposed to an oblique angle
- the PME 170 will aberrate the resulting image, in addition to extending the depth of field of the image reaching the camera 190/690.
- These aberrations are corrected algorithmically in the processor 199, e.g., by deconvolution of a measured or simulated point spread function as explained above.
- FIG. 1, 5, and 6 embodiments described above can advantageously simplify alignment, improve resolution, improve light throughput, and enhance the effective detection numerical aperture (with respect to the two objective lens embodiments). Superior signal-to-noise ratio can be achieved due to these improvements.
- This improved S/N can be leveraged in many ways, including imaging at faster rates with shorter exposure times in order to reach higher volumetric frame rates and imaging with lower laser intensity reducing photobleaching and damage caused to the samples.
- This configuration may also be able to reduce the cost of the system by using a microscope-style objective with a lower NA (e.g., NA - 0.5) and reducing the total footprint. It may be applied towards endoscopic
- the fluorescence may be excited at very low probabilities with lasers in the NIR-IR spectrum (700- 1600 nm). As a result, the fluorescence to be collected is significantly weaker than for single photon applications (with lasers from 300-700 nm) and could benefit from the improved light collection efficiency of the FIG. 1, 5, and 6 embodiments.
- 5, and 6 embodiments are all governed by the characteristics of a single primary objective 140.
- this lens e.g., using a turret
- it becomes possible to provide the modularity of a benchtop microscope e.g., with a number of standardized magnifications, resolutions, and depths of field
- this may involve switching in a different PME 170 (e.g., a different phase plate) in tandem with the switching of the objective, or altering the phase profile on a programmable phase modulating element (e.g., a deformable mirror or SLM) to account for the changed objective or an alternative modification.
- a programmable phase modulating element e.g., a deformable mirror or SLM
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
- Microscoopes, Condenser (AREA)
Abstract
A scanning element routes a sheet or beam of excitation light through a first set of optical components and into a sample at an oblique angle. The position of the excitation light within the sample varies depending on the orientation of the scanning element. Fluorophores within the sample emit fluorescent detection light. The first set of optical components route the detection light back to the scanning element, and the scanning element routes the detection light through a second set of optical components and into a camera. The second set of optical components includes a phase modulating element that induces a controlled aberration so as to homogenize point spread functions of points at, above, and below a focal plane when measured at the camera. In some embodiments, the images captured by the camera are processed to correct for aberration by performing deconvolution of a point spread function.
Description
SCAPE MICROSCOPY WITH PHASE MODULATING ELEMENT AND IMAGE RECONSTRUCTION
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This Application claims the benefit of US Provisional Application 62/343, 112, filed May 30, 2016, which is incorporated herein by reference in its entirety.
STATEMENT REGARDING FEDERALLY-SPONSORED RESEARCH
[0002] This invention is made with government support from the NIH under Grant
Nos. 5U01NS094296-01, 1R01NS076628, 1R01NS063226, and R21NS053684; NSF grants CBET-0954796 and IGERT 0801530; and DOD grant MURI W91 INF- 12-1-0594. The Government has certain rights in the invention.
BACKGROUND
[0003] SCAPE microscopy is a technique for high-speed 3D microscopy that uses swept, confocally aligned planar excitation. Some examples of SCAPE imaging systems are disclosed in publication WO 2015/109323, which is incorporated herein by reference in its entirety. In SCAPE, an oblique sheet of light (e.g., laser light) is swept through the sample, and fluorescence from the sample is captured by a camera to produce an image. SCAPE can achieve resolutions rivaling techniques such as light-sheet, confocal, and two-photon microscopy, but can operate at much higher speeds.
[0004] In SCAPE systems, the excitation path includes the optical components starting at the light source and ending at the sample; and the detection path includes the optical components starting at the sample and ending at the camera. Some embodiments of SCAPE microscopy use two objective lenses in the detection arm placed at an angle with respect to one another for image rotation. But this configuration results in a net loss of light captured at the detector, which can adversely affect the numerical aperture, resolution, and light throughput of the imaging system.
SUMMARY OF THE INVENTION
[0005] One aspect of the invention is directed to a first apparatus that comprises a first set of optical components having a proximal end and a distal end, the first set of optical components including an objective disposed at the distal end of the first set of optical
components. This apparatus also comprises a second set of optical components having a proximal end and a distal end, the second set of optical components including a phase modulating element. This apparatus also comprises a scanning element that is disposed proximally with respect to the proximal end of the first set of optical components and proximally with respect to the proximal end of the second set of optical components. The scanning element is arranged to route excitation light arriving at the scanning element so that the excitation light will pass through the first set of optical components in a proximal to distal direction and project into a sample that is positioned distally beyond the objective. The excitation light is projected into the sample at an oblique angle, and the excitation light is projected into the sample at a position that varies depending on an orientation of the scanning element. The first set of optical components is arranged to route detection light from the sample in a distal to proximal direction back to the scanning element. The scanning element is also arranged to route the detection light so that the detection light will pass through the second set of optical components in a proximal to distal direction. This apparatus also comprises a camera optically positioned to capture images formed by the detection light that has passed through the second set of optical components. The phase modulating element induces a controlled aberration so as to homogenize point spread functions of points at, above, and below a focal plane when measured at the camera.
[0006] In some embodiments of the first apparatus, the phase modulating element comprises a phase plate. In some embodiments of the first apparatus, the phase modulating element comprises a spatial light modulator. In some embodiments of the first apparatus, the phase modulating element comprises a deformable mirror.
[0007] In some embodiments of the first apparatus, the objective has a pupil plane, and the phase modulating element is positioned at a plane that is conjugate to the pupil plane of the objective. Some of these embodiments further comprise an aperture stop positioned (a) adjacent to the phase modulating element and (b) proximally with respect to the phase modulating element.
[0008] In some embodiments of the first apparatus, the detection light forms a stationary conjugate image plane between the proximal end and the distal end of the second set of optical components.
[0009] In some embodiments of the first apparatus, the light detector array comprises a 2D image sensor, and the camera sequentially captures a plurality of images formed by the detection light that has passed through the second set of optical components, each of the plurality of images corresponding to a respective frame of image date. In these embodiments,
the apparatus further comprises a processor programmed to correct for aberration in the frames of image data. In some of these embodiments, the processor corrects for aberration in the frames of image data by performing deconvolution of a point spread function. In some of these embodiments, the point spread function is either a measured point spread function or a simulated point spread function.
[0010] In some embodiments of the first apparatus, the light detector array comprises a linear image sensor. In some of these embodiments, the scanning element comprises an x-y galvanometer. In some of these embodiments, the excitation light has a wavelength such that excitation of a fluorophore in the sample requires near-simultaneous absorption of a plurality of photons.
[0011] Some embodiments of the first apparatus further comprise a beam splitter disposed between the proximal end of the second set of optical components and the scanning element, and a source of the excitation light. In these embodiments, the source of the excitation light is aimed so the excitation light is directed into the beam splitter, the beam splitter is arranged to route the excitation light towards the scanning element, and the beam splitter is arranged to route detection light arriving from the scanning element into the proximal end of the second set of optical components.
[0012] In some embodiments of the first apparatus, the excitation light comprises a sheet of excitation light. This sheet of excitation light may be generated by at least one of (a) a cylindrical lens arranged to expand light from a light source into the sheet of excitation light; (b) an aspheric mirror arranged to expand light from a light source into the sheet of excitation light; (c) a spatial light modulator arranged to expand light from a light source into the sheet of excitation light; (d) a second scanning element arranged to expand light from a light source into the sheet of excitation light; and (e) an oscillating galvanometer mirror arranged to expand light from a light source into the sheet of excitation light.
[0013] Some embodiments of the first apparatus further comprise a beam splitter disposed between the proximal end of the second set of optical components and the scanning element, and a source of the excitation light. In these embodiments, the source of the excitation light is aimed so the excitation light is directed into the beam splitter, the beam splitter is arranged to route the excitation light towards the scanning element, the beam splitter is arranged to route detection light arriving from the scanning element into the proximal end of the second set of optical components, the light detector array comprises a 2D image sensor, the excitation light comprises a sheet of excitation light, the camera sequentially captures a plurality of images formed by the detection light that has passed
through the second set of optical components, each of the plurality of images corresponding to a respective frame of image date, and the apparatus further comprises a processor programmed to correct for aberration in the frames of image data. In some of these embodiments, the processor corrects for aberration in the frames of image data by performing deconvolution of a point spread function. In some of these embodiments, the objective has a pupil plane, and the phase modulating element is positioned at a plane that is conjugate to the pupil plane of the objective. In some of these embodiments, the phase modulating element has a phase profile of
where x and y are normalized pupil coordinates and a is a tuning parameter set to |Ψ/2|, where
Ψ(υ¾ ¾¾ dz> = -ί1/2λ)* .Α2*{ ÷ ¾2)*dz/n
where λ is a wavelength of light passing through the phase modulating element, NA is a numerical aperture of the objective, u and v are normalized pupil coordinates, n is an index of refraction of an immersion medium underneath the objective, and dz is one half of a range of depths to be imaged.
[0014] In some embodiments of the first apparatus, the phase modulating element has a phase profile of
where x and y are normalized pupil coordinates and a is a tuning parameter set to |Ψ/2|, where
Ψ{υ¾ ¾ dz) = -{ ! 2/v)*:i iA"*:( li " ÷ v )*di/Vi
where λ is a wavelength of light passing through the phase modulating element, NA is a numerical aperture of the objective, u and v are normalized pupil coordinates, n is an index of refraction of an immersion medium underneath the objective, and dz is one half of a range of depths to be imaged.
[0015] Another aspect of the invention is directed to a second apparatus that comprises an objective and a plurality of optical elements disposed in a detection path arranged to receive light from the objective. The plurality of optical elements disposed in the detection path includes a scanning element. The scanning element is arranged to (a) route
excitation light into the objective so as to generate a sweeping excitation beam through a forward image plane of the objective and (b) simultaneously route image light returning through the objective along the detection path to form a conjugate image. This apparatus further comprises a light detector array positioned to capture images of the conjugate image, and a phase modulating element disposed in the detection path between the scanning element and the light detector array. The phase modulating element extends a depth of field in the detection path.
[0016] In some embodiments of the second apparatus, the phase modulating element extends the depth of field by inducing a controlled aberration so as to homogenize point spread functions of points at, above, and below a focal plane when measured at the light detector array.
[0017] In some embodiments of the second apparatus, the aberration is corrected by an image processing algorithm. In some embodiments of the second apparatus, the aberration is corrected by deconvolution of a measured or simulated point spread function. In some embodiments of the second apparatus, the light detector array comprises a 2D image sensor. In some embodiments of the second apparatus, the light detector array comprises a linear image sensor.
[0018] Another aspect of the invention is directed to a third apparatus that comprises a light source; a cylindrical lens or a scanner that expands light from the light source into a sheet of light; a beam splitter disposed in a path of the sheet of light; a scanning element disposed in a path of the sheet of light; a first telescope having a proximal end and a distal end, with an objective disposed at the distal end of the first telescope; and a second telescope having a proximal end and a distal end, the second telescope having an optical axis. The beam splitter routes the sheet of light towards the scanning element. The scanning element routes the sheet of light into the proximal end of the first telescope. The first telescope routes the sheet of light in a proximal to distal direction through the objective, accepts fluorescent light through the objective and routes the fluorescent light in a distal to proximal direction back to the scanning element. The scanning element routes the fluorescent light through the beam splitter and into the proximal end of the second telescope. The second telescope forms an image from the fluorescent light at a conjugate image plane. This apparatus further comprises a camera positioned on the same optical axis as the second telescope and configured to capture blurred images of the conjugate image plane; a phase modulating element disposed between the second telescope and the camera; and an image processor programmed to deblur images captured by the camera.
[0019] In some embodiments of the third apparatus, the light source comprises a laser. In some embodiments of the third apparatus, the phase modulating element comprises a phase plate. In some embodiments of the third apparatus, the phase modulating element comprises at least one of a spatial light modulator and a deformable mirror. In some embodiments of the third apparatus, the image processor is programmed to deblur the images using a deconvolution algorithm.
[0020] Another aspect of the invention is directed to a first method of imaging a sample. This method comprises projecting a sheet of excitation light into a sample, wherein the sheet of excitation light is projected into the sample at an oblique angle, and wherein the sheet of excitation light is projected into the sample at a position that varies depending on an orientation of a scanning element. This method also comprises routing detection light arriving from the sample back to the scanning element, and using the scanning element to reroute the detection light into an optical system that induces a controlled aberration so as to homogenize point spread functions of points at, above, and below a focal plane when measured at the camera. This method also comprises using the aberrated detection light to form a plurality of images at a plurality of times, each of the times corresponding to a different orientation of the scanning element, and capturing the plurality of images.
[0021] Some embodiments of the first method further comprise correcting for the aberration in the plurality of images.
[0022] In some embodiments of the first method, the correcting step comprises performing deconvolution of a point spread function. In some embodiments of the first method, the point spread function is either a measured point spread function or a simulated point spread function.
[0023] In some embodiments of the first method, the aberration is induced by a phase modulating element having a phase profile of
where x and y are normalized pupil coordinates and a is a tuning parameter set to |Ψ/2|, where
Ψ(ϋζ v¾ dz) = -<1/2λ}*ΜΑ2*{ U ÷ ½2}* z/n
where λ is a wavelength of light passing through the phase modulating element, NA is a numerical aperture of the objective, u and v are normalized pupil coordinates, n is an index of
refraction of an immersion medium underneath the objective, and dz is one half of a range of depths to be imaged.
BRIEF DESCRIPTION OF THE DRAWINGS
[0024] FIG. 1 depicts an embodiment of a SCAPE microscope that uses a phase modulating element.
[0025] FIG. 2A depicts a design for forming a sheet of light in which all positions on the sheet are illuminated simultaneously.
[0026] FIG. 2B depicts a design for rapidly scanning a pencil beam of light so as to form a virtual sheet of light.
[0027] FIGS. 3A and 3B depict, respectively, a phase function and a corresponding irradiance point spread function.
[0028] FIGS. 4 A and 4B depict the sectioning effect of the oblique light sheet used in
SCAPE microscopy.
[0029] FIG. 5 depicts an embodiment of a SCAPE microscope that uses a reflective phase modulating element.
[0030] FIG. 6 depicts an embodiment of a SCAPE microscope that uses a phase modulating element, a scanning element with two degrees of freedom, and a linear sensor.
[0031] Embodiments will hereinafter be described in detail below with reference to the accompanying drawings, wherein like reference numerals represent like elements.
DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0032] This application describes an alternative approach for implementing SCAPE microscopy that avoids many of the problems associated with those embodiments of SCAPE that use two objective lenses in the detection arm placed at an angle with respect to one another for image rotation. This alternative approach relies on a phase modulating element to extend the effective depth of field of the detection arm.
[0033] FIG. 1 depicts a first embodiment of a SCAPE microscope that uses a phase modulating element (PME) 170. Examples of phase modulating elements that may be used in this embodiment include but are not limited to phase plates (e.g., cubic, logarithmic, etc.) and spatial light modulators (SLMs).
[0034] A beam of light from a light source (e.g., laser 100 or an LED) having a wavelength within the excitation spectrum of a fluorophore of interest is passed through sheet forming optics 110. This sheet forming optics 110 converts the beam into a sheet of light and
also dictates the geometric properties of the light that will excite the fluorescence in the sample.
[0035] In some embodiments, the sheet forming optics 1 10 shapes the light from the light source 100 into a true sheet of light (i.e., a sheet in which all positions on the sheet of light are illuminated simultaneously). One approach for accomplishing this is depicted in FIG. 2A. A light source (e.g., laser 100 or an LED) emits light (e.g., monochromatic, coherent, polarized light; or incoherent light), and that light is directed through an isotropic beam expander 201, 202 that uniformly magnifies the profile of the light source. From there, the light passes through an anamorphic beam expander that magnifies beam from a circular shape to an elliptical shape. This anamorphic beam expander may be implemented, for example, using a pair of cylindrical lenses 21 1, 212 and/or matched prism pairs and/or aperture cropping with a slit, etc.
[0036] The light is then reflected off of a sliding mirror 220 and is focused along the expanded dimension by a third cylindrical lens 230 and subsequently clipped along the nonexpanded dimension by slit shaped aperture 240. The slit width of this aperture 240 is used to control the sheet thickness under the objective (140, shown in FIG. 1), and by extension the depth of field of the microscope. Note that as used herein, the term "objective" is used in its generic sense to refer to the last optical component prior to the sample 145. It could be, for example, a simple lens element, a microscope-style objective, etc. Optionally, the sliding mirror 220, the third cylindrical lens 230 and the aperture slit 240 may all be mechanically coupled to one another on a mount 250 that is capable of translating from side to side as a single unit, to provide adjustability. This adjustability may be motor-driven or manual and various means of adjustment having an equivalent effect will be evident to persons skilled in the relevant art. A wide variety of alternative approaches for generating a sheet of light in which all positions on the sheet are illuminated simultaneously will be readily apparent to persons skilled in the relevant arts.
[0037] In alternative embodiments, the sheet forming optics 1 10 shapes the light from the light source 100 by rapidly scanning a pencil beam of light so as to create a virtual sheet of light. In these embodiments, the illumination at different positions within the sheet occurs at different instants of time. These embodiments create an oblique line beyond the objective that is then scanned back and forth across the lateral field of view to form the virtual sheet.
[0038] FIG. 2B depicts one approach for implementing this virtual sheet embodiment.
A light source (e.g., laser 100) emits light (e.g., monochromatic, coherent, polarized light), and that light is directed through an isotropic beam expander 201, 202 that uniformly
magnifies the profile of the light source. The isotopically expanded beam is then directed onto a galvanometer mirror 260 that reorients the light and scans it into a sheet.
Subsequently, the sheet of light is directed through a relay telescope 271, 272 and an aperture slit 280. The function of the relay telescope 271, 272 in this embodiment is to provide additional magnification if necessary and image the galvanometer mirror 260 onto a plane conjugate to the back focal plane of the objective (140, shown in FIG. 1). And the function of the aperture slit 280 is similar to the FIG. 2A embodiment described above. Optionally, the galvanometer mirror 260, the relay telescope 271, 272, and the aperture slit 280 may all be mechanically coupled to each other on a mount 250 that is capable of translating from side to side as a single unit to provide similar adjustability to the FIG. 2A embodiment. A wide variety of alternative approaches for using the scanning to generate a virtual sheet of light will be readily apparent to persons skilled in the relevant arts.
[0039] Returning to FIG. 1, light exiting the sheet forming optics 110 enters the dichroic beam splitter 120, which separates the shorter wavelength light used for excitation of fluorescence from the longer wavelengths of light emitted by the fluorescent molecules. In this embodiment, the dichroic beam splitter 120 was chosen such that the microscope was set up in long-pass configuration wherein fluorescence is transmitted through the dichroic beam splitter. The dichroic beam splitter 120 reflects the shorter wavelength excitation light towards the scanning element 125.
[0040] In alternative embodiments (e.g., two-photon embodiments, where the excitation light has a longer wavelength than the fluorescence) the dichroic beam splitter 120 should be configured to reflect the longer wavelengths and pass the shorter wavelength. In some alternative embodiments, it is also possible to use a shortpass dichroic beam splitter wherein the microscope is set up in short-pass configuration. Collected fluorescence from the sample would be reflected off of the dichroic beam splitter 120 in this case.
[0041] In the FIG. 1 embodiment, the scanning element 125 may be implemented using a planar scanning mirror with a single degree of freedom. But in alternative embodiments, a wide variety of alternative approaches for implementing scanning may be used, including but not limited to non-planer scanning mirrors, moving prisms, etc. The excitation light arriving from the dichroic beam splitter 120 at the scanning element 125 is reflected and scanned by the scanning element 125, after which it continues on towards a first set of optical components (e.g., scan-tube lens telescope 131, 132 and objective 140). The first set of optical components has a proximal end and a distal end, and the scanning element
125 routes the excitation light so that it will pass through the first set of optical components 131-140 in a proximal to distal direction.
[0042] An objective 140 is disposed at the distal end of the first set of optical components . After passing through the first set of optical components 131 - 140, the excitation light will project into a sample 145 that is positioned distally beyond the objective 140. The excitation light is projected into the sample 145 at an oblique angle, and the position of the excitation light within the sample 145 varies depending on an orientation of the scanning element 125.
[0043] In some embodiments, the excitation light arrives at the back aperture of the objective 140 off-axis. As described previously by Bouchard et al (2015), this creates an oblique sheet of light 142 through the sample 145. Note that in those embodiments that employ an adjustable sliding mount (250 in FIGS. 2A and 2B), and the location of the adjustable sliding mount determines the angle of the oblique light sheet through the sample 145. It is also possible to adjust the location of the adjustable sliding mount 250 to accommodate for the various back aperture sizes of different objectives 140.
[0044] The oblique sheet of light 142 will excite fluorescence in the sample 145.
Fluorescent light collected from this sample is then collected by the same objective 140. The objective 140 and the remainder of the first set of optical components 131, 132 routes detection light from the sample in a distal to proximal direction back to the scanning element 125. A conjugate image of the detection light is formed between components 131 and 132. The location of this conjugate image plane changes in tandem with the laser light sweeping through the sample 145.
[0045] The detection light arriving at the scanning element is rerouted by the scanning element 125 so that the detection light will pass through the second set of optical components 149- 170 in a proximal to distal direction. In the illustrated embodiment, the second set of optical components includes an optional emission filter 149 followed by a telescope 151, 155, an aperture stop 160, and a PME 170. Note that in the FIG. 1 embodiment, the detection light will pass through the dichroic beam splitter 120 before entering the proximal-most element the second set of optical components (i.e., lens 151). A conjugate image plane exists between lenses 151 and 155. Notably, the scanning element 125 descans the incoming detection light. As a result of the descanning, this conjugate image plane remains a stationary, despite the fact that the excitation light is scanning different sections of the sample 145 in object space.
[0046] Light from the conjugate image plane then continues on through the distal lens
155 of the telescope and into the PME 170. The phase modulating element is positioned in the detection optical path at a position that is distal to the dichroic beam splitter 120. In some preferred embodiments, the PME 170 is positioned conjugate to the pupil plane of the objective 140.
[0047] In a traditional microscope, the sharpest image of the sample is formed when the camera is in focus with the primary lens's focal plane; the resolution at this plane is better than the resolution of the planes above and below and is characterized by the system's point spread function. The role of the PME 170 is to induce a controlled aberration into the system such that the point spread functions of points at, above, and below the focal plane (i.e., at different positions along the Z axis) when measured at the camera 190 are homogenized. This aberration degrades the point spread function of the microscope at the focal plane but retains the degraded point spread function over a wide range of depths. This, in effect, allows the microscope to image multiple planes (i.e. multiple depths) with the same - albeit reduced - in-plane resolution. The introduction of the PME 170 thus improves the system's resolution and extends the system's depth of field. The general technique is known in other fields as wavefront encoding. However, here it is employed to derotate a conjugate oblique image plane.
[0048] In some preferred embodiments, the PME is a cubic phase plate. In these embodiments, the phase profile of the cubic phase plate may be described by the following equation:
where x and y are normalized pupil coordinates and a is the tuning parameter generally set to |Ψ/2|, where
where λ is the wavelength of light passing through the phase plate, NA is the numerical aperture of the objective 140, u and v are the normalized pupil coordinates, n is the index of refraction of an immersion medium underneath the objective, and dz is one half the depth of focus (i.e. the range of depths to be imaged). Additional information regarding characteristics of the phase plate can be found in S. Quirin et al, "Instantaneous Three-Dimensional Sensing Using Spatial Light Modulator Illumination with Extended Depth of Field Imaging," Opt. Express 21, 16007-16021 (2013).
[0049] The phase function of the cubic phase plate can also be expressed as the sum of a set of Zernike polynomials as follows:
[0050] In one example, a mathematical representation of a cubic phase plate was created using FRED Optical Engineering Software and simulated to be between two thin lenses. An irradiance spread function of an on-axis point at the focal plane of one of the lenses was determined. A phase function that was calculated with an alpha value of 36 and its corresponding irradiance point spread function are shown in FIGS. 3A and 3B, respectively.
[0051] Phase plates are generally manufactured with tight tolerances (+/- 0.02λ) and are meant to be used over a specific wavelength range. In some embodiments, this wavelength range would correspond to the emission spectra of whatever fluorescent indicator is being excited in the sample. Typical ranges for these wavelengths will be between 500 and 700 nm. In some embodiments, the system may be configured to swap in one of a plurality of different phase plates depending on the wavelength of the fluorophore that is being excited in the sample 145.
[0052] As explained in S. Quirin et al, "Simultaneous Imaging of Neural Activity in
Three Dimensions.," Front. Neural Circuits 8, 29 (2014), the actual height of the phase plate may be calculated using the following equation:
Where h is the height of the phase plate and n is the refractive index of the material composing the phase plate.
[0053] Designing phase plates specified to tighter wavelength ranges would also reduce the chromatic aberrations resulting from deviations from the design wavelength. Multi-color imaging can be performed in the conventional sense by placing an image splitter after the lens following the phase plate. Alternatively, it is also possible to perform spectral separation of emitted fluorescence prior to the phase plate. By incorporating phase plates into a standard image splitter, each phase plate can be designed for a narrower spectral range.
[0054] In alternative embodiments, alternative equations may be implemented in the phase plate and/or a different class of phase plate (e.g., a logarithmic phase plate) may be used in place of the cubic phase plate described above. In some embodiments, the desired phase profile can be etched into a substrate such as glass (e.g. using lithography).
[0055] In some embodiments, a different type of phase modulating element 170 may be used in place of the phase plate described above. Examples of such alternative phase modulating element include spatial light modulators and deformable mirrors. Note that some of these alternative phase modulating elements are programmable (e.g., SLMs). In those cases, any of the phase profiles described above can be imparted onto the programmable phase modulating element.
[0056] Optionally, a circular aperture (aperture stop 160) may be placed in front of the PME 170 (i.e., on the proximal side of the PME) to modulate the numerical aperture of the microscope's detection-side optics. Optionally, a long-pass emission filter 149 may be positioned at any appropriate location in the second set of optical components (e.g., between the dichroic beam splitter 120 and lens 151, as depicted in FIG. 1, or between lens 155 and the camera 190).
[0057] After being modulated by the phase modulating element 170 (e.g., the phase plate), the light is captured using a camera 190. The camera is optically positioned to capture images formed by the detection light that has passed through the second set of optical components. In its simplest form, the camera 190 is a single lens 192 that converges image light onto a 2D camera sensor 195. In alternative embodiments, the illustrated single lens 192 can be replaced with a more complex zoom lens module to provide variable magnification to the system. Optionally, image intensifiers and image splitters may also be added for improved SNR and spectral separation respectively.
[0058] The sensor 195 of the camera 190 captures a plurality of frames, each of which represents a 2D image of the fluorescence emanating from a plane within the sample 145 at a different point in time.
[0059] Because the conjugate image plane that appears between lenses 151 and 155 is not perpendicular to the axial axis of the second set of optical components, when that image reaches the sensor 195 of the camera 190, most portions of the images captured by the sensor 195 will be blurred (i.e., out of focus). The PME 170 alleviates this blurred condition to a significant extent by extending the depth of field in the detection path.
[0060] This blurred condition can be improved dramatically by processing the 2D images that have been captured by the camera's sensor 195. More specifically, each of the images that are captured by the camera's sensor 195 is stored in memory (e.g., RAM, a hard drive, or an SSD), and those images can be processed by a suitably programmed processor 199 to correct for aberration in each of the images. In some embodiments, this processing corrects for aberration by performing deconvolution of a measured point spread function. In
other embodiments, this this processing corrects for aberration by performing deconvolution of a simulated point spread function. Alternative approaches for correcting the aberration may also be used. The aberration-corrected images are then stored in memory.
[0061] One suitable approach for processing each of the 2D images that has been captured by the camera's sensor 195 is as follows: Initially, we note that the transverse point spread function (X-Y) of a system with a cubic phase plate is depth dependent. In some embodiments, image restoration may be performed by storing an empirically determined transverse point spread function at each depth in a look-up table and performing
deconvolution with either a Weiner filter or iterative deconvolution. Any of a wide variety of techniques for implementing deconvolution may be used, including but not limited to linear autoregressive, ARMA, and Weiner Filters, Lucy-Richardson deconvolution - Bayesian maximum likelihood expectation maximization solution, Landweber deconvolution, Wavelet based deconvolution, and Maximum Entropy Based Deconvolution. The approaches disclosed in the following three publications, each of which is incorporated herein by reference, may also be used: S. Quirin at al., "Calcium imaging of neural circuits with extended depth-of-field Lightsheet Microscopy," Opt. Lett. 41, 855 (2016); 4. D. C. Andreo, et al., "Master in Photonics Fast Image Restoration in Light-Sheet Fluorescence Microscopy with Extended Depth of Field Using GPUs," (2015); O. Olarte, et al., "Decoupled
Illumination Detection in Light Sheet Microscopy for Fast Volumetric Imaging," Optica 2, 702-705 (2015).
[0062] FIGS. 4 A and 4B show the sectioning effect of the oblique light sheet 142 used in SCAPE microscopy. The sheet of light 142 scans back and forth within the sample 145 along the X direction and the camera captures an image of the sheet in the Y-Z' dimension. The sheet in SCAPE provides sectioning along the X dimension in a manner that is similar to the way that the light sheet in SPIM light sheet systems provides sectioning in the Z dimension. The secondary lobes along the X dimension of the transverse (X-Y) point spread function are projected onto the Z' axis. Because the transverse profile changes slightly over the range of depths, the image acquired by the camera frame will have a spatially varying PSF. Depending upon the design of the PME however, uniformity may be assumed over a part or all of the image. If complete uniformity is assumed, a single point spread function may be acquired/calculated within the center of the Y-Z' image and the aberrated image may be restored via non-blind deconvolution techniques.
[0063] If the point spread function varies dramatically over the range of depths (Ζ'), then a point spread function may be acquired/calculated at each depth and more complex
deconvolution techniques with spatially varying kernels will be used. Examples of suitable techniques for dealing with variations in the point spread function can be found in Lauer, Tod, "Deconvolution with a Spatially-Variant PSF," Astronomical Telescopes and
Instrumentation, International Society for Optics and Photonics (2002), which is incorporated herein by reference. Alternative techniques that will be apparent to persons skilled in the relevant arts may also be used.
[0064] Note that the quadratic shift observed in cubic phase plates are a property of higher order anti-symmetric phase masks. Rotationally symmetric phase masks, such as radial quartic and logarithmic functions induce no image artifacts but face a steeper tradeoff between contrast and depth of field, as explained in M. Demenikov et al., "Image Artifacts in Hybrid Imaging Systems with a Cubic Phase Mask.," Opt. Express 18, 8207-8212 (2010). Regardless, the implementation of such phase masks in SCAPE may follow an analogous process and would also use non-blind deconvolution techniques for image restoration.
[0065] Returning to FIG. 1, for any given position of the scanning element 125, a sheet of excitation light 142 is projected into the sample 145 at a corresponding position, and a 2D image of the fluorescence from the illuminated plane 142 within the sample 145 is captured by the camera's sensor 195. More specifically, fluorescence from the illuminated plane 142 will be imaged onto a conjugate image plane between lenses 131 and 132, be descanned by the scanning element 125, pass through the dichroic beam splitter 120 and be imaged onto a stationary conjugate image plane between lenses 151 and 155. This conjugate image plane is stationary due to the descanning performed by the scanning element 125. This conjugate image will then be aberrated by the PME 170 as described above and be imaged onto the sensor 195 of the camera 190.
[0066] Each frame will then be sequentially read into memory and deconvolved by the processor 199 with an empirically determined system point-spread-function to obtain a reconstructed (i.e. deblurred) image of the oblique section. As explained above, movement of the scanning element 125 causes the sheet of excitation light 142 to move to a different location within the sample. As a result, if a 2D image is captured corresponding to each of a plurality of positions of the scanning element 125, and if each of those images is reconstructed/deblurred as explained above, the result is a stack of deblurred 2-D frames that can be assembled into a 3D volume representing a target volume within the sample 145.
[0067] Optionally, a plurality of these 3D volumes may be captured at a plurality of different points in time (e.g., every tenth of a second) in order to depict changes in the volume being imaged (i.e., the volume within the sample) over time.
[0068] FIG. 5 depicts an alternative embodiment that is similar to the FIG. 1 embodiment, except that a reflective phase modulating element 370 is used in place of the transmissive phase modulating element (i.e., PME 170 in the FIG. 1 embodiment). Examples of reflective phase modulating elements 370 that may be used in these embodiments include reflective spatial light modulators (SLM) and deformable mirrors (DM), each of which is used to modulate the phase profile of the detected fluorescence. In these embodiments, the reflective phase modulating element 370 is preferably positioned at a plane conjugate to the pupil plane of the primary objective 140. In other regards, this FIG. 5 embodiment operates in the same way as the FIG. 1 embodiment described above.
[0069] FIG. 6 depicts another alternative embodiment that has many similarities to the FIG. 1 embodiment, but also has a number of significant differences. More specifically, the FIG. 6 embodiment scans a pencil-shaped beam through the sample using a scanning element 625 with two degrees of freedom (e.g., an X-Y galvanometer scanner). In addition, the FIG. 6 embodiment uses a linear sensor 695 (i.e. a one-dimensional sensor) instead of the 2D sensor described above in connection with the FIG. 1 embodiment.
[0070] The FIG. 6 embodiment includes a light source (e.g., laser 100) that emits light (e.g., monochromatic, coherent, polarized light), and this light is directed through beam- shaping optics 610 to improve the characteristics of the pencil-shaped beam. In some embodiments, the beam shaping optics 610 comprises an isotropic beam expander to uniformly magnify the profile of the light source, followed by a circular aperture that clips off the periphery of the beam. The size of this aperture can be used to control the beam thickness under the objective 140, and by extension the depth of field of the microscope.
[0071] Light exiting the beam-shaping optics 610 enters the dichroic beam splitter
120, which separates the shorter wavelength light used for excitation of fluorescence from the longer wavelengths of light emitted by the fluorescent molecules. The operation of this beam splitter 120 is similar to the operation described above in connection with FIG. 1. The beam splitter 120 directs the excitation light towards the scanning element 625.
[0072] The scanning element 625 may be implemented using a planar scanning mirror with two degrees of freedom such as an X-Y galvanometer. But in alternative embodiments, a wide variety of alternative approaches for implementing 2-axis scanning may be used, including but not limited to non-planer scanning mirrors, moving prisms, etc. The excitation light arriving from the dichroic beam splitter 120 at the scanning element 625 is reflected and scanned by the scanning element 625, after which it continues on through the
first set of optical components 131- 140. The operation of these components 131- 140 is similar to the operation described above in connection with FIG. 1.
[0073] After passing through the first set of optical components 131-140, the excitation light will project into a sample 145 that is positioned distally beyond the objective 140. The excitation light is projected into the sample 145 at an oblique angle, and the position of the excitation light within the sample 145 varies depending on an orientation of the scanning element 625. Note that the operation of this FIG. 6 embodiment to this point is similar to the operation of the FIG. 1 embodiment discussed above, with one significant exception: instead of projecting a true or virtual sheet of excitation light into the sample 145 at an oblique angle, the FIG. 6 embodiment projects a pencil-shaped beam of excitation light 642 into the sample 145 at an oblique angle. By controlling the position of the scanning element 625 in both the X and Y direction, the pencil-shaped beam of excitation light in this FIG. 6 embodiment can scan through the same volume within the sample that is scanned by the sheet in the FIG. 1 embodiment.
[0074] The oblique beam of light 642 will excite fluorescence in the sample 145.
Fluorescent light collected from this sample is then collected by the same objective 140. The objective 140 and the remainder of the first set of optical components 131, 132 routes detection light from the sample in a distal to proximal direction back to the scanning element 625. A conjugate image of the detection light is formed between components 131 and 132. The location of this conjugate image plane changes in tandem with the laser light sweeping through the sample 145.
[0075] The detection light arriving at the scanning element is rerouted by the scanning element 625 so that the detection light will pass through the second set of optical components 149- 170 in a proximal to distal direction. In the illustrated embodiment, the second set of optical components includes an optional emission filter 149 followed by a telescope 151, 155, an aperture stop 160, and a PME 170. Note that in the FIG. 6 embodiment, the detection light will pass through the dichroic beam splitter 120 before entering the proximal-most element the second set of optical components (i.e., lens 151). A conjugate image plane exists between lenses 151 and 155. Notably, the scanning element 625 descans the incoming detection light. As a result of the descanning, this conjugate image plane remains a stationary, despite the fact that the excitation light is scanning different sections of the sample 145 in object space.
[0076] Light from the conjugate image plane continues through the distal lens 155 of the telescope and into the phase modulating element 170. The phase modulating element is
positioned in the detection optical path at a position that is distal to the dichroic beam splitter
120. In some preferred embodiments, the PME 170 is positioned conjugate to the pupil plane of the objective 140. The implementation of the PME 170 in this FIG. 6 embodiment follows the same principles discussed above in connection with the FIG. 1 embodiment.
[0077] Optionally, a circular aperture 160 and/or a long -pass emission filter 149 may be included, as described above in connection with the FIG. 1 embodiment.
[0078] After being modulated by the PME 170, the light is captured using a camera
690. The camera is optically positioned to capture images formed by the detection light that has passed through the second set of optical components. In its simplest form, the camera 690 in this embodiment is a single lens 692 that converges image light onto a one-dimensional sensor 695 (i.e., a linear array).
[0079] At any given position of the scanning element 625, a beam of excitation light
642 is projected into the sample 145 at a corresponding position, and a linear image of the fluorescence from the illuminated line 642 within the sample 145 is captured by the camera's sensor 695. More specifically, fluorescence from the illuminated line 642 will be imaged onto a conjugate image plane between lenses 131 and 132, be descanned by the scanning element 625, pass through the dichroic beam splitter 120 and be imaged onto a stationary conjugate image plane between lenses 151 and 155. This conjugate image plane is stationary due to the descanning performed by the scanning element 625. This conjugate image will then be aberrated by the PME 170 as described above and be imaged onto the sensor 695 of the camera 690 as a frame of data.
[0080] Each frame of data corresponds to a single line of the fluorescence emanating from a line 642 within the sample 145 at a different point in time. This line will be composed of a plurality of pixels. In some embodiments, the size of each pixel in the linear sensor 695 is between 5 and 10 microns. In alternative embodiments, larger pixels are used, in which case there will be less resolution in the depth direction. In other alternative embodiments, the pixel size remains the same (i.e., 5-10 microns), but adjacent pixels are binned together (e.g., in groups of 4 or 8) to provide increased sensitivity, albeit with decreased resolution in the depth direction.
[0081] Because movement of the scanning element 625 causes any the beam of excitation light 642 to move to a different location within the sample, if a linear image is captured corresponding to each of a plurality of positions of the scanning element 625, the result is a bundle of frames (similar to a bundle of toothpicks) that can be assembled by
processor and associated memory 199 into a 3D volume representing a target volume within the sample 145.
[0082] Optionally, a plurality of these 3D volumes may be captured at a plurality of different points in time (e.g., every tenth of a second) in order to depict changes in the volume being imaged (i.e., the volume within the sample) over time.
[0083] In this FIG. 6 embodiment, deconvolution is not necessary. But in some embodiments, ID deconvolution may optionally be implemented to deblur each of the ID images before those images are assembled into a 3D volume. In alternative embodiments, linear filtering techniques may be used instead of ID deconvolution.
[0084] In alternative embodiments, the PME 170 of the FIG. 6 embodiment may be replaced with a reflective PME, similar to the way that the PME 170 in the FIG. 1 embodiment was replaced with a reflective PME 370 in the FIG. 3 embodiment.
[0085] In contrast to those embodiments of SCAPE that implement image rotation using two objective lenses in the detection arm placed at an angle with respect to one another, the optical components 149-170 in the detection arm and the camera 190/690 of the FIG. 1, 5, and 6 embodiments share a single optical axis. As a result, almost all the light that passes through the telescope 151-155 in the detection arm will continue onward towards the camera 190/690. This configuration advantageously avoids the large losses that are inherent in many prior art SCAPE designs that use an off-axis camera. One complication with this design is that when the camera 190/690 and the detection arm share a single optical axis, it introduces on aberration into the image. But this aberration is corrected algorithmically as explained above.
[0086] In this case, although the conjugate image plane is oblique with respect to the optical axis, the camera 190/690 (aligned along the optical axis, as opposed to an oblique angle) will image the oblique conjugate image plane as a unidimensionally compressed projection onto the surface of its sensor 195/695. The PME 170 will aberrate the resulting image, in addition to extending the depth of field of the image reaching the camera 190/690. These aberrations are corrected algorithmically in the processor 199, e.g., by deconvolution of a measured or simulated point spread function as explained above.
[0087] The FIG. 1, 5, and 6 embodiments described above can advantageously simplify alignment, improve resolution, improve light throughput, and enhance the effective detection numerical aperture (with respect to the two objective lens embodiments). Superior signal-to-noise ratio can be achieved due to these improvements. This improved S/N can be leveraged in many ways, including imaging at faster rates with shorter exposure times in
order to reach higher volumetric frame rates and imaging with lower laser intensity reducing photobleaching and damage caused to the samples. This configuration may also be able to reduce the cost of the system by using a microscope-style objective with a lower NA (e.g., NA - 0.5) and reducing the total footprint. It may be applied towards endoscopic
implementations of SCAPE for rapid in vivo pathology in human tissue or for light-starved, multi-photon implementations of SCAPE. In these implementations, the fluorescence may be excited at very low probabilities with lasers in the NIR-IR spectrum (700- 1600 nm). As a result, the fluorescence to be collected is significantly weaker than for single photon applications (with lasers from 300-700 nm) and could benefit from the improved light collection efficiency of the FIG. 1, 5, and 6 embodiments.
[0088] Notably, the numerical aperture, resolution, and light throughput of the FIG. 1,
5, and 6 embodiments are all governed by the characteristics of a single primary objective 140. By changing this lens (e.g., using a turret), it becomes possible to provide the modularity of a benchtop microscope (e.g., with a number of standardized magnifications, resolutions, and depths of field) while performing fast, volumetric imaging over a large field of view. In some embodiments, this may involve switching in a different PME 170 (e.g., a different phase plate) in tandem with the switching of the objective, or altering the phase profile on a programmable phase modulating element (e.g., a deformable mirror or SLM) to account for the changed objective or an alternative modification. Finally, in comparison to Olarte's, Tomer's and Quirin's decoupled illumination/detection scheme in light sheet microscopy, the SCAPE system's lack of orthogonal beam paths extends the application of the technologies to large samples such as rodent cortices.
[0089] While the present invention has been disclosed with reference to certain embodiments, numerous modifications, alterations, and changes to the described
embodiments are possible without departing from the sphere and scope of the present invention, as defined in the appended claims. Accordingly, it is intended that the present invention not be limited to the described embodiments, but that it has the full scope defined by the language of the following claims, and equivalents thereof.
Claims
1. An imaging apparatus comprising:
a first set of optical components having a proximal end and a distal end, wherein the first set of optical components includes an objective disposed at the distal end of the first set of optical components;
a second set of optical components having a proximal end and a distal end, wherein the second set of optical components includes a phase modulating element;
a scanning element that is disposed proximally with respect to the proximal end of the first set of optical components and proximally with respect to the proximal end of the second set of optical components,
wherein the scanning element is arranged to route excitation light arriving at the scanning element so that the excitation light will pass through the first set of optical components in a proximal to distal direction and project into a sample that is positioned distally beyond the objective, wherein the excitation light is projected into the sample at an oblique angle, and wherein the excitation light is projected into the sample at a position that varies depending on an orientation of the scanning element,
wherein the first set of optical components is arranged to route detection light from the sample in a distal to proximal direction back to the scanning element, and wherein the scanning element is also arranged to route the detection light so that the detection light will pass through the second set of optical components in a proximal to distal direction; and
a camera optically positioned to capture images formed by the detection light that has passed through the second set of optical components,
wherein the phase modulating element induces a controlled aberration so as to homogenize point spread functions of points at, above, and below a focal plane when measured at the camera.
2. The apparatus of claim 1, wherein the phase modulating element comprises a phase plate.
3. The apparatus of claim 1, wherein the phase modulating element comprises a spatial light modulator.
4. The apparatus of claim 1, wherein the phase modulating element comprises a
deformable mirror.
5. The apparatus of claim 1, wherein the objective has a pupil plane, and the phase modulating element is positioned at a plane that is conjugate to the pupil plane of the objective.
6. The apparatus of claim 5, further comprising an aperture stop positioned (a) adjacent to the phase modulating element and (b) proximally with respect to the phase modulating element.
7. The apparatus of claim 1, wherein the detection light forms a stationary conjugate image plane between the proximal end and the distal end of the second set of optical components.
8. The apparatus of claim 1, wherein the light detector array comprises a 2D image sensor,
wherein the camera sequentially captures a plurality of images formed by the detection light that has passed through the second set of optical components, each of the plurality of images corresponding to a respective frame of image date, and
wherein the apparatus further comprises a processor programmed to correct for aberration in the frames of image data.
9. The apparatus of claim 8, wherein the processor corrects for aberration in the frames of image data by performing deconvolution of a point spread function.
10. The apparatus of claim 9, wherein the point spread function is either a measured point spread function or a simulated point spread function.
1 1. The apparatus of claim 1, wherein the light detector array comprises a linear image sensor.
12. The apparatus of claim 1 1, wherein the scanning element comprises an x-y
galvanometer.
13. The apparatus of claim 1 1, wherein the excitation light has a wavelength such that excitation of a fluorophore in the sample requires near-simultaneous absorption of a plurality of photons.
14. The apparatus of claim 1, further comprising:
a beam splitter disposed between the proximal end of the second set of optical components and the scanning element; and
a source of the excitation light,
wherein the source of the excitation light is aimed so the excitation light is directed into the beam splitter,
wherein the beam splitter is arranged to route the excitation light towards the scanning element, and
wherein the beam splitter is arranged to route detection light arriving from the scanning element into the proximal end of the second set of optical components.
15. The apparatus of claim 1, wherein the excitation light comprises a sheet of excitation light.
16. The apparatus of claim 15, wherein the sheet of excitation light is generated by at least one of (a) a cylindrical lens arranged to expand light from a light source into the sheet of excitation light; (b) an aspheric mirror arranged to expand light from a light source into the sheet of excitation light; (c) a spatial light modulator arranged to expand light from a light source into the sheet of excitation light; (d) a second scanning element arranged to expand light from a light source into the sheet of excitation light; and (e) an oscillating galvanometer mirror arranged to expand light from a light source into the sheet of excitation light.
17. The apparatus of claim 1, further comprising:
a beam splitter disposed between the proximal end of the second set of optical components and the scanning element; and
a source of the excitation light,
wherein the source of the excitation light is aimed so the excitation light is directed into the beam splitter,
wherein the beam splitter is arranged to route the excitation light towards the scanning
element,
wherein the beam splitter is arranged to route detection light arriving from the scanning element into the proximal end of the second set of optical components,
wherein the light detector array comprises a 2D image sensor,
wherein the excitation light comprises a sheet of excitation light,
wherein the camera sequentially captures a plurality of images formed by the detection light that has passed through the second set of optical components, each of the plurality of images corresponding to a respective frame of image date, and
wherein the apparatus further comprises a processor programmed to correct for aberration in the frames of image data.
18. The apparatus of claim 17, wherein the processor corrects for aberration in the frames of image data by performing deconvolution of a point spread function.
19. The apparatus of claim 18, wherein the objective has a pupil plane, and the phase modulating element is positioned at a plane that is conjugate to the pupil plane of the objective.
20. The apparatus of claim 18, wherein the phase modulating element has a phase profile of
{ y) = ln:J{ \~' -÷ .···"' f where x and y are normalized pupil coordinates and a is a tuning parameter set to |Ψ/2|, where
Ψ: ii^ v¾ dz) = -(1 2λ}*ΝΑϊ*{ U + v2 5} r¾
where λ is a wavelength of light passing through the phase modulating element, NA is a numerical aperture of the objective, u and v are normalized pupil coordinates, n is an index of refraction of an immersion medium underneath the objective, and dz is one half of a range of depths to be imaged.
21. The apparatus of claim 1 , wherein the phase modulating element has a phase profile of
where x and y are normalized pupil coordinates and a is a tuning parameter set to |Ψ/2|, where
where λ is a wavelength of light passing through the phase modulating element, NA is a numerical aperture of the objective, u and v are normalized pupil coordinates, n is an index of refraction of an immersion medium underneath the objective, and dz is one half of a range of depths to be imaged.
22. An imaging apparatus comprising:
an objective;
a plurality of optical elements disposed in a detection path arranged to receive light from the objective, wherein the plurality of optical elements disposed in the detection path includes a scanning element,
wherein the scanning element is arranged to (a) route excitation light into the
objective so as to generate a sweeping excitation beam through a forward
image plane of the objective and (b) simultaneously route image light returning through the objective along the detection path to form a conjugate image;
a light detector array positioned to capture images of the conjugate image; and
a phase modulating element disposed in the detection path between the scanning element and the light detector array, wherein the phase modulating element extends a depth of field in the detection path.
23. The apparatus of claim 22, wherein the phase modulating element extends the depth of field by inducing a controlled aberration so as to homogenize point spread functions of points at, above, and below a focal plane when measured at the light detector array.
24. The apparatus of claim 23, wherein the aberration is corrected by an image processing algorithm.
25. The apparatus of claim 23, wherein the aberration is corrected by deconvolution of a measured or simulated point spread function.
26. The apparatus of claim 25, wherein the light detector array comprises a 2D image sensor.
27. The apparatus of claim 22, wherein the light detector array comprises a linear image sensor.
28. An imaging apparatus comprising:
a light source;
a cylindrical lens or a scanner that expands light from the light source into a sheet of light; a beam splitter disposed in a path of the sheet of light;
a scanning element disposed in a path of the sheet of light;
a first telescope having a proximal end and a distal end, with an objective disposed at the distal end of the first telescope;
a second telescope having a proximal end and a distal end, the second telescope having an optical axis,
wherein the beam splitter routes the sheet of light towards the scanning element, wherein the scanning element routes the sheet of light into the proximal end of the first telescope, wherein the first telescope routes the sheet of light in a proximal to distal direction through the objective, accepts fluorescent light through the objective and routes the fluorescent light in a distal to proximal direction back to the scanning element, wherein the scanning element routes
the fluorescent light through the beam splitter and into the proximal end of the second telescope, and wherein the second telescope forms an image from the fluorescent light at a conjugate image plane;
a camera positioned on the same optical axis as the second telescope and configured to
capture blurred images of the conjugate image plane;
a phase modulating element disposed between the second telescope and the camera; and an image processor programmed to deblur images captured by the camera.
29. The apparatus of claim 28 wherein the light source comprises a laser.
30. The apparatus of claim 28 wherein the phase modulating element comprises a phase plate.
31. The apparatus of claim 28 wherein the phase modulating element comprises at least one of a spatial light modulator and a deformable mirror.
32. The apparatus of claim 28 wherein the image processor is programmed to deblur the images using a deconvolution algorithm.
33. A method of imaging a sample comprising:
projecting a sheet of excitation light into a sample, wherein the sheet of excitation light is projected into the sample at an oblique angle, and wherein the sheet of excitation light is projected into the sample at a position that varies depending on an orientation of a scanning element;
routing detection light arriving from the sample back to the scanning element;
using the scanning element to reroute the detection light into an optical system that induces a controlled aberration so as to homogenize point spread functions of points at, above, and below a focal plane when measured at the camera;
using the aberrated detection light to form a plurality of images at a plurality of times, each of the times corresponding to a different orientation of the scanning element; and capturing the plurality of images.
34. The method of claim 33, further comprising correcting for the aberration in the
plurality of images.
35. The method of claim 34, wherein the correcting step comprises performing
deconvolution of a point spread function.
36. The method of claim 35, wherein the point spread function is either a measured point spread function or a simulated point spread function.
The method of claim 33, wherein the aberration is induced by a phase modulating element having a phase profile of
where x and y are normalized pupil coordinates and a is a tuning parameter set to |Ψ/2|, where ¾" dz) = -< j/2A * NA2*{ u.a÷
where λ is a wavelength of light passing through the phase modulating element, NA i a numerical aperture of the objective, u and v are normalized pupil coordinates, n is an index of refraction of an immersion medium underneath the objective, and dz is one half of a range of depths to be imaged.
Priority Applications (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201780042065.4A CN109477954A (en) | 2016-05-30 | 2017-05-30 | SCAPE microscopy and image reconstruction with phase modulating elements |
| JP2018562646A JP2019518242A (en) | 2016-05-30 | 2017-05-30 | SCAPE microscopy using a phase modulation element and an image reconstruction unit |
| US16/304,755 US10908088B2 (en) | 2016-05-30 | 2017-05-30 | SCAPE microscopy with phase modulating element and image reconstruction |
| EP17807328.4A EP3465319A4 (en) | 2016-05-30 | 2017-05-30 | SCAPE MICROSCOPY WITH PHASE MODULATION ELEMENT AND IMAGE RECONSTRUCTION |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US201662343112P | 2016-05-30 | 2016-05-30 | |
| US62/343,112 | 2016-05-30 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2017210182A1 true WO2017210182A1 (en) | 2017-12-07 |
Family
ID=60478997
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/US2017/034984 Ceased WO2017210182A1 (en) | 2016-05-30 | 2017-05-30 | Scape microscopy with phase modulating element and image reconstruction |
Country Status (5)
| Country | Link |
|---|---|
| US (1) | US10908088B2 (en) |
| EP (1) | EP3465319A4 (en) |
| JP (1) | JP2019518242A (en) |
| CN (1) | CN109477954A (en) |
| WO (1) | WO2017210182A1 (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10831014B2 (en) | 2014-01-17 | 2020-11-10 | The Trustees Of Columbia University In The City Of New York | Systems and methods for three dimensional imaging |
| US10835111B2 (en) | 2016-07-10 | 2020-11-17 | The Trustees Of Columbia University In The City Of New York | Three-dimensional imaging using swept, confocally aligned planar excitation with an image relay |
| US10852520B2 (en) | 2016-09-16 | 2020-12-01 | The Trustees Of Columbia University In The City Of New York | Three-dimensional imaging using swept, confocally aligned planar excitation and a customized image splitter |
| US10908088B2 (en) | 2016-05-30 | 2021-02-02 | The Trustees Of Columbia University In The City Of New York | SCAPE microscopy with phase modulating element and image reconstruction |
| US10955652B2 (en) | 2016-09-30 | 2021-03-23 | The Trustees Of Columbia University In The City Of New York | Three-dimensional imaging using swept, confocally aligned planar excitation with a Powell lens and/or deliberate misalignment |
| US11036037B2 (en) | 2016-11-12 | 2021-06-15 | The Trustees Of Columbia University In The City Of New York | Microscopy devices, methods and systems |
| US20210349027A1 (en) * | 2018-06-27 | 2021-11-11 | Enumerix, Inc. | Light sheet fluorescence microscopic imaging device for imaging transparent droplet and test method |
| US11243391B2 (en) | 2016-05-30 | 2022-02-08 | The Trustees Of Columbia University In The City Of New York | Three-dimensional imaging using swept confocally aligned planar excitation with asymmetrical magnification |
| CN115184317A (en) * | 2022-06-16 | 2022-10-14 | 华中科技大学 | Programmable point spread function 3D fluorescence imaging method and system |
| US12270815B2 (en) | 2021-03-05 | 2025-04-08 | Countable Labs, Inc. | Systems and methods for generating droplets and performing digital analyses |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3497501B1 (en) * | 2016-08-15 | 2022-01-19 | Leica Microsystems CMS GmbH | Light sheet microscope |
| WO2019075424A1 (en) * | 2017-10-12 | 2019-04-18 | Howard Hughes Medical Institute | High-resolution, real-time imaging with adaptive optics and lattice light sheets |
| EP3885813B1 (en) * | 2020-03-27 | 2025-12-10 | Leica Microsystems CMS GmbH | Method and device for estimating a sted resolution |
| EP4264352A4 (en) * | 2020-12-15 | 2024-11-20 | The Trustees of Columbia University in the City of New York | Point-of-care microscope for real-time acquisition of volumetric histological images in vivo |
| EP4305471A4 (en) | 2021-03-11 | 2025-04-09 | The Trustees of Columbia University in the City of New York | PLANAR EXCITATION, CONFOCAL ALIGNMENT, AND SCANNING (SCAPE) MICROSCOPY |
| CN112857568A (en) * | 2021-03-17 | 2021-05-28 | 中国科学技术大学 | Rapid spectrum acquisition system based on extended depth of field |
| US20220364848A1 (en) * | 2021-05-13 | 2022-11-17 | Industrial Technology Research Institute | Depth measurement apparatus and depth measurement method |
| US12394655B2 (en) * | 2022-11-23 | 2025-08-19 | Applied Materials, Inc. | Subsurface alignment metrology system for packaging applications |
| CN116107076B (en) * | 2023-04-17 | 2023-07-07 | 武汉慧观生物科技有限公司 | A single-objective light-sheet three-dimensional fluorescence imaging system |
| KR102870720B1 (en) * | 2023-10-25 | 2025-10-15 | 울산과학기술원 | Method, device and microscope for taking large-area and high-resolution image |
| CN117562488A (en) * | 2023-11-14 | 2024-02-20 | 卓外(上海)医疗电子科技有限公司 | Fluorescence endoscope camera system and imaging method based on wavefront encoding technology |
| US20250231081A1 (en) * | 2024-01-11 | 2025-07-17 | Viavi Solutions Inc. | Inspection of optical components in optical systems |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060114542A1 (en) * | 2004-11-26 | 2006-06-01 | Bloom David M | Differential interferometric light modulator and image display device |
| US20100214404A1 (en) * | 2007-07-06 | 2010-08-26 | Nanguang Chen | Fluorescence focal modulation microscopy system and method |
| US20100245822A1 (en) * | 2007-09-28 | 2010-09-30 | MAGYAR TUDOMANYOS AKADEMIA Szegedi Biologial Kozpont | Differential polarization measuring extension unit for a laser-scanning microscope |
| US20140045347A1 (en) * | 2005-12-05 | 2014-02-13 | The Trustees Of Columbia University In The City Of New York | Systems and methods for processing a film, and thin films |
| WO2015109323A2 (en) | 2014-01-17 | 2015-07-23 | The Trustees Of Columbia University In The City Of New York | Systems and methods for three-dimensional imaging |
Family Cites Families (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE10338472B4 (en) * | 2003-08-21 | 2020-08-06 | Carl Zeiss Meditec Ag | Optical imaging system with extended depth of field |
| JP2006067171A (en) * | 2004-08-26 | 2006-03-09 | Ohkura Industry Co | Color image pickup device |
| EP1952091A1 (en) * | 2005-10-17 | 2008-08-06 | Arryx, Inc. | Apparatus and method for detecting deformability of cells using spatially modulated optical force microscopy |
| DE102006047912A1 (en) | 2006-10-06 | 2008-04-10 | Carl Zeiss Microimaging Gmbh | Method and device for parallelized microscopic imaging |
| WO2009005748A1 (en) | 2007-06-29 | 2009-01-08 | The Trustees Of Columbia University In The City Ofnew York | Optical imaging or spectroscopy systems and methods |
| WO2011059833A2 (en) * | 2009-10-29 | 2011-05-19 | California Institute Of Technology | Dual-mode raster point scanning/light sheet illumination microscope |
| US8619237B2 (en) | 2009-12-04 | 2013-12-31 | The Trustees Of Columbia University In The City Of New York | Laser-scanning intersecting plane tomography such as for high speed volumetric optical imaging |
| CN102980875B (en) * | 2012-11-19 | 2015-04-22 | 深圳大学 | Method, optical module and system for extended field depth three-dimensional nanoscale-resolution imaging |
| CN105379253A (en) * | 2013-01-25 | 2016-03-02 | 纽约市哥伦比亚大学理事会 | Depth of field 3d imaging slm microscope |
| US9581798B2 (en) | 2013-07-22 | 2017-02-28 | Fundacio Institut De Ciencies Fotoniques | Light sheet-based imaging device with extended depth of field |
| US10061111B2 (en) | 2014-01-17 | 2018-08-28 | The Trustees Of Columbia University In The City Of New York | Systems and methods for three dimensional imaging |
| DE102014102215A1 (en) * | 2014-02-20 | 2015-08-20 | Carl Zeiss Microscopy Gmbh | Method and arrangement for light-sheet microscopy |
| CN204719330U (en) * | 2015-04-09 | 2015-10-21 | 中国科学院西安光学精密机械研究所 | Wavefront coded imaging system |
| US10187626B2 (en) * | 2015-04-10 | 2019-01-22 | The Board Of Trustees Of The Leland Stanford Junior University | Apparatuses and methods for three-dimensional imaging of an object |
| WO2017210182A1 (en) | 2016-05-30 | 2017-12-07 | The Trustees Of Columbia University In The City Of New York | Scape microscopy with phase modulating element and image reconstruction |
| US11243391B2 (en) | 2016-05-30 | 2022-02-08 | The Trustees Of Columbia University In The City Of New York | Three-dimensional imaging using swept confocally aligned planar excitation with asymmetrical magnification |
| US10835111B2 (en) | 2016-07-10 | 2020-11-17 | The Trustees Of Columbia University In The City Of New York | Three-dimensional imaging using swept, confocally aligned planar excitation with an image relay |
| US10852520B2 (en) | 2016-09-16 | 2020-12-01 | The Trustees Of Columbia University In The City Of New York | Three-dimensional imaging using swept, confocally aligned planar excitation and a customized image splitter |
| US10955652B2 (en) | 2016-09-30 | 2021-03-23 | The Trustees Of Columbia University In The City Of New York | Three-dimensional imaging using swept, confocally aligned planar excitation with a Powell lens and/or deliberate misalignment |
| CN110178069B (en) | 2016-11-12 | 2022-05-17 | 纽约市哥伦比亚大学理事会 | Microscope apparatus, method and system |
| DE102018122652B4 (en) * | 2018-09-17 | 2025-11-27 | Carl Zeiss Microscopy Gmbh | Spectrally resolved, high-resolution 3D localization microscopy |
-
2017
- 2017-05-30 WO PCT/US2017/034984 patent/WO2017210182A1/en not_active Ceased
- 2017-05-30 JP JP2018562646A patent/JP2019518242A/en active Pending
- 2017-05-30 US US16/304,755 patent/US10908088B2/en active Active
- 2017-05-30 CN CN201780042065.4A patent/CN109477954A/en active Pending
- 2017-05-30 EP EP17807328.4A patent/EP3465319A4/en not_active Withdrawn
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060114542A1 (en) * | 2004-11-26 | 2006-06-01 | Bloom David M | Differential interferometric light modulator and image display device |
| US20140045347A1 (en) * | 2005-12-05 | 2014-02-13 | The Trustees Of Columbia University In The City Of New York | Systems and methods for processing a film, and thin films |
| US20100214404A1 (en) * | 2007-07-06 | 2010-08-26 | Nanguang Chen | Fluorescence focal modulation microscopy system and method |
| US20100245822A1 (en) * | 2007-09-28 | 2010-09-30 | MAGYAR TUDOMANYOS AKADEMIA Szegedi Biologial Kozpont | Differential polarization measuring extension unit for a laser-scanning microscope |
| WO2015109323A2 (en) | 2014-01-17 | 2015-07-23 | The Trustees Of Columbia University In The City Of New York | Systems and methods for three-dimensional imaging |
Non-Patent Citations (8)
| Title |
|---|
| D. C. ANDREO ET AL., MASTER IN PHOTONICS FAST IMAGE RESTORATION IN LIGHT-SHEET FLUORESCENCE MICROSCOPY WITH EXTENDED DEPTH OF FIELD USING GPUS, 2015 |
| LAUER, TOD: "Deconvolution with a Spatially-Variant PSF", ASTRONOMICAL TELESCOPES AND INSTRUMENTATION, INTERNATIONAL SOCIETY FOR OPTICS AND PHOTONICS, 2002 |
| M. DEMENIKOV ET AL.: "Image Artifacts in Hybrid Imaging Systems with a Cubic Phase Mask.", OPT. EXPRESS, vol. 18, 2010, pages 8207 - 8212 |
| O. OLARTE ET AL.: "Decoupled Illumination Detection in Light Sheet Microscopy for Fast Volumetric Imaging", OPTICA, vol. 2, 2015, pages 702 - 705 |
| S. QUIRIN ET AL.: "Instantaneous Three-Dimensional Sensing Using Spatial Light Modulator Illumination with Extended Depth of Field Imaging", OPT. EXPRESS, vol. 21, 2013, pages 16007 - 16021 |
| S. QUIRIN ET AL.: "Simultaneous Imaging of Neural Activity in Three Dimensions", FRONT. NEURAL CIRCUITS, vol. 8, 2014, pages 29, XP055605124, doi:10.3389/fncir.2014.00029 |
| S. QUIRIN: "Calcium imaging of neural circuits with extended depth-of-field Lightsheet Microscopy", OPT. LETT., vol. 41, 2016, pages 855 |
| See also references of EP3465319A4 * |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10831014B2 (en) | 2014-01-17 | 2020-11-10 | The Trustees Of Columbia University In The City Of New York | Systems and methods for three dimensional imaging |
| US11460685B2 (en) | 2014-01-17 | 2022-10-04 | The Trusteees Of Columbia University In The City Of New York | Systems and methods for three-dimensional imaging |
| US10908088B2 (en) | 2016-05-30 | 2021-02-02 | The Trustees Of Columbia University In The City Of New York | SCAPE microscopy with phase modulating element and image reconstruction |
| US11243391B2 (en) | 2016-05-30 | 2022-02-08 | The Trustees Of Columbia University In The City Of New York | Three-dimensional imaging using swept confocally aligned planar excitation with asymmetrical magnification |
| US10835111B2 (en) | 2016-07-10 | 2020-11-17 | The Trustees Of Columbia University In The City Of New York | Three-dimensional imaging using swept, confocally aligned planar excitation with an image relay |
| US11317787B2 (en) | 2016-07-10 | 2022-05-03 | The Trustees Of Columbia University In The City Of New York | Catheter-based three-dimensional imaging using swept, confocally aligned planar excitation |
| US11269169B2 (en) | 2016-09-16 | 2022-03-08 | The Trustees Of Columbia University In The City Of New York | Three-dimensional imaging using swept, confocally aligned planar excitation and sample perturbation |
| US10852520B2 (en) | 2016-09-16 | 2020-12-01 | The Trustees Of Columbia University In The City Of New York | Three-dimensional imaging using swept, confocally aligned planar excitation and a customized image splitter |
| US10955652B2 (en) | 2016-09-30 | 2021-03-23 | The Trustees Of Columbia University In The City Of New York | Three-dimensional imaging using swept, confocally aligned planar excitation with a Powell lens and/or deliberate misalignment |
| US11333874B2 (en) | 2016-09-30 | 2022-05-17 | The Trustees Of Columbia University In The City Of New York | Three-dimensional imaging using swept, confocally aligned planar excitation with a Powell lens and/or deliberate misalignment |
| US11036037B2 (en) | 2016-11-12 | 2021-06-15 | The Trustees Of Columbia University In The City Of New York | Microscopy devices, methods and systems |
| US11604342B2 (en) | 2016-11-12 | 2023-03-14 | The Trustees Of Columbia University In The City Of New York | Microscopy devices, methods and systems |
| US20210349027A1 (en) * | 2018-06-27 | 2021-11-11 | Enumerix, Inc. | Light sheet fluorescence microscopic imaging device for imaging transparent droplet and test method |
| US12270815B2 (en) | 2021-03-05 | 2025-04-08 | Countable Labs, Inc. | Systems and methods for generating droplets and performing digital analyses |
| US12540948B2 (en) | 2021-03-05 | 2026-02-03 | Countable Labs, Inc. | Systems and methods for generating droplets and performing digital analyses |
| CN115184317A (en) * | 2022-06-16 | 2022-10-14 | 华中科技大学 | Programmable point spread function 3D fluorescence imaging method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109477954A (en) | 2019-03-15 |
| EP3465319A4 (en) | 2020-02-19 |
| US10908088B2 (en) | 2021-02-02 |
| JP2019518242A (en) | 2019-06-27 |
| EP3465319A1 (en) | 2019-04-10 |
| US20200249168A1 (en) | 2020-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10908088B2 (en) | SCAPE microscopy with phase modulating element and image reconstruction | |
| US11604342B2 (en) | Microscopy devices, methods and systems | |
| CN109416321B (en) | Hyperspectral confocal microscope based on spatial light modulator and use method thereof | |
| JP6795624B2 (en) | Spatial Light Modulator Based Hyperspectral Confocal Microscope and Usage | |
| US9581798B2 (en) | Light sheet-based imaging device with extended depth of field | |
| CN110262026B (en) | Aperture scanning Fourier ptychographic imaging | |
| CN106461925B (en) | Systems and methods for Raman scattering optical microscopy with adaptive optics | |
| US20140008549A1 (en) | Method and system for obtaining an extended-depth-of-field volumetric image using laser scanning imaging | |
| CN112557360A (en) | System and method for hyperspectral imaging | |
| US10514533B2 (en) | Method for creating a microscope image, microscopy device, and deflecting device | |
| JP2018527607A (en) | System and method for three-dimensional imaging | |
| US10191263B2 (en) | Scanning microscopy system | |
| EP3757647A1 (en) | Multi-view light-sheet microscope with an optical arm combiner | |
| CN107850765A (en) | The method and sub-assembly of beam-shaping and photosphere microtechnic | |
| CA3035876A1 (en) | Line scan photon reassignment microscopy systems | |
| CA3065917A1 (en) | Super-resolution line scanning confocal microscopy with pupil filtering | |
| Murray | Methods for imaging thick specimens: confocal microscopy, deconvolution, and structured illumination | |
| US20250291131A1 (en) | Multimodal Microscope and Microscopy Method | |
| US11156818B2 (en) | Flexible light sheet generation by field synthesis | |
| EP3627205A1 (en) | Confocal laser scanning microscope configured for generating line foci | |
| CN121299904A (en) | Microscope and Microscopic Methods | |
| Power et al. | Illumination in Light Sheet Fluorescence Microscopy | |
| CN121232424A (en) | microscope | |
| Landry | High-Speed Isotropic Resolution and Structured Illumination Using Phased Arrays In Light Sheet Fluorescence Microscopy |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17807328 Country of ref document: EP Kind code of ref document: A1 |
|
| ENP | Entry into the national phase |
Ref document number: 2018562646 Country of ref document: JP Kind code of ref document: A |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| ENP | Entry into the national phase |
Ref document number: 2017807328 Country of ref document: EP Effective date: 20190102 |