WO2019046774A1 - Systèmes et procédés de génération d'images médicales 3d par balayage d'un bloc de tissu entier - Google Patents

Systèmes et procédés de génération d'images médicales 3d par balayage d'un bloc de tissu entier Download PDF

Info

Publication number
WO2019046774A1
WO2019046774A1 PCT/US2018/049185 US2018049185W WO2019046774A1 WO 2019046774 A1 WO2019046774 A1 WO 2019046774A1 US 2018049185 W US2018049185 W US 2018049185W WO 2019046774 A1 WO2019046774 A1 WO 2019046774A1
Authority
WO
WIPO (PCT)
Prior art keywords
stained
virtual
slices
virtual slices
tissue
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2018/049185
Other languages
English (en)
Inventor
Yukako Yagi
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Memorial Sloan Kettering Cancer Center
Original Assignee
Memorial Sloan Kettering Cancer Center
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Memorial Sloan Kettering Cancer Center filed Critical Memorial Sloan Kettering Cancer Center
Publication of WO2019046774A1 publication Critical patent/WO2019046774A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N1/00—Sampling; Preparing specimens for investigation
    • G01N1/28—Preparing specimens for investigation including physical details of (bio-)chemical methods covered elsewhere, e.g. G01N33/50, C12Q
    • G01N1/30—Staining; Impregnating ; Fixation; Dehydration; Multistep processes for preparing samples of tissue, cell or nucleic acid material and the like for analysis
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N1/00—Sampling; Preparing specimens for investigation
    • G01N1/02—Devices for withdrawing samples
    • G01N1/04—Devices for withdrawing samples in the solid state, e.g. by cutting
    • G01N1/06—Devices for withdrawing samples in the solid state, e.g. by cutting providing a thin slice, e.g. microtome
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/251—Fusion techniques of input or preprocessed data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images

Definitions

  • This disclosure generally relates to systems and methods for generating three- dimensional (3D) images by scanning a whole tissue block. More particularly, the disclosure relates to systems and methods for scanning an entire tissue block, such as a formalin-fixed paraffin-embedded block, to generate a 3D image representing the tissue sample of the block.
  • an entire tissue block such as a formalin-fixed paraffin-embedded block
  • Three-dimensional (3D) medical image generation can be a costly and tedious process.
  • a tissue sample is first embedded into a block, such as a formalin-fixed, paraffin-embedded (FFPE) block.
  • FFPE formalin-fixed, paraffin-embedded
  • the block is cut into slices, and each slice is mounted on a glass slide.
  • the slides are then individually scanned using medical imaging equipment to produce an image corresponding to each slide.
  • image processing software is used to generate a 3D representation of the block based on the individual slide images. Because many of these steps are performed manually, the process can take a long time and can be very expensive.
  • tissue samples are embedded in formalin-fixed, paraffin-embedded (FFPE) blocks. Each block is cut into slices. The slices are mounted on respective glass slides, which are then individually scanned using medical imaging equipment to produce an image corresponding to each slide. The 3D tissue sample is then reconstructed using image processing software based on the individual slide images. In some cases, it can take at least one week to complete this process for a single block. In some instances, the cost for processing the single block can be around $40,000. Additionally, due the computational difficulty of processing a large number of slides, medical professionals typically restrict the number of slides generated from each block.
  • FFPE formalin-fixed, paraffin-embedded
  • a block may only be divided into 200-300 slices that become slides.
  • the spatial resolution of a 3D image generated by this type of process is limited, which inhibits the ability of a physician, pathologist or other medical care provider to accurately identify features present in the tissue sample based on the 3D image (e.g., for diagnostic purposes).
  • Methods, systems, and apparatus are provided relating to generating three- dimensional (3D) images by scanning a whole tissue block.
  • This disclosure provides innovative systems and methods for producing 3D images of tissue samples.
  • the solutions of this disclosure can allow for an entire block to be scanned at once, without the need to physically slice the block and produce glass slides. As a result, significant time and cost can be saved.
  • the resolution of the scanning techniques disclosed herein is not limited to any particular number of glass slides, and is only limited by the resolution of the scanning equipment used.
  • a 3D image produced using the techniques of this disclosure can have substantially higher resolution than images reconstructed from glass slides as described above. This can allow a physician, pathologist or other medical care provider to more accurately characterize features using the 3D image.
  • a system for generating a 3D image can include a scanning device, such as a micro computed tomography (microCT) scanner.
  • the scanning device can include hardware configured to hold an entire paraffin block in position to allow the block to be scanned.
  • the scanning device can generate a plurality of "virtual slide" images representing different layers stacked vertically within the block.
  • image processing software can use the virtual slide images to reconstruct a 3D representation of the tissue sample captured in the paraffin block.
  • the image processing software can implement novel algorithms that improve the ability of the software to register consecutive images produced by the scanning device.
  • the consecutive images can be registered to generate the 3D image.
  • the image processing software can generate the 3D image more quickly and more accurately than the conventional techniques described above.
  • 3D medical images can be generated more quickly and at lower cost, relative to conventional techniques.
  • the 3D images may have a significantly higher spatial resolution than those generated through conventional techniques. Greater spatial resolution allows a physician, pathologist or other medical care provider to more accurately characterize the tissue sample. It can be difficult or impossible to identify certain features by examining a 2D image of a tissue sample, and the 3D images that are produced using conventional techniques may not have sufficient resolution to overcome the limitations of 3D images to detect these features. For example, some tissue samples exhibit "islands" of tumor cells that are only identifiable by examining a 3D representation of the tissue sample.
  • the solutions of this disclosure can allow a physician, pathologist or other medical care provider to identify such features that may have been difficult or impossible to observe using conventional techniques, thereby providing earlier and more accurate diagnosis of tumor cells in a patient.
  • At least one aspect of this disclosure is directed to a method for generating a three- dimensional (3D) image representing a tissue sample.
  • the method can include identifying a tissue block comprising a solid tissue sample.
  • the method can include identifying a plurality of virtual slices of the tissue block.
  • the method can include digitally staining each of the virtual slices to produce a plurality of stained virtual slices.
  • the method can include constructing a 3D image of the solid tissue sample based on the plurality of stained virtual slices.
  • identifying the plurality of virtual slices of the tissue block can include identifying at least 400 virtual slices of the tissue block.
  • each of the plurality of virtual slices can have a thickness between one micron and ten microns.
  • digitally staining each of the virtual slices can include identifying a plurality of pixels of each virtual slice. Each pixel can have an intensity value.
  • the method can include modifying each pixel to have a color selected based on its intensity value.
  • the method can include selecting, for each pixel of each slice, the color based on a difference between the intensity value of the pixel and at least one of a maximum pixel intensity value and a minimum pixel intensity value.
  • the method can include separating the tissue block into a plurality of slides.
  • the method can also include applying a physical stain to each of the slides to produce a plurality of stained slides.
  • the method can also include comparing a visual characteristic of at least one of the plurality of stained virtual slices to a corresponding visual characteristic of at least one of the plurality of stained slides.
  • the method can also include updating a machine learning model to digitally stain virtual slices based on the comparison of the visual characteristic of the at least one of the plurality of stained virtual slices to the corresponding visual characteristic of the at least one of the plurality of stained slides.
  • the visual characteristic can be indicative of at least one tumor cell within the tissue sample.
  • the method can include segmenting at least one of the plurality of stained virtual slices into a first plurality of portions corresponding to the tissue sample and a second plurality of portions corresponding to a substrate material of the tissue block. In some implementations, the method can include discarding the second plurality of portions corresponding to the substrate material of the tissue block, prior to constructing the 3D image of the solid tissue sample based on the plurality of stained virtual slices.
  • the method can include applying a registration algorithm to at least one pair of consecutive stained virtual slices of the tissue block to align the at least one pair of consecutive stained virtual slices with respect to one another, prior to constructing the 3D image of the solid tissue sample based on the plurality of stained virtual slices.
  • the system can include a memory and at least one processor.
  • the processor can be configured to identify a tissue block comprising a solid tissue sample.
  • the processor can be configured to identify a plurality of virtual slices of the tissue block.
  • the processor can be configured to digitally stain each of the virtual slices to produce a plurality of stained virtual slices.
  • the processor can be configured to construct a 3D image of the solid tissue sample based on the virtual slices.
  • the processor can be configured to identify at least 400 virtual slices of the tissue block.
  • each of the plurality of virtual slices can have a thickness between one micron and ten microns.
  • the processor can be configured to digitally stain each of the virtual slices by identifying a plurality of pixels of the virtual slice for each slice. Each pixel can have an intensity value. The processor can also be configured to modifying each pixel to have a color selected based on its intensity value. In some implementations, the processor can be further configured to select, for each pixel of each slice, the color based on a difference between the intensity value of the pixel and at least one of a maximum pixel intensity value and a minimum pixel intensity value. [0018] In some implementations, the processor can be further configured to compare a visual characteristic of at least one of the plurality of stained virtual slices to a corresponding visual characteristic of at least one of a plurality of stained slides.
  • the plurality of stained slides can be produced by separating the tissue block into a plurality of slides and applying a physical stain to each of the plurality of slides.
  • the processor can also be configured to update a machine learning model to digitally stain virtual slices based on the comparison of the visual characteristic of the at least one of the plurality of stained virtual slices to the corresponding visual characteristic of the at least one of the plurality of stained slides.
  • the visual characteristic can be indicative of at least one tumor cell within the tissue sample.
  • the processor can be further configured to segment at least one of the plurality of stained virtual slices into a first plurality of portions corresponding to the tissue sample and a second plurality of portions corresponding to a substrate material of the tissue block. In some implementations, the processor can be further configured to discard the second plurality of portions corresponding to the substrate material of the tissue block, prior to constructing the 3D image of the solid tissue sample based on the plurality of stained virtual slices.
  • the processor can be further configured to apply a registration algorithm to at least one pair of consecutive stained virtual slices of the tissue block to align the at least one pair of consecutive stained virtual slices with respect to one another, prior to constructing the 3D image of the solid tissue sample based on the plurality of stained virtual slices.
  • FIG. 1 A is a block diagram depicting an embodiment of a network environment comprising a client device in communication with a server device;
  • FIG. IB is a block diagram depicting a cloud computing environment comprising a client device in communication with cloud service providers;
  • FIGS. 1C and ID are block diagrams depicting embodiments of computing devices useful in connection with the methods and systems described herein.
  • FIG. 2 is a flowchart for an example method of generating a 3D image by sectioning a tissue block.
  • FIG. 3 A is a block diagram of a system for generating a 3D image of a tissue by scanning a whole tissue block.
  • FIG. 3B is a flow chart for an example method of generating a 3D image by scanning a whole tissue block.
  • FIG. 4A-4B show various views of an example 3D model of a tissue sample.
  • FIG. 5A shows a stained slide, according to an illustrative implementation.
  • FIG. SB shows a stained virtual slice corresponding to the stained slide of FIG. 5 A, according to an illustrative implementation.
  • FIG. 6A shows a micro-CT scan of a tissue sample corresponding to a great toe that exhibits an osteosarcoma.
  • FIG. 6B shows a virtually stained micro-CT, which corresponds to the same tissue sample depicted in FIG. 6A.
  • FIG. 6C shows a stained physical slide corresponding to the tissue sample shown in FIG. 6A.
  • Section A describes a network environment and computing environment which may be useful for practicing embodiments described herein.
  • Section B describes embodiments of systems and methods for generating a 3D image by scanning a whole tissue block.
  • Appendix A provides additional technical details relating to this disclosure.
  • FIG. 1A an embodiment of a network environment is depicted.
  • the network environment includes one or more clients 102a-102n (also generally referred to as local machine(s) 102, client(s) 102, client node(s) 102, client machine(s) 102, client computers) 102, client device(s) 102, endpoint(s) 102, or endpoint node(s) 102) in communication with one or more servers 106a-106n (also generally referred to as servers) 106, node 106, or remote machine(s) 106) via one or more networks 104.
  • a client 102 has the capacity to function as both a client node seeking access to resources provided by a server and as a server providing access to hosted resources for other clients 102a-102n.
  • FIG. 1A shows a network 104 between the clients 102 and the servers 106
  • the clients 102 and the servers 106 may be on the same network 104.
  • a network 104' (not shown) may be a private network and a network 104 may be a public network.
  • a network 104 may be a private network and a network 104' a public network.
  • networks 104 and 104' may both be private networks.
  • the network 104 may be connected via wired or wireless links.
  • Wired links may include Digital Subscriber Line (DSL), coaxial cable lines, or optical fiber lines.
  • the wireless links may include BLUETOOTH, Wi-Fi, Worldwide Interoperability for Microwave Access (WiMAX), an infrared channel or satellite band.
  • the wireless links may also include any cellular network standards used to communicate among mobile devices, including standards that qualify as 1G, 2G, 3G, or 4G.
  • the network standards may qualify as one or more generation of mobile telecommunication standards by fulfilling a specification or standards such as the specifications maintained by International Telecommunication Union.
  • the 3G standards may correspond to the International Mobile Telecommunications- 2000 (IMT-2000) specification, and the 4G standards may correspond to the International Mobile Telecommunications Advanced (TMT-Advanced) specification.
  • Examples of cellular network standards include AMPS, GSM, GPRS, UMTS, LTE, LTE Advanced, Mobile WiMAX, and WiMAX- Advanced .
  • Cellular network standards may use various channel access methods e.g. FDMA, TDMA, CDMA, or SDMA.
  • different types of data may be transmitted via different links and standards.
  • the same types of data may be transmitted via different links and standards.
  • the network 104 may be any type and/or form of network.
  • the geographical scope of the network 104 may vary widely and the network 104 can be a body area network (BAN), a personal area network (PAN), a local-area network (LAN), e.g. Intranet, a metropolitan area network (MAN), a wide area network (WAN), or the Internet.
  • the topology of the network 104 may be of any form and may include, e.g., any of the following: point-to-point, bus, star, ring, mesh, or tree.
  • the network 104 may be an overlay network which is virtual and sits on top of one or more layers of other networks 104' .
  • the network 104 may be of any such network topology as known to those ordinarily skilled in the art capable of supporting the operations described herein.
  • the network 104 may utilize different techniques and layers or stacks of protocols, including, e.g., the Ethernet protocol, the internet protocol suite (TCP/IP), the ATM (Asynchronous Transfer Mode) technique, the SONET (Synchronous Optical Networking) protocol, or the SDH (Synchronous Digital Hierarchy) protocol.
  • the TCP/IP internet protocol suite may include application layer, transport layer, internet layer (including, e.g., IPv6), or the link layer.
  • the network 104 may be a type of a broadcast network, a telecommunications network, a data communication network, or a computer network.
  • the system may include multiple, logically-grouped servers 106.
  • the logical group of servers may be referred to as a server farm 38 (not shown) or a machine farm 38.
  • the servers 106 may be geographically dispersed.
  • a machine farm 38 may be administered as a single entity.
  • the machine farm 38 includes a plurality of machine farms 38.
  • the servers 106 within each machine farm 38 can be heterogeneous - one or more of the servers 106 or machines 106 can operate according to one type of operating system platform (e.g., WINDOWS NT, manufactured by Microsoft Corp. of Redmond, Washington), while one or more of the other servers 106 can operate on according to another type of operating system platform (e.g., Unix, Linux, or Mac OS X).
  • operating system platform e.g., Unix, Linux, or Mac OS X
  • servers 106 in the machine farm 38 may be stored in high- density rack systems, along with associated storage systems, and located in an enterprise data center. In this embodiment, consolidating the servers 106 in this way may improve system manageability, data security, the physical security of the system, and system performance by locating servers 106 and high performance storage systems on localized high performance networks. Centralizing the servers 106 and storage systems and coupling them with advanced system management tools allows more efficient use of server resources.
  • the servers 106 of each machine farm 38 do not need to be physically proximate to another server 106 in the same machine farm 38.
  • the group of servers 106 logically grouped as a machine farm 38 may be interconnected using a wide-area network (WAN) connection or a metropolitan-area network (MAN) connection.
  • WAN wide-area network
  • MAN metropolitan-area network
  • a machine farm 38 may include servers 106 physically located in different continents or different regions of a continent, country, state, city, campus, or room. Data transmission speeds between servers 106 in the machine farm 38 can be increased if the servers 106 are connected using a local- area network (LAN) connection or some form of direct connection.
  • LAN local- area network
  • a heterogeneous machine farm 38 may include one or more servers 106 operating according to a type of operating system, while one or more other servers 106 execute one or more types of hypervisors rather than operating systems.
  • hypervisors may be used to emulate virtual hardware, partition physical hardware, virtualize physical hardware, and execute virtual machines that provide access to computing environments, allowing multiple operating systems to run concurrently on a host computer.
  • Native hypervisors may run directly on the host computer.
  • Hypervisors may include VMware ESX/ESXi, manufactured by VMWare, Inc., of Palo Alto, California; the Xen hypervisor, an open source product whose development is overseen by Citrix Systems, Inc.; the HYPER-V hypervisors provided by Microsoft or others.
  • Hosted hypervisors may run within an operating system on a second software level. Examples of hosted hypervisors may include VMware Workstation and VIRTUALBOX.
  • Management of the machine farm 38 may be de-centralized.
  • one or more servers 106 may comprise components, subsystems and modules to support one or more management services for the machine farm 38.
  • one or more servers 106 provide functionality for management of dynamic data, including techniques for handling failover, data replication, and increasing the robustness of the machine farm 38.
  • Each server 106 may communicate with a persistent store and, in some embodiments, with a dynamic store.
  • Server 106 may be a file server, application server, web server, proxy server, appliance, network appliance, gateway, gateway server, virtualization server, deployment server, SSL VPN server, or firewall.
  • the server 106 may be referred to as a remote machine or a node.
  • a plurality of nodes 290 may be in the path between any two communicating servers.
  • a cloud computing environment may provide client 102 with one or more resources provided by a network environment.
  • the cloud computing environment may include one or more clients 102a-102n, in communication with the cloud 108 over one or more networks 104.
  • Clients 102 may include, e.g., thick clients, thin clients, and zero clients.
  • a thick client may provide at least some functionality even when disconnected from the cloud 108 or servers 106.
  • a thin client or a zero client may depend on the connection to the cloud 108 or server 106 to provide functionality.
  • a zero client may depend on the cloud 108 or other networks 104 or servers 106 to retrieve operating system data for the client device.
  • the cloud 108 may include back end platforms, e.g., servers 106, storage, server farms or data centers.
  • the cloud 108 may be public, private, or hybrid.
  • Public clouds may include public servers 106 that are maintained by third parties to the clients 102 or the owners of the clients.
  • the servers 106 may be located off-site in remote geographical locations as disclosed above or otherwise.
  • Public clouds may be connected to the servers 106 over a public network.
  • Private clouds may include private servers 106 that are physically maintained by clients 102 or owners of clients.
  • Private clouds may be connected to the servers 106 over a private network 104.
  • Hybrid clouds 108 may include both the private and public networks 104 and servers 106.
  • the cloud 108 may also include a cloud based delivery, e.g. Software as a Service (SaaS) 110, Platform as a Service (PaaS) 112, and Infrastructure as a Service (IaaS) 114.
  • SaaS Software as a Service
  • PaaS Platform as a Service
  • IaaS Infrastructure as a Service
  • IaaS may refer to a user renting the use of infrastructure resources that are needed during a specified time period.
  • IaaS providers may offer storage, networking, servers or virtualization resources from large pools, allowing the users to quickly scale up by accessing more resources as needed. Examples of IaaS include AMAZON WEB SERVICES provided by Amazon.com, Inc., of Seattle, Washington, RACKSPACE CLOUD provided by Rackspace US, Inc., of San Antonio, Texas, Google Compute Engine provided by Google Inc.
  • PaaS providers may offer functionality provided by IaaS, including, e.g., storage, networking, servers or virtualization, as well as additional resources such as, e.g., the operating system, middleware, or runtime resources.
  • IaaS examples include WINDOWS AZURE provided by Microsoft Corporation of Redmond, Washington, Google App Engine provided by Google Inc., and HEROKU provided by Heroku, Inc. of San Francisco,
  • SaaS providers may offer the resources that PaaS provides, including storage, networking, servers, virtualization, operating system, middleware, or runtime resources. In some embodiments, SaaS providers may offer additional resources including, e.g., data and application resources. Examples of SaaS include GOOGLE APPS provided by Google Inc., SALESFORCE provided by Salesforce.com Inc. of San Francisco, California, or OFFICE 365 provided by Microsoft Corporation. Examples of SaaS may also include data storage providers, e.g. DROPBOX provided by Dropbox, Inc. of San Francisco, California, Microsoft SKYDRTVE provided by Microsoft Corporation, Google Drive provided by Google Inc., or Apple ICLOUD provided by Apple Inc. of Cupertino, California.
  • DROPBOX provided by Dropbox, Inc. of San Francisco, California
  • Microsoft SKYDRTVE provided by Microsoft Corporation
  • Google Drive provided by Google Inc.
  • Apple ICLOUD provided by Apple Inc. of Cupertino, California.
  • Clients 102 may access IaaS resources with one or more IaaS standards, including, e.g., Amazon Elastic Compute Cloud (EC2), Open Cloud Computing Interface (OCCI), Cloud Infrastructure Management Interface (CIMI), or OpenStack standards.
  • IaaS standards may allow clients access to resources over HTTP, and may use Representational State Transfer (REST) protocol or Simple Object Access Protocol (SOAP).
  • REST Representational State Transfer
  • SOAP Simple Object Access Protocol
  • Clients 102 may access PaaS resources with different PaaS interfaces.
  • Some PaaS interfaces use HTTP packages, standard Java APIs, JavaMail APL Java Data Objects (JDO), Java Persistence API (JPA), Python APIs, web integration APIs for different programming languages including, e.g., Rack for Ruby, WSGI for Python, or PSGI for Perl, or other APIs that may be built on REST, HTTP, XML, or other protocols.
  • Clients 102 may access SaaS resources through the use of web-based user interfaces, provided by a web browser (e.g. GOOGLE CHROME, Microsoft INTERNET EXPLORER, or Mozilla Firefox provided by Mozilla Foundation of Mountain View, California).
  • Clients 102 may also access SaaS resources through smartphone or tablet applications, including, e.g., Salesforce Sales Cloud, or Google Drive app.
  • Clients 102 may also access SaaS resources through the client operating system, including, e.g., Windows file system for DROPBOX.
  • access to IaaS, PaaS, or SaaS resources may be authenticated.
  • a server or authentication server may authenticate a user via security certificates, HTTPS, or API keys.
  • API keys may include various encryption standards such as, e.g., Advanced Encryption Standard (AES).
  • Data resources may be sent over Transport Layer Security (TLS) or Secure Sockets Layer (SSL).
  • TLS Transport Layer Security
  • SSL Secure Sockets Layer
  • the client 102 and server 106 may be deployed as and/or executed on any type and form of computing device, e.g. a computer, network device or appliance capable of communicating on any type and form of network and performing the operations described herein.
  • FIGs. 1C and ID depict block diagrams of a computing device 100 useful for practicing an embodiment of the client 102 or a server 106. As shown in FIGs. 1C and ID, each computing device 100 includes a central processing unit 121, and a main memory unit 122. As shown in FIG.
  • a computing device 100 may include a storage device 128, an installation device 116, a network interface 118, an I/O controller 123, display devices 124a- 124n, a keyboard 126 and a pointing device 127, e.g. a mouse.
  • the storage device 128 may include, without limitation, an operating system, software, and a software of an image classification system 120.
  • each computing device 100 may also include additional optional elements, e.g. a memory port 103, a bridge 170, one or more input/output devices 130a-130n (generally referred to using reference numeral 130), and a cache memory 140 in communication with the central processing unit 121.
  • the central processing unit 121 is any logic circuitry that responds to and processes instructions fetched from the main memory unit 122.
  • the central processing unit 121 is provided by a microprocessor unit, e.g.: those manufactured by Intel Corporation of Mountain View, California; those manufactured by Motorola Corporation of Schaumburg, Illinois; the ARM processor and TEGRA system on a chip (SoC) manufactured by Nvidia of Santa Clara, California; the POWER7 processor, those manufactured by
  • the computing device 100 may be based on any of these processors, or any other processor capable of operating as described herein.
  • the central processing unit 121 may utilize instruction level parallelism, thread level parallelism, different levels of cache, and multi-core processors.
  • a multi-core processor may include two or more processing units on a single computing component. Examples of a multi- core processors include the AMD PHENOM ⁇ 2, INTEL CORE i5 and INTEL CORE i7.
  • Main memory unit 122 may include one or more memory chips capable of storing data and allowing any storage location to be directly accessed by the microprocessor 121.
  • Main memory unit 122 may be volatile and faster than storage 128 memory.
  • Main memory units 122 may be Dynamic random access memory (DRAM) or any variants, including static random access memory (SRAM), Burst SRAM or SynchBurst SRAM (BSRAM), Fast Page Mode DRAM (FPM DRAM), Enhanced DRAM (EDRAM), Extended Data Output RAM (EDO RAM), Extended Data Output DRAM (EDO DRAM), Burst Extended Data Output DRAM (BEDO DRAM), Single Data Rate Synchronous DRAM (SDR SDRAM), Double Data Rate SDRAM (DDR SDRAM), Direct Rambus DRAM (DRDRAM), or Extreme Data Rate DRAM (XDR DRAM).
  • DRAM Dynamic random access memory
  • SRAM static random access memory
  • BSRAM Burst SRAM or SynchBurst SRAM
  • FPM DRAM Fast Page Mode DRAM
  • the main memory 122 or the storage 128 may be non-volatile; e.g., non-volatile read access memory (NVRAM), flash memory nonvolatile static RAM (nvSRAM), Ferroelectric RAM (FeRAM), Magnetoresistive RAM (MRAM), Phase-change memory (PRAM), conductive-bridging RAM (CBRAM), Silicon- Oxide-Nitride-Oxide-Silicon (SONOS), Resistive RAM (RRAM), Racetrack, Nano-RAM (NRAM), or Millipede memory.
  • NVRAM non-volatile read access memory
  • nvSRAM flash memory nonvolatile static RAM
  • FeRAM Ferroelectric RAM
  • MRAM Magnetoresistive RAM
  • PRAM Phase-change memory
  • CBRAM conductive-bridging RAM
  • SONOS Silicon- Oxide-Nitride-Oxide-Silicon
  • RRAM Racetrack
  • Nano-RAM NRAM
  • Millipede memory Millipede memory.
  • the main memory 122
  • FIG. ID depicts an embodiment of a computing device 100 in which the processor communicates directly with main memory 122 via a memory port 103.
  • the main memory 122 may be DRDRAM.
  • FIG. ID depicts an embodiment in which the main processor 121 communicates directly with cache memory 140 via a secondary bus, sometimes referred to as a backside bus.
  • the main processor 121 communicates with cache memory 140 using the system bus ISO.
  • Cache memory 140 typically has a faster response time than main memory 122 and is typically provided by SRAM, BSRAM, or EDRAM.
  • the processor 121 communicates with various I/O devices 130 via a local system bus ISO.
  • Various buses may be used to connect the central processing unit 121 to any of the I/O devices 130, including a PCI bus, a PCI-X bus, or a PCI-Express bus, or a NuBus.
  • the processor 121 may use an Advanced Graphics Port (AGP) to communicate with the display 124 or the I/O controller 123 for the display 124.
  • AGP Advanced Graphics Port
  • FIG. ID depicts an embodiment of a computer 100 in which the main processor 121 communicates directly with I/O device 130b or other processors 121 ' via HYPERTRANSPORT, RAPIDIO, or INFINIBAND communications technology.
  • FIG. ID also depicts an embodiment in which local busses and direct communication are mixed: the processor 121 communicates with I/O device 130a using a local interconnect bus while communicating with I/O device 130b directly.
  • I/O devices 130a-130n may be present in the computing device 100.
  • Input devices may include keyboards, mice, trackpads, trackballs, touchpads, touch mice, multi-touch touchpads and touch mice, microphones, multi-array microphones, drawing tablets, cameras, single-lens reflex camera (SLR), digital SLR (DSLR), CMOS sensors, accelerometers, infrared optical sensors, pressure sensors, magnetometer sensors, angular rate sensors, depth sensors, proximity sensors, ambient light sensors, gyroscopic sensors, or other sensors.
  • Output devices may include video displays, graphical displays, speakers, headphones, inkjet printers, laser printers, and 3D printers.
  • Devices 130a-130n may include a combination of multiple input or output devices, including, e.g., Microsoft KINECT, Nintendo Wiimote for the WII, Nintendo WH U
  • Some devices 130a-130n allow gesture recognition inputs through combining some of the inputs and outputs. Some devices 130a-130n provides for facial recognition which may be utilized as an input for different purposes including authentication and other commands. Some devices 130a-130n provides for voice recognition and inputs, including, e.g., Microsoft KINECT, SRI for IPHONE by Apple, Google Now or Google Voice Search.
  • Additional devices 130a-130n have both input and output capabilities, including, e.g., haptic feedback devices, touchscreen displays, or multi-touch displays.
  • Touchscreen, multi-touch displays, touchpads, touch mice, or other touch sensing devices may use different technologies to sense touch, including, e.g., capacitive, surface capacitive, projected capacitive touch (PCT), in-cell capacitive, resistive, infrared, waveguide, dispersive signal touch (DST), in-cell optical, surface acoustic wave (SAW), bending wave touch (BWT), or force-based sensing technologies.
  • PCT surface capacitive, projected capacitive touch
  • DST dispersive signal touch
  • SAW surface acoustic wave
  • BWT bending wave touch
  • Some multi-touch devices may allow two or more contact points with the surface, allowing advanced functionality including, e.g., pinch, spread, rotate, scroll, or other gestures.
  • Some touchscreen devices including, e.g., Microsoft PDiELSENSE or Multi-Touch Collaboration Wall, may have larger surfaces, such as on a table-top or on a wall, and may also interact with other electronic devices.
  • Some I/O devices 130a-130n, display devices 124a-124n or group of devices may be augment reality devices. The I/O devices may be controlled by an VO controller 123 as shown in FIG. 1C.
  • the VO controller may control one or more I/O devices, such as, e.g., a keyboard 126 and a pointing device 127, e.g., a mouse or optical pen. Furthermore, an I/O device may also provide storage and/or an installation medium 116 for the computing device 100. In still other embodiments, the computing device 100 may provide USB connections (not shown) to receive handheld USB storage devices. In further embodiments, an VO device 130 may be a bridge between the system bus ISO and an external communication bus, e.g. a USB bus, a SCSI bus, a FireWire bus, an Ethernet bus, a Gigabit Ethernet bus, a Fibre Channel bus, or a Thunderbolt bus.
  • an external communication bus e.g. a USB bus, a SCSI bus, a FireWire bus, an Ethernet bus, a Gigabit Ethernet bus, a Fibre Channel bus, or a Thunderbolt bus.
  • display devices 124a-124n may be connected to I/O controller 123.
  • Display devices may include, e.g., liquid crystal displays (LCD), thin film transistor LCD (TFT-LCD), blue phase LCD, electronic papers (e-ink) displays, flexile displays, light emitting diode displays (LED), digital light processing (DLP) displays, liquid crystal on silicon (LCOS) displays, organic light-emitting diode (OLED) displays, active- matrix organic light-emitting diode (AMOLED) displays, liquid crystal laser displays, time- multiplexed optical shutter (TMOS) displays, or 3D displays. Examples of 3D displays may use, e.g.
  • Display devices 124a-124n may also be a head-mounted display (HMD).
  • display devices 124a-124n or the corresponding I/O controllers 123 may be controlled through or have hardware support for OPENGL or DIRECTX API or other graphics libraries.
  • the computing device 100 may include or connect to multiple display devices 124a-124n, which each may be of the same or different type and/or form.
  • any of the VO devices 130a-130n and/or the I/O controller 123 may include any type and/or form of suitable hardware, software, or combination of hardware and software to support, enable or provide for the connection and use of multiple display devices 124a-124n by the computing device 100.
  • the computing device 100 may include any type and/or form of video adapter, video card, driver, and/or library to interface, communicate, connect or otherwise use the display devices 124a-124n.
  • a video adapter may include multiple connectors to interface to multiple display devices 124a- 124n.
  • the computing device 100 may include multiple video adapters, with each video adapter connected to one or more of the display devices 124a- 124n. In some embodiments, any portion of the operating system of the computing device 100 may be configured for using multiple displays 124a-124n. In other embodiments, one or more of the display devices 124a-124n may be provided by one or more other computing devices 100a or 100b connected to the computing device 100, via the network 104. In some embodiments software may be designed and constructed to use another computer's display device as a second display device 124a for the computing device 100. For example, in one embodiment, an Apple iPad may connect to a computing device 100 and use the display of the device 100 as an additional display screen that may be used as an extended desktop.
  • a computing device 100 may be configured to have multiple display devices 124a-124n.
  • the computing device 100 may comprise a storage device 128 (e.g. one or more hard disk drives or redundant arrays of independent disks) for storing an operating system or other related software, and for storing application software programs such as any program related to the image classification system software 120.
  • a storage device 128 e.g. one or more hard disk drives or redundant arrays of independent disks
  • Examples of storage device 128 include, e.g., hard disk drive (HDD); optical drive including CD drive, DVD drive, or BLU-RAY drive; solid-state drive (SSD); USB flash drive; or any other device suitable for storing data.
  • Some storage devices may include multiple volatile and non-volatile memories, including, e.g., solid state hybrid drives that combine hard disks with solid state cache.
  • Some storage device 128 may be non- volatile, mutable, or read-only.
  • Some storage device 128 may be internal and connect to the computing device 100 via a bus ISO.
  • Some storage device 128 may be external and connect to the computing device 100 via a I/O device 130 that provides an external bus.
  • Some storage device 128 may connect to the computing device 100 via the network interface 118 over a network 104, including, e.g., the Remote Disk for MACBOOK AIR by Apple. Some client devices 100 may not require a nonvolatile storage device 128 and may be thin clients or zero clients 102. Some storage device 128 may also be used as an installation device 116, and may be suitable for installing software and programs. Additionally, the operating system and the software can be run from a bootable medium, for example, a bootable CD, e.g. KNOPPDC, a bootable CD for
  • GNU/Linux that is available as a GNU/Linux distribution from knoppix.net.
  • Client device 100 may also install software or application from an application distribution platform.
  • application distribution platforms include the App Store for iOS provided by Apple, Inc., the Mac App Store provided by Apple, Inc., GOOGLE PLAY for Android OS provided by Google Inc., Chrome Webstore for CHROME OS provided by Google Inc., and Amazon Appstore for Android OS and KINDLE FIRE provided by Amazon.com, Inc.
  • An application distribution platform may facilitate installation of software on a client device 102.
  • An application distribution platform may include a repository of applications on a server 106 or a cloud 108, which the clients 102a-102n may access over a network 104.
  • An application distribution platform may include application developed and provided by various developers. A user of a client device 102 may select, purchase and/or download an application via the application distribution platform.
  • the computing device 100 may include a network interface 118 to interface to the network 104 through a variety of connections including, but not limited to, standard telephone lines LAN or WAN links ⁇ e.g., 802.11, Tl, T3, Gigabit Ethernet, Infiniband), broadband connections (e.g., ISDN, Frame Relay, ATM, Gigabit Ethernet, Ethernet-over-SONET, ADSL, VDSL, BPON, GPON, fiber optical including FiOS), wireless connections, or some combination of any or all of the above.
  • standard telephone lines LAN or WAN links e.g., 802.11, Tl, T3, Gigabit Ethernet, Infiniband
  • broadband connections e.g., ISDN, Frame Relay, ATM, Gigabit Ethernet, Ethernet-over-SONET, ADSL, VDSL, BPON, GPON, fiber optical including FiOS
  • wireless connections or some combination of any or all of the above.
  • Connections can be established using a variety of communication protocols (e.g., TCP/IP, Ethernet, ARCNET, SONET, SDH, Fiber Distributed Data Interface (FDDI), IEEE 802.1 la/b/g/n/ac CDMA, GSM, WiMax and direct asynchronous connections).
  • communication protocols e.g., TCP/IP, Ethernet, ARCNET, SONET, SDH, Fiber Distributed Data Interface (FDDI), IEEE 802.1 la/b/g/n/ac CDMA, GSM, WiMax and direct asynchronous connections.
  • FDDI Fiber Distributed Data Interface
  • GSM Global System for Mobile communications
  • WiMax Worldwide Interoperability for Microwave Access
  • the network interface 118 may comprise a built-in network adapter, network interface card, PCMCIA network card, EXPRESSCARD network card, card bus network adapter, wireless network adapter, USB network adapter, modem or any other device suitable for interfacing the computing device 100 to any type of network capable of communication and performing the operations described herein.
  • SSL Secure Socket Layer
  • TLS Transport Layer Security
  • the network interface 118 may comprise a built-in network adapter, network interface card, PCMCIA network card, EXPRESSCARD network card, card bus network adapter, wireless network adapter, USB network adapter, modem or any other device suitable for interfacing the computing device 100 to any type of network capable of communication and performing the operations described herein.
  • a computing device 100 of the sort depicted in FIGs. IB and 1C may operate under the control of an operating system, which controls scheduling of tasks and access to system resources.
  • the computing device 100 can be running any operating system such as any of the versions of the MICROSOFT WINDOWS operating systems, the different releases of the Unix and Linux operating systems, any version of the MAC OS for Macintosh computers, any embedded operating system, any real-time operating system, any open source operating system, any proprietary operating system, any operating systems for mobile computing devices, or any other operating system capable of running on the computing device and performing the operations described herein.
  • Typical operating systems include, but are not limited to: WINDOWS 2000, WINDOWS Server 2012, WINDOWS CE, WINDOWS Phone, WINDOWS XP, WINDOWS VISTA, WINDOWS 7, WINDOWS RT, WINDOWS 8 and WINDOWS 10 all of which are manufactured by Microsoft Corporation of Redmond, Washington; MAC OS and iOS, manufactured by Apple, Inc. of Cupertino, California; and Linux, a freely-available operating system, e.g. Linux Mint distribution ("distro") or Ubuntu, distributed by Canonical Ltd. of London, United Kingom; or Unix or other Unix-like derivative operating systems; and Android, designed by Google, of Mountain View,
  • the computer system 100 can be any workstation, telephone, desktop computer, laptop or notebook computer, netbook, ULTRABOOK, tablet, server, handheld computer, mobile telephone, smartphone or other portable telecommunications device, media playing device, a gaming system, mobile computing device, or any other type and/or form of computing, telecommunications or media device that is capable of communication.
  • the computer system 100 has sufficient processor power and memory capacity to perform the operations described herein.
  • the computing device 100 may have different processors, operating systems, and input devices consistent with the device.
  • the Samsung GALAXY smartphones e.g., operate under the control of Android operating system developed by Google, Inc. GALAXY smartphones receive input via a touch interface.
  • the computing device 100 is a gaming system.
  • the computer system 100 may comprise a PLAYSTATION 3, or PERSONAL
  • PLAYSTATION PORTABLE PSP
  • PLAYSTATION VITA PLAYSTATION VITA device manufactured by the Sony Corporation of Tokyo, Japan
  • NINTENDO DS NINTENDO 3DS
  • NINTENDO WII or a NINTENDO WII U device manufactured by Nintendo Co., Ltd., of Kyoto, Japan
  • the computing device 100 is a digital audio player such as the Apple IPOD, IPOD Touch, and IPOD NANO lines of devices, manufactured by Apple Computer of Cupertino, California.
  • Some digital audio players may have other functionality, including, e.g., a gaming system or any functionality made available by an application from a digital application distribution platform.
  • the IPOD Touch may access the Apple App Store.
  • the computing device 100 is a portable media player or digital audio player supporting file formats including, but not limited to, MP3, WAV, M4A/AAC, WMA Protected AAC, AIFF, Audible audiobook, Apple Lossless audio file formats and .mov, .m4v, and .mp4 MPEG-4 (H.264/MPEG-4 AVC) video file formats.
  • file formats including, but not limited to, MP3, WAV, M4A/AAC, WMA Protected AAC, AIFF, Audible audiobook, Apple Lossless audio file formats and .mov, .m4v, and .mp4 MPEG-4 (H.264/MPEG-4 AVC) video file formats.
  • the computing device 100 is a tablet e.g. the IP AD line of devices by Apple; GALAXY TAB family of devices by Samsung; or KINDLE FIRE, by Amazon.com, Inc. of Seattle, Washington.
  • the computing device 100 is an eBook reader, e.g. the KINDLE family of devices by Amazon.com, or NOOK family of devices by Barnes & Noble, Inc. of New York City, New York.
  • the communications device 102 includes a combination of devices, e.g. a smartphone combined with a digital audio player or portable media player.
  • a smartphone e.g. the IPHONE family of smartphones manufactured by Apple, Inc.; a Samsung GALAXY family of smartphones manufactured by Samsung, Inc; or a Motorola DROID family of smartphones.
  • the communications device 102 is a laptop or desktop computer equipped with a web browser and a microphone and speaker system, e.g. a telephony headset.
  • the communications devices 102 are web-enabled and can receive and initiate phone calls.
  • a laptop or desktop computer is also equipped with a webcam or other video capture device that enables video chat and video call.
  • the status of one or more machines 102, 106 in the network 104 is monitored, generally as part of network management.
  • the status of a machine may include an identification of load information (e.g., the number of processes on the machine, CPU and memory utilization), of port information (e.g., the number of available communication ports and the port addresses), or of session status (e.g., the duration and type of processes, and whether a process is active or idle).
  • this information may be identified by a plurality of metrics, and the plurality of metrics can be applied at least in part towards decisions in load distribution, network traffic management, and network failure recovery as well as any aspects of operations of the present solution described herein.
  • Methods, systems, and apparatus are provided relating to generating three- dimensional (3D) images by scanning a whole tissue block.
  • Traditional processes for producing 3D image of a tissue sample can be entail high financial costs due to the number of manual processing steps required.
  • tissue samples are embedded in formalin-fixed paraffin-embedded (FFPE) blocks. Each block is cut into slices. The slices are mounted on respective glass slides, which are then individually scanned using medical imaging equipment to produce an image corresponding to each slide.
  • the 3D tissue sample is then reconstructed using image processing software based on the individual slide images. In some cases, it can take at least one week to complete this process for a single block. In some instances, the cost for processing the single block can be around $40,000.
  • a block may only be divided into 200-300 slices that become slides.
  • the spatial resolution of a 3D image generated by this type of process is limited, which inhibits the ability of a physician, pathologist or other medical care provider to accurately identify features present in the tissue sample based on the 3D image (e.g., for diagnostic purposes).
  • This disclosure provides innovative systems and methods for producing 3D images of tissue samples.
  • the solutions of this disclosure can allow for an entire block to be scanned at once, without the need to slice the block and produce glass slides. As a result, significant time and cost can be saved.
  • the resolution of the scanning techniques disclosed herein is not limited to any particular number of glass slides, and is only limited by the resolution of the scanning equipment used.
  • a 3D image produced using the techniques of this disclosure can have substantially higher resolution than images reconstructed from glass slides as described above. This can allow a physician, pathologist or other medical care provider to more accurately characterize features using the 3D image.
  • a system for generating a 3D image can include a scanning device, such as a micro computed tomography (microCT) scanner.
  • the scanning device can include hardware configured to hold an entire paraffin block in position to allow the block to be scanned.
  • the scanning device can generate a plurality of "virtual slide" images representing different layers stacked vertically within the block.
  • image processing software can use the virtual slide images to reconstruct a 3D representation of the tissue sample captured in the paraffin block.
  • the image processing software can implement novel algorithms that improve the ability of the software to register (for example, align in a horizontal direction) consecutive images produced by the scanning device.
  • the image processing software can generate the 3D image more quickly and more accurately than the conventional techniques described above.
  • 3D medical images can be generated more quickly and at lower cost, relative to conventional techniques.
  • the 3D images may have a significantly higher spatial resolution than those generated through conventional techniques. Greater spatial resolution allows a physician, pathologist or other medical care provider to more accurately characterize the tissue sample. It can be difficult or impossible to identify certain features by examining a 2D image of a tissue sample, and the 3D images that are produced using conventional techniques may not have sufficient resolution to overcome the limitations of 3D images to detect these features. For example, some tissue samples exhibit "islands" of tumor cells that are only identifiable by examining a 3D representation of the tissue sample.
  • the solutions of this disclosure can allow a physician, pathologist or other medical care provider to identify such features that may have been difficult or impossible to observe using conventional techniques, thereby providing earlier and more accurate diagnosis of tumor cells in a patient.
  • FIG. 2 is a flowchart for an example method 200 of generating a 3D image by physically sectioning a tissue block.
  • the method 200 corresponds to the traditional techniques for producing 3D medical images introduced above.
  • the method 200 includes providing a tissue block (step 205), sectioning the tissue block to generate a plurality of slices (step 210), adhering the plurality of slices to respective slides and applying a stain to the slides (step 215), serially scanning the stained slides to produce images for each respective slide (step 220), and reconstructing a 3D image of the tissue in the block based on the images for each slide (step 225).
  • the method 200 includes providing a tissue block (step 205).
  • the block can be any type or form or solid tissue pathology specimen that has been prepared for long-term preservation.
  • the tissue block can be a formalin-fixed, paraffin-embedded (FFPE) tissue block.
  • the tissue embedded in the tissue block may be a clinical biopsy or a tumor sample that is to be analyzed using medical imaging equipment.
  • the tissue block can have a thickness in the range of about 2 millimeters to about 7 millimeters.
  • the tissue sample can be fixed in a neutral buffered formalin for a period of time in a tissue cassette, and then embedded in paraffin wax.
  • a tissue sample e.g., a lung tissue sample
  • a tissue sample can be inflated with formalin and then sampled using a cylindrical cutting tool.
  • the cylindrical core tissue can then be infiltrated with paraffin using a tissue processor to create the tissue block.
  • other techniques can be used for creating or providing the tissue block.
  • the method 200 also includes sectioning the tissue block to generate a plurality of slices (step 210).
  • the tissue block may have a thickness of several millimeters, making it difficult to examine features within the center of the tissue block.
  • the tissue block is cut into thin slices.
  • the tissue block can be cut into 50-300 slices.
  • the number of slices can depend on the thickness of the tissue block.
  • the method 200 also includes adhering the plurality of slices to respective slides and applying a stain to the slides (step 215).
  • staining the slices can include any technique used to enhance contrast between features in the slices. Staining can include applying one or more stains or dyes to each slice.
  • the stain can be a hemotoxylin and eosin stain.
  • the stained tissue slices can be adhered to respective slides to produce stained slides.
  • the slides are formed from an optically clear material such as glass.
  • applying the stain and adhering each slice to a slide can be done manually by a pathologist or other physician or technician.
  • this step can take considerable time to complete.
  • preparation of stained slides can cost around $10,000 for a single tissue block. In some instances, the cost may be higher.
  • the method 200 also includes serially scanning the stained slides to produce images for each respective slide (step 220).
  • this step can be performed with the use of a scanning device such as a micro computed tomography (microCT) scanner, a magnetic resonance image (MRS) machine, or any other type of medical scanning device.
  • a scanning device such as a micro computed tomography (microCT) scanner, a magnetic resonance image (MRS) machine, or any other type of medical scanning device.
  • microCT micro computed tomography
  • MRS magnetic resonance image
  • production of the images representing each respective slide in a tissue block may have a cost of around $30,000.
  • the method 200 also includes reconstructing a 3D image of the tissue in the block based on the images for each slide (step 22S).
  • this step can be performed by imaging software.
  • the software can determine an alignment (also referred to as a registration) between consecutive images (i.e., images corresponding to slides that were adjacent to one another in the tissue block prior to sectioning of the tissue block).
  • the software can create a 3D model of the tissue block by virtually stacking the images together in the correct order, and with the correct registration.
  • the algorithms used for reconstructing the 3D image in this manner can be computationally intensive.
  • the time to complete this step can be proportional to the number of slides that were scanned to produce a respective image.
  • the method 200 may have to be repeated several times in order to image a full organ, as each tissue block may only be large enough to include a portion of the organ.
  • the total cost for performing steps 215 and 220 of the method 200 may in some instances be around $40,000.
  • the total cost for imaging a complete organ according to the method 200 can be in the range of about $600,000 to about $1.2 million.
  • the systems and methods described below can be used to produce 3D medical images in a faster and more cost-effective manner.
  • FIG. 3 A is a block diagram of a system 300 for generating a 3D image of a tissue by scanning a whole tissue block.
  • the system 300 can be a part of the image classification system 120 shown in FIG. 1C.
  • the system 300 includes a virtual slice generator 305, a digital slice stainer 310, a slice registration engine 315, a 3D reconstruction engine 320, a tissue segmentation engine 325, a tumor island detector 330, a feature detector 335, and a database 340.
  • the components of the system 300 shown in FIG. 3A can include or can be implemented using the systems and devices described above in connection with FIGS. 1 A-1D.
  • the 3D image generation system 300 and any of its components may be implemented using computing devices similar to those shown in FIGS. 1C and ID and may include any of the features of those devices, such as the CPU 121, the memory 122, the I/O devices 130a-130n, the network interface 118, etc.
  • the virtual slice generator 305, a digital slice stainer 310, a slice registration engine 315, a 3D reconstruction engine 320, a tissue segmentation engine 325, a tumor island detector 330, a feature detector 335, and a database 340 can each be implemented as a set of software instructions, computer code, or logic that performs the functionality of each of these components as described further below.
  • a digital slice stainer 310, a slice registration engine 315, a 3D reconstruction engine 320, a tissue segmentation engine 325, a tumor island detector 330, a feature detector 335, and a database 340 can each be implemented as a set of software instructions, computer code, or logic that performs the functionality of each of these components as described further below.
  • these components may instead by implemented by hardware, for example using one or more field programmable gate arrays (FPGAs) and/or one or more application- specific integrated circuits (ASICs). In some implementations, these components can be implemented as a combination of hardware and software.
  • FPGAs field programmable gate arrays
  • ASICs application- specific integrated circuits
  • the system 300 can be configured to scan an entire tissue block to create a set of virtual slices of the tissue block, and can produce a 3D image representation of the tissue block more efficiently, relative to the conventional steps and equipment described above.
  • FIG. 3B is a flow chart for an example method 360 of generating a 3D image by scanning a whole tissue block.
  • the system 300 and its components can be used to implement some of the steps of the method 360. Therefore, the functionality of the system 300 shown in FIG. 3 A is described further below in connection with the method 360 shown in FIG. 3B.
  • the method 360 includes providing a tissue block (step 365), identifying a plurality of virtual slices of the tissue block (step 370), digitally staining the plurality of virtual slices (step 375), and reconstructing a 3D image of the tissue based on the virtual slices (step 380).
  • the method 360 includes providing a tissue block (step 365).
  • this step can be performed in a manner similar to that of step 205 of the method 200 shown in FIG. 2.
  • the tissue block can be any type or form of solid tissue pathology specimen that has been prepared for long-term preservation.
  • the tissue block can be a FFPE tissue block.
  • the tissue embedded in the tissue block may be a clinical biopsy or a tumor sample that is to be analyzed using medical imaging equipment.
  • the tissue block can have a thickness in the range of about 2 millimeters to about 7 millimeters.
  • the method 360 also includes identifying a plurality of virtual slices of the tissue block (step 370).
  • this step can be performed by the virtual slice generator 305 shown in FIG. 3 A.
  • the virtual slice generator 305 can be or can include imaging equipment, such as a microCT scanner.
  • the imaging equipment can be configured to receive the whole tissue block for imaging.
  • the imaging equipment can use two-dimension (2D) radiographic imaging to generate virtual slices of the tissue block.
  • the virtual slice generator 305 (or other components of the system 300) can be used to acquire a plurality of 2D radiographs of the tissue block while the tissue block is rotated about an axis. These 2D radiographs can be processed, for example by the virtual slice generator 305, to determine a radiographic density throughout the tissue block. Based on this radiographic density data, the virtual slice generator 305 can produce a plurality of virtual slices of the tissue block.
  • the virtual slice generator 305 can generate virtual slices by imaging the whole tissue block in layers. For example, the virtual slice generator 305 can first image an upper surface of the tissue block to a create a virtual slice
  • the virtual slice generator 305 can image a layer of the tissue block just below the surface to create a second virtual slice.
  • the second virtual slice can be generated by scanning a portion of the tissue block positioned at a depth of about 1 micron to about 5 microns below the upper surface. In some implementations, other depths may be used. For example, the second virtual slice can be at a depth of between about 1 micron and about ten microns below the upper surface of the tissue block.
  • the second virtual slice can be at a depth of at least 1 micron, at least 2 microns, at least 3 microns, at least 4 microns, at least S microns, at least 6 microns, at least 7 microns, at least 8 microns, at least 9 microns, at least 10 microns, at least 15 microns, at least 20 microns, at least 25 microns, at least 50 microns, at least 60 microns, at least 70 microns, at least 80 microns, at least 90 microns, or at least 100 microns below the upper surface of the tissue block.
  • the virtual slice generator 305 can repeat this process at sequential depths throughout the thickness of the tissue block to generate an arbitrarily selected number of virtual slices.
  • the virtual slice generator 305 can store all of the virtual slices, for example in the database 340.
  • the virtual slice generator is able to produce images similar to those produced in step 220 of the method 200 described above.
  • the method 360 can save time and expense relative to the method 200.
  • producing virtual slices in this manner can help to overcome other limitations of physically slicing the tissue block to produce physical slides.
  • the thickness of physical slices can be limited, for example, by the cutting tool used and/or the dexterity of the human operator or automated equipment
  • sequential scanning of layers to produce virtual slices can be done at resolutions that may not be achievable with physical slices.
  • sequentially scanning the plurality of successive layers of the tissue block to produce images corresponding to virtual slices of the tissue block can include sequentially scanning at least 300 layers, at least 350 layers, at least 400 layers, at least 500 layers, or at least 1000 layers of the tissue block.
  • physical slices can typically be limited to around 200 or 300 slices per block, due to time constraints and physical limitations.
  • Each of the plurality of successive layers scanned to produce the virtual slices can have a thickness between one micron and five microns, which can be substantially smaller than the typical thickness of physical slices of a tissue block. In some other implementations, each virtual slice may have a thickness between about 1 micron and about ten microns.
  • each virtual slice may have a thickness of at least 1 micron, at least 2 microns, at least 3 microns, at least 4 microns, at least 5 microns, at least 6 microns, at least 7 microns, at least 8 microns, at least 9 microns, at least 10 microns, at least IS microns, at least 20 microns, at least 25 microns, at least SO microns, at least 60 microns, at least 70 microns, at least 80 microns, at least 90 microns, or at least 100 microns. In some implementations, each virtual slice may have a thickness of greater than 100 microns.
  • the method 360 also includes digitally staining the plurality of virtual slices (step 37S).
  • this step can be performed by the digital slice stainer 310 shown in FIG. 3A.
  • Digitally staining the virtual slices can produce a plurality of stained virtual slices.
  • the digital slice stainer 310 can be configured to alter each digital slice produced by the virtual slice generator 30S in a manner similar to the physical staining process described above.
  • the digital slice stainer can apply a filter or can otherwise modify or alter each virtual slice to enhance the contrast and pixel intensity values of pixels corresponding to biological structures in the slice that may be of interest.
  • the digital slice stainer 310 can be configured to alter each virtual slice in a manner that mimics physical hematoxylin and eosin staining, as described above.
  • the digital slice stainer 310 can retrieve the virtual slices from the database 340, apply the virtual stain to each slice, and then store the stained slices in the database 340.
  • the virtual slice stainer 310 can stain the virtual slices in an automated fashion. For example, an algorithm can be used to determine, for each pixel that makes up a virtual slice, a particular color for that pixel.
  • each virtual slice can include a plurality of pixels. Each pixel can have a pixel address
  • the virtual slice stainer 310 can be configured to assign, to each pixel of a virtual slice, a color value determined by converting the intensity value of the pixel to a particular color value.
  • the virtual slice stainer 310 can include a machine learning algorithm that has been trained to convert intensity values to color values. Stated in another way, the color selected for a pixel can be related to a pixel intensity value of the pixel, and the virtual slice stainer 310 can be configured to execute the algorithm that can correlate pixel intensity values of the original virtual slice with colors to be selected for corresponding pixels of the stained virtual slice.
  • An intensity value for a pixel can be a numerical representation of its brightness.
  • a white pixel can have a high intensity value, while a black pixel can have a low intensity value, and grey pixels can have intermediate intensity values.
  • the virtual slice stainer 310 can also apply a normalization technique to the pixels of the original virtual slice, for example to ensure that the intensity values for all pixels of the virtual slice have intensity values within a predetermined range.
  • the virtual slice stainer 310 can use machine learning to virtually stain each slice.
  • the algorithm applied by the virtual slice stainer 310 can be a machine learning model.
  • the results of virtually staining a slice can also be used to refine the machine learning model.
  • the virtual slice stainer 310 can stain each virtual slice by selecting a color for each pixel of the stained virtual slice.
  • the virtual slice stainer 310 can apply a digital stain to each slice using one or more color application techniques. Examples of some techniques include pseudocolor, density slicing and choropleths.
  • the color for each pixel can be selected based on that pixel's intensity value, as described above.
  • the color can be selected based on other factors as well.
  • the color selected for each pixel of a stained virtual slice can be chosen based on any combination of the color applied to virtually staining a slice.
  • the virtual slice stainer 310 can select the color for a pixel based on a radiographic density characteristic of a corresponding pixel in the virtual slice.
  • the color selected for an individual pixel can be based on a comparison of the pixel's intensity value to either a maximum pixel intensity value or a minimum pixel intensity value included in the original virtual slice.
  • the color selected for an individual pixel can also be based on characteristics of other pixels, such as a comparison of the pixel's intensity value to the intensity values of other pixels, such as one or more adjacent pixels.
  • pixels that appear darker in the original virtual slice i.e., pixels having relatively low intensity values
  • corresponding pixels having relatively darker appearances e.g., pixels having a relatively dark purple hue
  • the method 360 can include comparing stained virtual slices of a tissue block to corresponding physical slices of the tissue block. For example, after the tissue block is scanned and the system generates virtual stained slices, the tissue block can be physically separated into a plurality of physical slices. Each physical slice can correspond to one of the virtual slices. The physical slices can be stained, for example by forming a slide for each physical slice and applying a hematoxylin and eosin stain to the physical slides. As described above, the virtual slices can be stained in a manner that is intended to mimic the results of such a physical staining process. Thus, in some implementations, the stained virtual slices can be compared to the stained slides of the physical tissue block.
  • the 3D image generation system 300 can be configured to capture or receive digital images corresponding to each of the physical stained slides of the tissue block. Then, the 3D image generation system 300 can compare the stained virtual slices to the images of the stained physical slides. The 3D image generation system 300 can match the virtual slices to corresponding physical slides based on registration of the virtual slice to a scanned image of the physical slide. The 3D image generation system 300 can further match a virtual slice to a corresponding physical slide based on a position of the virtual slice relative to the tissue block and the thickness of the virtual slices as well as the thickness of the physical samples used to generate the physical slides.
  • regions or pixels of a stained virtual slice can be compared to corresponding regions or pixels of a corresponding stained slide.
  • the comparison can be carried out on a pixel-by-pixel basis.
  • the comparison can measure how closely the color or pixel intensity of the pixels of the stained virtual slides matches the color or pixel intensity of the pixels of the stained slide images.
  • the 3D image generation system 300 can use the results of this comparison to update the machine learning model used to produce the stained virtual slides. For example, the 3D image generation system 300 can feed the results of the comparison into the model as additional training data, in order to improve the model.
  • tissue block into slides may not be required in all cases.
  • the stained virtual slices can be assembled into a 3D model and/or used for diagnostic purposes without reference to any corresponding physical slides, and therefore in many instances, the tissue block may not need to be sectioned into physical slides at all.
  • sectioning the tissue block into physical slides for purposes of performing a comparison as described above may be useful for providing additional training data for the machine learning model that is used to stain the virtual slides, but may not be required for any virtual staining, diagnostic, or 3D model generation purposes.
  • the method 360 also includes reconstructing a 3D image of the tissue (step 380).
  • the 3D image can be reconstructed based on the virtual slices.
  • the slice registration engine 315, the 3D reconstruction engine 320, and the tissue segmentation engine 325 shown in FIG. 3 A can work together to perform this step.
  • the slice registration engine 315 can retrieve the stained slices from the database 340, and can determine a registration or alignment between each consecutive slice. Stated another way, the slice registration engine 315 can be configured to determine a horizontal alignment between each pair of virtual slices that correspond to sections of the tissue block adjacent to one another in a vertical direction.
  • the slice registration engine 315 can implement an algorithm that incorporates pixel intensity values and/or pixel color values for each pair of adjacent virtual slices to determine the proper registration for the virtual slices. Incorporating such information into a registration algorithm can make the process of determining registrations for each virtual slice significantly more efficient than conventional registration techniques, such as those used in traditional 3D imaging methods similar to the method 200 described above.
  • the 3D reconstruction engine 320 can use the virtual slices and the registration information determined by the slice registration engine 315 to produce a 3D model of the tissue block. For example, once the registration information is determined, the 3D
  • each virtual slice may include some area corresponding to tissue and some area corresponding to empty space or the substrate material (e.g., paraffin) used to form the solid tissue block. Because the substrate material and empty space are not of interest for purposes of the 3D model, image information corresponding to these portions can be discarded.
  • the tissue segmentation engine 325 can distinguish portions of the model corresponding to tissue from portions of the model not corresponding to tissue by examining color and intensity values for the pixels in the 3D model.
  • the digital slice stainer 310 can apply the digital stain to each virtual slice in such a manner that only alters the color of tissue cells, but does not change the color of the substrate material.
  • the tissue segmentation engine 325 can examine the 3D model and discard portions of it corresponding to pixels whose color values indicate they have not been altered by the digital staining process.
  • the tissue segmentation engine 325 may segment the stained virtual slides.
  • the tissue segmentation engine 32S can be configured to segment the stained virtual slice to identify regions of interest as well as regions that may be considered insignificant (e.g., portions of the stained virtual slice that correspond to substrate material or empty space). It should be understood that, in some implementations, the tissue segmentation engine 325 can be configured to perform segmentation using other techniques.
  • the feature detector 335 can be configured to analyze the 3D model of the tissue block to detect one or more features that may be used to identify one or more conditions or to grade a severity of the one or more conditions.
  • the feature detector 335 can be configured to analyze one or more regions of one or more individual stained virtual slices or the stained tissue block to detect certain features.
  • the feature detector 335 can perform feature detection by identifying pixel intensity values or pixel color values of one or more clusters of pixels.
  • the feature detector 335 can use the identified values to determine if the cluster of pixels corresponds to a known feature.
  • the features can identify a tissue type or fluid type.
  • the features can correspond to a region that is to be annotated for review by a medical professional.
  • the features can correspond to a particular grade of a condition.
  • a machine learning model can be trained to identify such features.
  • the feature detector 335 can include or utilize a machine learning model to identify certain features, determine a grading of the feature and generate annotations that can be used to assist a medical professional reviewing the 3D model.
  • the feature detector 335 can be configured to identify tumor regions within prostate tissue samples and further be configured to grade the tumor region that can be used to determine a severity of the tumor.
  • the feature detector 335 can determine, through unsupervised learning, additional features that may correspond to a disease or condition that previously were unknown.
  • the feature detector 335 can execute a machine learning algorithm that has been trained to identify any feature that may be of interest for detecting one or more conditions or anomalies, such as tumor regions, based on one or more stained virtual slices.
  • such features can be or can include features that are indicative of one or more tumor cells.
  • the feature detector 335 can be configured to execute a machine learning algorithm that can correlate characteristics of a stained virtual slice with a probability that the stained virtual slice includes one or more features of interest.
  • the feature detector 33S can examine the characteristics of the stained virtual slice on a pixel-by- pixel basis to determine whether each individual pixel may indicate a feature of interest, such as a tumor.
  • the pixel characteristics can be a color, an intensity, a brightness, a size, a position, or any other characteristic.
  • the feature detector 33S can also examine the characteristics of a stained virtual slice by examining groups of pixels together. For example, the feature detector 33S can examine groups of contiguous pixels to determine any characteristic of the group, such as changes in intensity or contrast levels, and can use the machine learning algorithm to determine whether such characteristics may indicate a feature of interest, such as a tumor cell.
  • the machine learning model can be trained based on annotations provided on images of physical slides or virtual slices.
  • the annotations can be provided by a medical professional, such as a pathologist, a technician among others.
  • the machine learning model can receive images of slides or slices, a corresponding label indicating that the slide or slice includes a tumor or not, and annotations that may identify the region that includes the tumor.
  • the slide or slice may not include annotations but includes a slide level classification of being tumorous or not.
  • the machine learning model can then identify one or more features from the images that are indicative of tumor regions and features that are not indicative of tumor regions. Once the machine learning model is sufficiently trained, the machine learning model can receive a new image and indicate whether or not the slide include a tumor or not, and in some embodiments, can identify the region on the image including the tumor based on the features.
  • the machine learning model can continuously learn based on feedback received on virtual slices on which the machine learning model provides an output.
  • the feature detector 335 can apply the machine learning algorithm to a stained virtual slice to determine whether the slice is likely to include a feature of interest such as a tumor cell (e.g., greater than 50% probability, based on the model).
  • a feature of interest such as a tumor cell (e.g., greater than 50% probability, based on the model).
  • a pathologist can examine either the stained virtual slice or a corresponding stained physical slide to make an independent determination of whether the stained virtual slice does in fact include a feature of interest. This result can be compared with the result generated by the feature detector 335 using the machine learning algorithm. In some implementations, a large number of such comparisons can be performed to generate a large dataset that can be used as training data to refine the machine learning algorithm or model.
  • the feature detector 335 can also be configured to identify features of interest that may exist in the 3D image generated according to the method 360. Such a feature detection technique can be more effective than examining individual virtual slices of the 3D image.
  • the feature detector 335 can be configured to examine the reconstructed 3D image in a manner similar to that described above in connection with examining individual slices.
  • the feature detector 335 can be configured to examine the 3D image as a whole, or sections of the 3D image that contain more than one individual virtual slice.
  • the feature detector 335 can separate the 3D image into a plurality of smaller 3D portions that each include image data corresponding to more than one virtual slice.
  • the feature detector 335 can execute a machine learning algorithm to determine whether any of these 3D portions (or the 3D image as a whole) includes a feature of interest Thus, the feature detector 335 may be more likely to correctly identify a 3D feature, such as a tumor island, in this manner, as compared to examining only a single virtual slice at a time.
  • the results obtained by the feature detector 335 can also be used to refine the machine learning algorithm that is applied to the 3D image.
  • the feature detector 335 can apply the machine learning algorithm to a 3D image to determine whether the 3D image is likely to include a feature of interest such as a tumor island (e.g., greater than 50% probability, based on the model).
  • a pathologist can examine either the 3D image or the tissue block or tissue sample itself to make an independent determination of whether the 3D image includes a feature of interest. This result can be compared with the result generated by the feature detector 335 using the machine learning algorithm.
  • a large number of such comparisons can be performed to generate a large dataset that can be used as training data to refine the machine learning algorithm, which can subsequently be more efficient at detecting features of interest in other 3D images. Examples of such 3D images, which can be examined by the feature detector 335 to identify various features of interest, are described further below in connection with FIGS. 4A-4D.
  • FIG. 4A illustrated is a view of a 3D model 400 of a tissue sample that can be generated according to the method 360.
  • the 3D model 400 can be formed by virtually stacking stained virtual slices of a physical tissue block.
  • FIG. 4A shows two regions that may correspond to tumor tissue.
  • the method 360 also can include using the 3D model to detect various features within the tissue block.
  • the tumor island detector 330 can be configured to detect "islands" of tumor cells within the 3D model.
  • An island of tumor cells can be, for example, a cluster of tumor cells having a thickness in the vertical direction within the 3D model, such that it is only observable by examining non-coplanar portions of the 3D model (i.e., by examining portions of the 3D model corresponding to multiple virtual slices).
  • the tumor island detector 330 can detect tumor islands, for example, by examining the color of intensity values of pixels in a set of contiguous virtual slides of the 3D model to determine whether such an island of tumor cells appears to be present.
  • the feature detector 335 can be configured to identify or locate other features that may be of interest. Such features can include any type or biological structure within the tissue block that may be relevant for diagnostic or analysis purposes.
  • FIGS. 4B-4D show different portions of the 3D model of the tissue block 400 depicted in FIG. 4A.
  • the views of FIGS. 4B-4D can help to highlight the three-dimensional nature of the tumor tissue 40S and 410 (e.g., tumor islands), which can extend through multiple virtual slices of the 3D model 400.
  • the tumor tissue 405 and 410 is not labeled in FIGS. 4B-4D.
  • the method 360 can be used to produce a 3D image of a whole tissue block without sectioning the block into physical slices or applying a physical stain to the block, thereby saving substantial time and expense relative to the method 200 shown in FIG. 2.
  • the method 360 can allow for a much larger number of virtual slices to be included in the 3D image, relative to the number of slides that can be used in the method 200. The increased number of virtual slides achieves an increase in spatial resolution of the 3D image produced by the method 360.
  • FIG. 5A shows a stained slide 500.
  • the stained slide 500 is a physical slide that can be produced, for example, by sectioning a tissue block containing a prostate tissue sample and then physically staining the slide. Arrows S02 and S04 point to features of the slide that may be of interest for diagnostic purposes.
  • FIG. SB shows a stained virtual slice 550 of the same tissue sample depicted in FIG. SA.
  • the arrows S52 and SS4 in the stained virtual slice SSO correspond to the arrows 502 and 504 respectively, of the stained slide 500.
  • visual features indicated by these arrows can exhibit higher contrast and greater visibility in the stained virtual slice SSO as compared to their appearance in the stained slide S00.
  • the stained virtual slice 550 can be a more useful diagnostic tool than the stained slide 500.
  • FIG. 6A shows a micro-CT scan 600 of a tissue sample corresponding to a great toe that exhibits an osteosarcoma.
  • the micro-CT scan 600 can be formed by digitally scanning a tissue block containing the tissue sample of the great toe.
  • FIG. 6B shows a virtually stained micro-CT 610, which corresponds to the same tissue sample depicted in FIG. 6A.
  • the virtually stained micro-CT 610 can be formed according to the method 360 of FIG. 3B.
  • FIG. 6C shows a stained physical slide 620 corresponding to the tissue sample shown in FIG. 6A.
  • the stained physical slide 620 can be produced by applying a physical stain to a physical section of the tissue block that was scanned to produce the micro-CT scan 600 of FIG. 6A.
  • visual features of the tissue sample can exhibit higher contrast and greater visibility in the micro-CT scan 600 and the virtually stained micro-CT scan 610 as compared to the appearance of the features in the stained physical slide 620.
  • the micro-CT scan 600 and the virtually stained micro-CT scan 610 can be a more useful diagnostic tool than the stained physical slide 620.
  • systems described above may provide multiple ones of any or each of those components and these components may be provided on either a standalone machine or, in some embodiments, on multiple machines in a distributed system.
  • the systems and methods described above may be implemented as a method, apparatus or article of manufacture using programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof.
  • the systems and methods described above may be provided as one or more computer-readable programs embodied on or in one or more articles of manufacture.
  • article of manufacture as used herein is intended to encompass code or logic accessible from and embedded in one or more computer-readable devices, firmware, programmable logic, memory devices (e.g., EEPROMs, ROMs, PROMs, RAMs, SRAMs, etc.), hardware (e.g., integrated circuit chip, Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), etc.), electronic devices, a computer readable non-volatile storage unit (e.g., CD-ROM, floppy disk, hard disk drive, etc.).
  • the article of manufacture may be accessible from a file server providing access to the computer-readable programs via a network transmission line, wireless transmission media, signals propagating through space, radio waves, infrared signals, etc.
  • the article of manufacture may be a flash memory card or a magnetic tape.
  • the article of manufacture includes hardware logic as well as software or programmable code embedded in a computer readable medium that is executed by a processor.
  • the computer-readable programs may be implemented in any programming language, such as LISP, PERL, C, C++, C#, PROLOG, or in any byte code language such as JAVA.
  • the software programs may be stored on or in one or more articles of manufacture as object code.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Physics & Mathematics (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne des systèmes et des procédés de production d'une image 3D à l'aide d'un bloc de tissu entier. Un bloc de tissu peut être fourni. Une pluralité de tranches virtuelles du bloc de tissu peuvent être identifiées. La pluralité de tranches virtuelles peuvent être colorées numériquement. Un modèle 3D du tissu correspondant au bloc de tissu peut être reconstruit en fonction des tranches virtuelles colorées.
PCT/US2018/049185 2017-09-01 2018-08-31 Systèmes et procédés de génération d'images médicales 3d par balayage d'un bloc de tissu entier Ceased WO2019046774A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201762553643P 2017-09-01 2017-09-01
US62/553,643 2017-09-01

Publications (1)

Publication Number Publication Date
WO2019046774A1 true WO2019046774A1 (fr) 2019-03-07

Family

ID=65527688

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2018/049185 Ceased WO2019046774A1 (fr) 2017-09-01 2018-08-31 Systèmes et procédés de génération d'images médicales 3d par balayage d'un bloc de tissu entier

Country Status (1)

Country Link
WO (1) WO2019046774A1 (fr)

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10729325B2 (en) 2012-03-19 2020-08-04 Genetic Innovations, Inc. Devices, systems, and methods for virtual staining
WO2021053035A2 (fr) 2019-09-18 2021-03-25 Inveox Gmbh Système et procédés de génération d'un modèle 3d d'un échantillon de pathologie
WO2021076757A1 (fr) * 2019-10-15 2021-04-22 Magic Leap, Inc. Système de réalité étendue prenant en charge de multiples types de dispositifs
WO2022147154A1 (fr) * 2020-12-30 2022-07-07 Leica Biosystems Imaging, Inc. Système et procédé de mise en correspondance d'échantillons histologiques en blocs et en tranches
US11386629B2 (en) 2018-08-13 2022-07-12 Magic Leap, Inc. Cross reality system
US11386627B2 (en) 2019-11-12 2022-07-12 Magic Leap, Inc. Cross reality system with localization service and shared location-based content
WO2022150554A1 (fr) * 2021-01-07 2022-07-14 Memorial Sloan Kettering Cancer Center Quantification d'états sur des images biomédicales au travers de multiples modalités de coloration à l'aide d'un cadriciel d'apprentissage profond multitâche
US11410395B2 (en) 2020-02-13 2022-08-09 Magic Leap, Inc. Cross reality system with accurate shared maps
US11551430B2 (en) 2020-02-26 2023-01-10 Magic Leap, Inc. Cross reality system with fast localization
US11562542B2 (en) 2019-12-09 2023-01-24 Magic Leap, Inc. Cross reality system with simplified programming of virtual content
US11562525B2 (en) 2020-02-13 2023-01-24 Magic Leap, Inc. Cross reality system with map processing using multi-resolution frame descriptors
US11568605B2 (en) 2019-10-15 2023-01-31 Magic Leap, Inc. Cross reality system with localization service
US11632679B2 (en) 2019-10-15 2023-04-18 Magic Leap, Inc. Cross reality system with wireless fingerprints
CN116888678A (zh) * 2021-02-26 2023-10-13 莱卡生物系统墨尔本私人有限公司 用于组织样本的混合虚拟和化学染色的系统和方法
US11789524B2 (en) 2018-10-05 2023-10-17 Magic Leap, Inc. Rendering location specific virtual content in any location
US11830149B2 (en) 2020-02-13 2023-11-28 Magic Leap, Inc. Cross reality system with prioritization of geolocation information for localization
US11900547B2 (en) 2020-04-29 2024-02-13 Magic Leap, Inc. Cross reality system for large scale environments
US11978159B2 (en) 2018-08-13 2024-05-07 Magic Leap, Inc. Cross reality system
CN118435036A (zh) * 2021-12-23 2024-08-02 帝肯贸易股份公司 用于组织切片上的位置处的裂解的3d打印掩模
US12087432B1 (en) * 2023-08-03 2024-09-10 Pramana, Inc. Apparatus and method for visualization of digitized glass slides belonging to a patient case
US12100108B2 (en) 2019-10-31 2024-09-24 Magic Leap, Inc. Cross reality system with quality information about persistent coordinate frames

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140221813A1 (en) * 2012-03-19 2014-08-07 Genetic Innovations, Inc. Devices, systems, and methods for virtual staining
US20150287194A1 (en) * 2013-02-11 2015-10-08 Definiens Ag Updating Landmarks to Improve Coregistration as Regions of Interest are Corrected
US20160093042A1 (en) * 2014-09-25 2016-03-31 Cerner Innovation, Inc. Systems for automated tissue sample processing and imaging
US20160367228A1 (en) * 2014-02-14 2016-12-22 Memorial Sloan Kettering Cancer Center System and method for providing assessment of tumor and other biological components contained in tissue biopsy samples
US20170053398A1 (en) * 2015-08-19 2017-02-23 Colorado Seminary, Owner and Operator of University of Denver Methods and Systems for Human Tissue Analysis using Shearlet Transforms

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140221813A1 (en) * 2012-03-19 2014-08-07 Genetic Innovations, Inc. Devices, systems, and methods for virtual staining
US20150287194A1 (en) * 2013-02-11 2015-10-08 Definiens Ag Updating Landmarks to Improve Coregistration as Regions of Interest are Corrected
US20160367228A1 (en) * 2014-02-14 2016-12-22 Memorial Sloan Kettering Cancer Center System and method for providing assessment of tumor and other biological components contained in tissue biopsy samples
US20160093042A1 (en) * 2014-09-25 2016-03-31 Cerner Innovation, Inc. Systems for automated tissue sample processing and imaging
US20170053398A1 (en) * 2015-08-19 2017-02-23 Colorado Seminary, Owner and Operator of University of Denver Methods and Systems for Human Tissue Analysis using Shearlet Transforms

Cited By (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11684264B2 (en) 2012-03-19 2023-06-27 Genetic Innovations, Inc. Devices, systems, and methods for virtual staining
US10729325B2 (en) 2012-03-19 2020-08-04 Genetic Innovations, Inc. Devices, systems, and methods for virtual staining
US12303233B2 (en) 2012-03-19 2025-05-20 Genetic Innovations, Inc. Devices, systems, and methods for virtual staining
US11978159B2 (en) 2018-08-13 2024-05-07 Magic Leap, Inc. Cross reality system
US11386629B2 (en) 2018-08-13 2022-07-12 Magic Leap, Inc. Cross reality system
US12307004B2 (en) 2018-10-05 2025-05-20 Magic Leap, Inc. Rendering location specific virtual content in any location
US11789524B2 (en) 2018-10-05 2023-10-17 Magic Leap, Inc. Rendering location specific virtual content in any location
WO2021053035A2 (fr) 2019-09-18 2021-03-25 Inveox Gmbh Système et procédés de génération d'un modèle 3d d'un échantillon de pathologie
WO2021053035A3 (fr) * 2019-09-18 2021-04-29 Inveox Gmbh Système et procédés de génération d'un modèle 3d d'un échantillon de pathologie
US12182928B2 (en) 2019-09-18 2024-12-31 Inveox Gmbh System and methods for generating a 3D model of a pathology sample
US12293471B2 (en) 2019-10-15 2025-05-06 Magic Leap, Inc. Cross reality system supporting multiple device types
US11995782B2 (en) 2019-10-15 2024-05-28 Magic Leap, Inc. Cross reality system with localization service
US11568605B2 (en) 2019-10-15 2023-01-31 Magic Leap, Inc. Cross reality system with localization service
US11632679B2 (en) 2019-10-15 2023-04-18 Magic Leap, Inc. Cross reality system with wireless fingerprints
US12170910B2 (en) 2019-10-15 2024-12-17 Magic Leap, Inc. Cross reality system with wireless fingerprints
US11257294B2 (en) 2019-10-15 2022-02-22 Magic Leap, Inc. Cross reality system supporting multiple device types
WO2021076757A1 (fr) * 2019-10-15 2021-04-22 Magic Leap, Inc. Système de réalité étendue prenant en charge de multiples types de dispositifs
US12100108B2 (en) 2019-10-31 2024-09-24 Magic Leap, Inc. Cross reality system with quality information about persistent coordinate frames
US11869158B2 (en) 2019-11-12 2024-01-09 Magic Leap, Inc. Cross reality system with localization service and shared location-based content
US11386627B2 (en) 2019-11-12 2022-07-12 Magic Leap, Inc. Cross reality system with localization service and shared location-based content
US12243178B2 (en) 2019-11-12 2025-03-04 Magic Leap, Inc. Cross reality system with localization service and shared location-based content
US11748963B2 (en) 2019-12-09 2023-09-05 Magic Leap, Inc. Cross reality system with simplified programming of virtual content
US12430861B2 (en) 2019-12-09 2025-09-30 Magic Leap, Inc. Cross reality system with simplified programming of virtual content
US11562542B2 (en) 2019-12-09 2023-01-24 Magic Leap, Inc. Cross reality system with simplified programming of virtual content
US12067687B2 (en) 2019-12-09 2024-08-20 Magic Leap, Inc. Cross reality system with simplified programming of virtual content
US11967020B2 (en) 2020-02-13 2024-04-23 Magic Leap, Inc. Cross reality system with map processing using multi-resolution frame descriptors
US12283012B2 (en) 2020-02-13 2025-04-22 Magic Leap, Inc. Cross reality system with prioritization of geolocation information for localization
US11562525B2 (en) 2020-02-13 2023-01-24 Magic Leap, Inc. Cross reality system with map processing using multi-resolution frame descriptors
US12499618B2 (en) 2020-02-13 2025-12-16 Magic Leap, Inc. Cross reality system with map processing using multi-resolution frame descriptors
US11830149B2 (en) 2020-02-13 2023-11-28 Magic Leap, Inc. Cross reality system with prioritization of geolocation information for localization
US11410395B2 (en) 2020-02-13 2022-08-09 Magic Leap, Inc. Cross reality system with accurate shared maps
US11790619B2 (en) 2020-02-13 2023-10-17 Magic Leap, Inc. Cross reality system with accurate shared maps
US12283011B2 (en) 2020-02-13 2025-04-22 Magic Leap, Inc. Cross reality system with accurate shared maps
US12315097B2 (en) 2020-02-26 2025-05-27 Magic Leap, Inc. Cross reality system with fast localization
US11551430B2 (en) 2020-02-26 2023-01-10 Magic Leap, Inc. Cross reality system with fast localization
US11900547B2 (en) 2020-04-29 2024-02-13 Magic Leap, Inc. Cross reality system for large scale environments
CN116802680A (zh) * 2020-12-30 2023-09-22 徕卡生物系统成像股份有限公司 用于匹配块组织学样本和切片组织学样本的系统和方法
WO2022147154A1 (fr) * 2020-12-30 2022-07-07 Leica Biosystems Imaging, Inc. Système et procédé de mise en correspondance d'échantillons histologiques en blocs et en tranches
US12541851B2 (en) 2020-12-30 2026-02-03 Leica Biosystems Imaging, Inc. System and method for matching of block and slice histological samples
WO2022150554A1 (fr) * 2021-01-07 2022-07-14 Memorial Sloan Kettering Cancer Center Quantification d'états sur des images biomédicales au travers de multiples modalités de coloration à l'aide d'un cadriciel d'apprentissage profond multitâche
US20240054639A1 (en) * 2021-01-07 2024-02-15 Memorial Sloan-Kettering Cancer Center Quantification of conditions on biomedical images across staining modalities using a multi-task deep learning framework
CN116888678A (zh) * 2021-02-26 2023-10-13 莱卡生物系统墨尔本私人有限公司 用于组织样本的混合虚拟和化学染色的系统和方法
CN118435036A (zh) * 2021-12-23 2024-08-02 帝肯贸易股份公司 用于组织切片上的位置处的裂解的3d打印掩模
US12087432B1 (en) * 2023-08-03 2024-09-10 Pramana, Inc. Apparatus and method for visualization of digitized glass slides belonging to a patient case

Similar Documents

Publication Publication Date Title
WO2019046774A1 (fr) Systèmes et procédés de génération d'images médicales 3d par balayage d'un bloc de tissu entier
US12100191B2 (en) System, method and computer-accessible medium for quantification of blur in digital images
US12260558B2 (en) Deep multi-magnification networks for multi-class image segmentation
US10685255B2 (en) Weakly supervised image classifier
US10445879B1 (en) Systems and methods for multiple instance learning for classification and localization in biomedical imaging
EP3835986B1 (fr) Systèmes et procédés de gestion d'abonnements de groupes de classification spécifiques sur la base d'actions d'un utilisateur
WO2023023171A1 (fr) Analyse intelligente automatique de tendance de groupes
US20240394408A1 (en) Systems and methods for anonymization of image data
US20200398493A1 (en) Systems and methods for 3d printing using a correction layer

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18849727

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18849727

Country of ref document: EP

Kind code of ref document: A1