WO2017201540A1 - Classification par apprentissage machine de particules ou de substances dans des images de microscopie numérique - Google Patents

Classification par apprentissage machine de particules ou de substances dans des images de microscopie numérique Download PDF

Info

Publication number
WO2017201540A1
WO2017201540A1 PCT/US2017/033876 US2017033876W WO2017201540A1 WO 2017201540 A1 WO2017201540 A1 WO 2017201540A1 US 2017033876 W US2017033876 W US 2017033876W WO 2017201540 A1 WO2017201540 A1 WO 2017201540A1
Authority
WO
WIPO (PCT)
Prior art keywords
sample
classification
images
particles
machine learning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2017/033876
Other languages
English (en)
Inventor
Benjamin Pulaski CAHOON
Ralph James YARRO
Benjamin Scott LARSON
Richard Boyd SMITH
S. Russell ZIMMERMAN
Hyrum Anderson
Paul Cram
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Techcyte Inc
Original Assignee
Techcyte Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Techcyte Inc filed Critical Techcyte Inc
Publication of WO2017201540A1 publication Critical patent/WO2017201540A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/02—Investigating particle size or size distribution
    • G01N15/0205—Investigating particle size or size distribution by optical means
    • G01N15/0227—Investigating particle size or size distribution by optical means using imaging; using holography
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N15/14—Optical investigation techniques, e.g. flow cytometry
    • G01N15/1429—Signal processing
    • G01N15/1433—Signal processing using image recognition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/01—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials specially adapted for biological cells, e.g. blood cells
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N2015/1006—Investigating individual particles for cytology
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N15/14—Optical investigation techniques, e.g. flow cytometry
    • G01N2015/1493—Particle size
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume or surface-area of porous materials
    • G01N15/10—Investigating individual particles
    • G01N15/14—Optical investigation techniques, e.g. flow cytometry
    • G01N2015/1497—Particle shape
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N35/00—Automatic analysis not limited to methods or materials provided for in any single one of groups G01N1/00 - G01N33/00; Handling materials therefor
    • G01N35/00584—Control arrangements for automatic analysers
    • G01N35/00722—Communications; Identification
    • G01N35/00871—Communications between instruments or with remote terminals
    • G01N2035/00881—Communications between instruments or with remote terminals network configurations
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N35/00—Automatic analysis not limited to methods or materials provided for in any single one of groups G01N1/00 - G01N33/00; Handling materials therefor
    • G01N35/00584—Control arrangements for automatic analysers
    • G01N35/00722—Communications; Identification
    • G01N2035/00891—Displaying information to the operator
    • G01N2035/0091—GUI [graphical user interfaces]

Definitions

  • the present disclosure relates to systems, methods, and platforms for identifying particles and more particularly relates to systems, methods, and platforms for imaging and machine learning classification or detection of particles or substances.
  • FIG. 1 illustrates a schematic diagram of a system for imaging and machine learning classification or detection of particulates or materials in accordance with the teachings and principles of the disclosure
  • FIG. 2 illustrates a schematic diagram of a sample slide in accordance with the teachings and principles of the disclosure
  • FIG. 3 illustrates is a schematic diagram illustrating a section and subsections of a digitized image obtained of a sample in accordance with the teachings and principles of the disclosure
  • FIG. 4 is a schematic block diagram illustrating operation of a classification system in accordance with the teachings and principles of the disclosure
  • FIG. 5 graphically illustrates a method for classifying or detecting particles in an image to generate a heat map or bounding boxes in accordance with the teachings and principles of the disclosure
  • FIG. 6 is a schematic flow chart diagram illustrating a method for classifying or detecting particles or materials in a sample in accordance with the teachings and principles of the disclosure
  • FIG. 7 illustrates a block diagram of an example computing device in accordance with the teachings and principles of the disclosure
  • FIG. 8 is a schematic flow chart diagram illustrating a method for classifying or detecting particles or materials in a tissue sample in accordance with the teachings and principles of the disclosure
  • FIG. 9 is a schematic flow chart diagram illustrating a method for classifying or detecting particles or materials in a bone sample in accordance with the teachings and principles of the disclosure
  • FIG. 10 is a schematic flow chart diagram illustrating a method for classifying or detecting particles or materials in a fecal sample in accordance with the teachings and principles of the disclosure
  • FIG. 11 is a schematic flow chart diagram illustrating a method for classifying or detecting particles or materials in a blood sample in accordance with the teachings and principles of the disclosure
  • FIG. 12 is a schematic flow chart diagram illustrating a method for classifying or detecting particles or materials in a urine sample in accordance with the teachings and principles of the disclosure
  • FIG. 13 is a schematic flow chart diagram illustrating a method for classifying or detecting particles or materials in a sputum sample in accordance with the teachings and principles of the disclosure
  • FIG. 14 is a schematic flow chart diagram illustrating a method for classifying or detecting particles or materials in an infectious disease sample in accordance with the teachings and principles of the disclosure
  • FIG. 15 is a schematic flow chart diagram illustrating a method for classifying or detecting particles or materials in a cervical mucus sample in accordance with the teachings and principles of the disclosure
  • FIG. 16 is a schematic flow chart diagram illustrating a method for classifying or detecting particles or materials in a vaginal fluid sample in accordance with the teachings and principles of the disclosure
  • FIG. 17 is a schematic flow chart diagram illustrating a method for classifying or detecting particles or materials in a milk sample in accordance with the teachings and principles of the disclosure
  • FIG. 18 is a schematic flow chart diagram illustrating a method for classifying or detecting particles or materials in a semen sample in accordance with the teachings and principles of the disclosure
  • FIG. 19 is a schematic flow chart diagram illustrating a method for classifying or detecting particles or materials in a geological sample in accordance with the teachings and principles of the disclosure
  • FIG. 20 is a schematic flow chart diagram illustrating a method for classifying or detecting particles or materials in a forensic sample in accordance with the teachings and principles of the disclosure
  • FIG. 21 is a schematic flow chart diagram illustrating a method for classifying or detecting particles or materials in an agriculture sample in accordance with the teachings and principles of the disclosure
  • FIG. 22 is a schematic flow chart diagram illustrating a method for classifying or detecting particles or materials in an insect or small animal sample in accordance with the teachings and principles of the disclosure
  • FIG. 23 illustrates a screenshot where jobs for imaging or classification of air particulate may be displayed, in accordance with the teachings and principles of the disclosure
  • FIG. 24 illustrates an interface for viewing and/or classification of mold spore samples, in accordance with the teachings and principles of the disclosure
  • FIG. 25 illustrates an interface for viewing and/or classification of a mold spore, in accordance with the teachings and principles of the disclosure
  • FIG. 26 illustrates an interface for viewing and/or classification of a mold spore, in accordance with the teachings and principles of the disclosure
  • FIG. 27 illustrates an interface for displaying and viewing images or air particulates at a wide zoom, in accordance with the teachings and principles of the disclosure
  • FIG. 28 illustrates an interface for displaying and viewing images or air particulates at a narrower zoom, in accordance with the teachings and principles of the disclosure.
  • FIG. 29 illustrates a schematic diagram of a system for providing a machine learning platform and marketplace in accordance with the teachings and principles of the disclosure.
  • the present application discloses systems, methods, and devices for obtaining and classifying or detecting particulates or materials using digital imaging, such as digital microscopy.
  • Information about the particles or materials present in a location or environment may be of great interest for research, medical, or other purposes.
  • particles in a body, soil, living environment, suspended in air, or any other location may be used to determine information about medical or ecology health, forensics, or the like.
  • Example sample types include tissue, bone, fecal, blood, urine, sputum, infection disease, cervical mucus, vaginal fluid, milk, semen, geological, forensics, agricultural samples, insect or small animal samples, or air particulate samples such as mold spores. Further details about sample types and their uses are provided below.
  • samples of particles or materials may be obtained and placed on a sample slide.
  • Sample slides may include slides on which a sample material can be placed for review and imaging.
  • a sample may include particulates that were suspended or located in air, liquid, soil, body or plant tissue, or on a surface in a building, furniture, appliance or other location.
  • the sample slide may include a transparent slide that may be used to protect and/or view the particulates captured in the sample. For example, a technician or user may obtain a sample and place the sample on a slide.
  • Information in addition to the slide may also be obtained.
  • a user may also provide other information about the location or environment where the particles or material were obtained. Additionally, other information may be included such as health symptoms, ages, a serial number linked to a location or customer, a type of sample, or the like.
  • the slides and other information may be received from a user, customer, technician, or other entity that has obtained and forwarded one or more samples.
  • a lab worker may receive the one or more slides and load the slides into a scanner for imaging.
  • the lab worker may scan a barcode on the slide that links the slide with other information about the slide (e.g., a customer, location, health symptoms, and sample type).
  • the barcode may be used to automate where to look on the slide to locate the particulates from the sample.
  • the barcode may identify a manufacturer, customer, or party or entity that obtained the slide because the manufacturer, customer, or party or entity that obtained the slide may indicate where the sample is actually located.
  • particulates such as mold spores for example, if you don't know where they are located on the slide.
  • the slide may be much larger than the actual sample so it is often efficient to only scan/image the portion of the slide where the particulates are actually located. Knowing the entity, customer, or slide manufacturer (or brand) may allow a scanning system to automate location and scanning of the relevant portion of the slide.
  • Samples may be imaged using a wide range of different imaging techniques and at a wide range of different zoom levels.
  • Example scanners or imagers that may be used include a digital microscope, bright-field microscope, polarized imager, phase contrast image, fluorescence imager, scanning electron microscope, dark- field microscope, or other types of scanners/imagers.
  • the scanner such as a digital microscope
  • the scanner may be used to scan or image the whole area where the sample is located (e.g., where any particulates or materials are located).
  • These obtained images may be quite large in pixel count and memory size.
  • the images may be in full -color (16 bit, 24 bit, 32 bit or more) with very high resolution (pixel count and/or dots per inch).
  • the imaging/scanning process obtains not only images of the whole area, but also images at different resolutions.
  • the sample area may be divided up into a grid of smaller sized areas, which are each imaged at a high zoom level and then multiple grid areas may be imaged together at a lower/wider zoom level. Different zoom levels may be helpful in
  • the resulting digital images may be stored or associated with a serial number identifying the location where the sample was taken, a customer that obtained the image samples, or any other information about the location, sample, type, study type, medical conditions, or the like.
  • the digital images may be stored and transmitted to a cloud storage or remote storage location for aggregation, analysis, classification, association or aggregation with other data, or the like.
  • the lab may acquire the images and upload to a file server.
  • the file server may include a listener that detects the uploading of new images and uploads those to a remote classification system for classification, storage, reporting, and/or sharing.
  • Related data may also be uploaded with the image for storage at a remote or cloud location.
  • Custody tracking of the sample, digitized images, and associated data may be provided to ensure security and accuracy of the data.
  • images, customer information, health information, or the like may be associated with a common serial number or other identifier so that correlations between various data can be determined.
  • Data stored in the remote or cloud storage may include data, including images and related data, from a large number of different labs, customers, locations, or the like.
  • the stored data may be accessible to a classification system that includes a classification model, neural network, or other machine learning algorithm.
  • the classification system may classify each image (or sample associated with the image) as including a particular type of particle.
  • the classification system may analyze each image to classify or detect particles within the images.
  • a particle may be classified as a particular type of particle, such as a particular type of tissue, crystal, cell, parasite, bacteria, mineral, pollen, or the like.
  • the classification system may generate a heat map for an image indicating which regions of the image include different types of particles.
  • the classification system may generate bounding boxes and then
  • the bounding boxes may indicate regions where there is likely something to be present to be classified or detected. This may allow analysis or processing of only portions of the image using a neural network or algorithm to locations where particles or materials are present and ignoring regions where there are not particles. For example, large regions of an image may be blank or white space where no particles are present. Processing blank or white space regions using neural networks may be a waste or computing resources.
  • the model or neural network used by the classification system may include a system trained based on human classified images or samples. For example, the neural network may be trained using supervised learning. In one embodiment, learning and training may be performed using unsupervised machine learning.
  • the classification system may provide one or more images (or portions of images) of samples to users for classification.
  • previously classified or unclassified images may be provided to one or more experts for classification.
  • the experts may provide their own classification, which may be used to either confirm or change an original classification.
  • the machine learning algorithms, models, or neural networks may be retrained based on the updated classifications to further improve machine learning models and algorithms.
  • changes to classifications for specific particles or images may be tracked. Tracked classifications may provide additional data about the accuracy of classifications and can lead to further refinement in machine learning and classification algorithms and models.
  • reports may be generated for the type of study that is being performed for a location, patient, or customer.
  • the report may be generated based on the classification of particles within images of samples, particle counts for different particle types, health conditions related to the presence or counts for specific particle types, or the like.
  • the report may be automatically generated specific to the serial number, customer, and/or location associated with the images and the corresponding sample.
  • a report may include a report for the types of particles detected, the number of particles, likely conditions in the sampled environment or patient, recommended steps to be performed, or the like.
  • the report may be provided as a general report for a specific particle type or may be more general to health or conditions for a sample environment.
  • the term "particle” is given to mean any small unit or portion of material such as dust, cells or groups of cells, fibers, small chunks of materials, organism(s), tissue, biological matter, minerals, or any other item or material discussed herein as being classified or detected. Additionally, the classification, detection, or identification of particles may include identifying a specific type of particle or condition of a specific particle or material. For example, cells may not only be identified as a specific cell type, but also as having or displaying a certain condition, such as a condition that corresponds to an abnormality or to cancer.
  • Embodiments disclosed herein may provide significant utility and benefits. For example, automated particle classification, report generation, and/or the like may significantly reduce expert time and/or errors (such as typographical errors), thereby increasing efficiency and accuracy. At least some embodiments disclosed herein enable the full classification of each particle within a whole sample. Generally, technicians do not have enough time, nor are they required to, analyze every particle or the full sample for particle type or classification. Additionally, it can take a large amount of time for a technician to perform particle analysis and classification within a sample. This time can be saved by using machine learning algorithms and/or deep neural networks for automated computer or machine learning classification. Accuracy may be increased because a greater portion of the slide (or sample) is actually analyzed and because embodiments of machine learning algorithms or models may provide greater classification accuracy for a particle type and even for a larger number of different particle types.
  • Embodiments disclosed herein further allow for the long term storage and use of samples because they are stored as digitized images and stored in a central location.
  • Machine learning algorithms may be refined based on the large corpus of data and thus improved particle identification algorithms and machine learning results may be obtained. Error in report creation may also be decreased because typographical errors by humans may be reduced or eliminated. For example, even if a generated report is modified by a human user or technician after generation, the report details may reduce the chance of filling out a report with information from an incorrect sample, customer, location, or the like.
  • the tracking of the movement of the sample, digital images, associated health or location data, changes in particle classification, or the like may lead to quicker and more accurate reporting to clients.
  • FIG. 1 is a schematic block diagram illustrating a system and environment for obtaining, classifying or detecting, and/or sharing digitized images of particle or material samples, according to one embodiment.
  • the system 100 includes an imaging system 102 and a particulate classification system 104.
  • samples may be obtained from a location or environment by a technician or professional 106.
  • the technician or professional 106 may include a person that has been trained or licensed to obtain samples.
  • the samples may include samples from any source, location, or environment disclosed herein. For example, samples related to health or medical services may be obtained from a patient or animal by a medical professional or veterinarian.
  • Samples from any of the locations, environments, or materials discussed herein may be obtained by other professionals, lay persons, or customers in any manner discussed herein or known in respective industries or fields.
  • Example methods to obtain samples include obtaining a portion of whatever material is to be imaged or processed such as bodily fluids or tissues, plant matter, soil, lake or other water sources, dust, surface samples (e.g., tape samples or swabs).
  • the sample may be stored in a container such as a vial, bag, bottle, or other container.
  • a technician or professional 106 may also obtain any other information about the sample including a source of the sample, a location, a sample type, a study or testing purpose, a customer or patient identifier, a date, temperature or weather conditions, or any other information about or related to the sample.
  • This information may be logged by the technician or professional 106 into a computer system or other database and associated with the sample, such as by associating the sample, customer, or patient identifier.
  • This information may be provided to and/or stored in the classification system 104 and/or a data store 112.
  • the technician or professional 106 may obtain the sample in response to a request or approval by a customer or patient. The customer or patient may agree that the data will be used for research/big data purposes.
  • the samples may be applied to a slide 108 for preservation, shipping, standardization, or the like.
  • the slide 108 may be provided to a lab or an imaging system 102 for imaging or digitization of the sample.
  • the lab logs the samples into the lab's software to a customer's job.
  • the customer at this point may be the original customer, patient, or the inspecting organization.
  • a customer account may be created in the labs software and/or in software for the classification system 104.
  • the lab may create a job for the sample or customer in the lab's or particulate classification system's software.
  • the software may link the sample to the customer and job based on a sample serial number.
  • an account may be created and any necessary software may be installed on a lab computer. For example, a user account with information about the lab or customer may be generated and authentication details may be provided for using the account. As another example, the classification system 104 or another party may, at times, need to uniquely identify the scanner or lab from which the digital samples will be provided.
  • the lab may prepare the slide for digitization.
  • the slide may be stained, covered with a cover slip, or the like, as necessary.
  • the slides may be loaded into the scanner (such as a digital microscope or other imaging device discussed herein).
  • the scanner such as a digital microscope or other imaging device discussed herein.
  • the slide may be loaded such that a pre-selected area will be scanned.
  • samples or slides for different sources may position the sample consistently at a specific location in the slide. This can make it easier for a technician or the imaging system 102 to determine what portion of the slide to image.
  • the particulates or materials of a sample can be very small and may not be visible to the naked eye. Thus, it can be difficult for a user to see what portion of the slide needs to be imaged.
  • the sample is a tape sample or other sample that may not have a consistent location, a user may need to select the sample size and set the focus points manually.
  • the scanner then images the sample and automatically loads the sample images up to the classification system 104 over a network 110.
  • the sample images may be stored within or associated to the account of a lab or customer account and/or may be associated with any other data gathered about or in relation to the sample.
  • the classification system 104 may receive images from the imaging system 102 and/or images from a large number of imaging systems 102 for the same or different labs.
  • the images may be stored in a data store 112.
  • a notification may indicate that there are new samples ready to be processed.
  • samples, or particles within samples may be automatically classified by a deep neural network or other machine learning classification algorithm of the classification system 104.
  • images may be fed to a neural network classifier, which outputs an indication of what type of particle(s) is/are in the image.
  • a scaled or raw subsection of an image having pixel dimensions required by the neural network or classification algorithm may be provided to the neural network or classification algorithm.
  • the same image or section of image may be processed by a plurality of neural networks to determine what types of particles are in the image.
  • the classification algorithm or network may indicate a probability that the image includes particles of a particular type. Further discussion of example methods or processes for classification is provided in relation to FIGS. 4-6 and 8-22.
  • the classification system 104 may process all images correlating to a sample imaged by the imaging system 102. For example, the classification system 104 may perform a machine learning algorithm on all images and/or all portions of the image to obtain an extremely accurate classification of most or all particles within the images. This provides considerable accuracy and completeness because human technicians generally don't have the time or ability to analyze all portions of a sample where particulates are located. With machine learning processing of each image corresponding to a sample, an extremely accurate particle count, particle type count, and the like may be achieved.
  • samples, or particles within samples may be classified or reviewed for classification by a human worker. For example, as a neural network or machine learning algorithm is being trained, a human review or initial classification of a particle may be required.
  • samples When samples are ready to review, samples may be grabbed and worked on by a technician working on a local or remote computer 114. After selecting the sample from a list needing attention, the technician is presented with the cut out particulate images. For example, the images may be cropped in order to provide an image where the particle or material is large enough to be viewed or observed by the user.
  • the technician may be able to provide an initial classification, or review and re-classify or modify a classification given by a neural network, which may then be fed back into the machine learning as new samples for further training of a model or network.
  • images may be shared by a technician with a colleague by clicking on a share option.
  • the image may be sent by the classification system 104 via email to a registered user with a link that takes them to the image.
  • any notes from the technician could be added to an image or section of the image. If the context of the image within the sample is needed for the report, the user may indicate the larger area surrounding the sub section to be saved to notes to be included as part of the report or for later use.
  • the technician may provide an indication that gives their approval that the sample has been reviewed and that a particle or material is correctly classified. In one embodiment, a technician could repeat this process until all of the slides/images pertaining to this particular customer job have been reviewed and approved.
  • the classification system 104 may include a reporting component for generating reports.
  • the classification system 104 may automatically generate a report based on the identification of particles in an image of the slide and/or a report, investigation, or examination type corresponding to the slide or sample.
  • the report may include information about particle types, particle counts, health impacts, instructions to check for certain health symptoms, potential remedies to reduce or increase particulate counts, or the like.
  • a report generation process can be started by selecting a button or menu on a list of jobs (e.g., customer jobs for sample processing).
  • the classification system 104 may generate a report template that shows the types of materials and particulates found, a customer or serial number, account information, health symptoms, or any other information related to a specific report or examination type.
  • the template report may include health concerns if these particulates are found and potential investigation/remediation steps.
  • the generated report or report template may be provided to a technician for review, confirmation, and/or transmission to a client.
  • the technician or other report writer may be able to provide input to modify the report, add pictures to the report from a list of saved pictures for that customer, save and sign a report, and/or trigger sending of the report (such as via email in a .pdf file) a company or customer that requested the report.
  • lab management software of the imaging system 102 or classification system 104 is notified that the report has been delivered.
  • the classification system 104 may export information for external report generation.
  • the classification system 104 may be able export information in a format or condition that another software program can generate a report based on the information.
  • the user should be able to export the data and pictures to lab management software of the user's choice.
  • the text information may be exported in a .csv file or .xml file, in one embodiment.
  • the images may be exported to a unique directory or file for access by the external report generation program or user.
  • a specific job may be marked as complete.
  • the information may be archived. For example, 6 months after a completion date for the job, the customer job may go to an archived state for 5, 10, 20, or more years.
  • the data may be kept indefinitely for research use or as test/research data for improving a machine learning or deep learning algorithm or model.
  • archived information may be searched based on any relevant information, such as location, barcode of slide, or any other information associated with an image, slide, customer, or job.
  • the classification system 104 is accessible from a remote location, such as via the Internet, significantly improved machine learning and classification may be possible.
  • the remotely accessible classification system 104 may train algorithms based on all images of the same type of material and thus those accessing the classification system 104 may obtain the benefits of large datasets that a party may not otherwise be able to obtain.
  • some types of examinations may not occur frequently enough within a given organization to obtain enough data to train a machine learning model or neural network. By sharing data among different locations and even organizations, even examinations that occur infrequently for each organization may occur frequently enough in combination to adequately train machine learning models or networks.
  • FIG. 2 is a schematic diagram illustrating an example sample slide 200, according to one embodiment.
  • the slide 200 includes a region 202 where the sample is located.
  • a tape sample, air sample, liquid sample, smear, or other particles from a sample may be located only in the region 202 so that it is not necessary to image any other portions of the slide 200.
  • the slide 200 also includes a label 204 which may include a barcode, serial number, brand, or other identifier. The identifier may be used to determine that the sample is located within the region 202.
  • the imaging system 102 or technician may know or determine where to position and/or scan the slide 200 based on the label 204.
  • FIG. 3 is a schematic diagram illustrating a section 300 and subsections 302, 304 of a digitized image obtained of a sample slide, according to one embodiment.
  • the section 300 may include a portion of the region 202 of the slide 200, subsection 302 includes a quarter of the section 300 and subsection 304 includes a sixteenth of section 300.
  • full resolution images of the section 300 may be obtained at different zoom levels.
  • a first zoom level image may be obtained at a wider zoom to capture the whole section 300.
  • Four images at a second zoom level covering a region the size of subsection 302 may be obtained of the section 300.
  • Sixteen images at a third zoom level covering a region the size of subsection 304 may be obtained of the section 300.
  • zoom levels may be useful in identifying or classifying or detecting different size particles.
  • mold spores are generally smaller than insects and thus may require a higher zoom level image to accurately detect or classify.
  • Other zoom levels may be used to obtain desired levels of detail based on the specific types of particles or materials that are to be detected.
  • FIG. 4 is a schematic block diagram illustrating operation of a classification system 104, according to one embodiment.
  • a network or machine learning algorithm 402 (which may also be referred to as a hypothesis), may be trained and used for identifying and classifying or detecting particles in an image.
  • the network or machine learning algorithm 402 may include a neural network, such as a deep convolution neural network, or other machine learning model or algorithm for classifying or identifying particle types.
  • the network or machine learning algorithm 402 is trained using a training algorithm 404 based on training data 406.
  • the training data 406 may include images of particles or materials and their designated classifications.
  • the training data may include images classified as including particles or materials of a first type and images classified as including particles or materials of a second type.
  • the types of the particles or materials may vary significantly based on the type of examination or report that is needed.
  • Training data for any other type of particle, material type, or the like may be used.
  • training data for any particles that are to be identified by the machine learning algorithm 402 may be provided.
  • the training algorithm 404 may train the machine learning algorithm 402.
  • the training algorithm 404 may use any type or combination of supervised or unsupervised machine learning algorithms.
  • the network or machine learning algorithm 402 may be used to identify or predict the type of particle within an image.
  • an unclassified image 410 (or previously classified image with the classification information removed) is provided to the network or machine learning algorithm 402 and the network or machine learning algorithm 402 outputs a classification 412.
  • the classification 412 may indicate a yes or no for the presence of a specific type of particle.
  • the network or machine learning algorithm 402 may be targeted to detecting whether a specific type of bacteria, particle, or material is present in the un-classified image 410.
  • the classification 412 may indicate one of many types that may be detected by the network or machine learning algorithm 402.
  • the network or machine learning algorithm 402 may provide a classification that indicates which type of particle is present in the un-classified image 410.
  • the classification may be compared to a human classification to determine how accurate the network or machine learning algorithm 402 is. If the classification 412 is incorrect, the un-classified image 410 may be assigned a classification from a human and used as training data 406 to further improve the network or machine learning algorithm 402.
  • both offline and online training of the network or machine learning algorithm 402 may be performed. For example, after an initial number of rounds of training, an initial accuracy level may be achieved.
  • the network or machine learning algorithm 402 may then be used to assist in classification with close review by human workers. As additional data comes in the data may be classified by the network or machine learning algorithm 402, reviewed by a human, and then added to a body of training data for use in further refining training of the network or machine learning algorithm 402.
  • the more the network or machine learning algorithm 402 is used the better accuracy it may achieve. As the accuracy is improved, less and less oversight of human workers may be needed.
  • FIG. 5 provides a graphical representation of classifying or detecting particles in an image 502 using a sliding window 504 to classify sub-portions of the image and generate a heat map 506.
  • the classification system 104 may generate the heat map 506 by analyzing a portion of the image 502 within a current position of the window 504 using the network or machine learning algorithm 402 to classify or detect particles within the window 504.
  • the window 504 may start at the upper left corner of the image 502 and the network or machine learning algorithm 402 may output a classification for that region.
  • the classification may include an indication of the particle type, an indication that there is a particle, and/or an indication that there is no particle at the position (e.g., blank white space).
  • the window 504 may be moved over to the right.
  • the new position may immediately border or partially overlap with the previous position of the window 504.
  • the section of the image 502 within the window 504 at the new location may be analyzed.
  • the classifications may be tracked for each position as the process is iterated to move the window 504 across and down (or in any other systematic or random pattern) so that every section of the image 502 has been within the window 504 at least once during the process.
  • a heat map 506 may be output.
  • the heat map 506 includes a plurality of regions 508 (designated by regions surrounded by dotted lines) that indicate regions where certain types of particles have been detected.
  • each region 508 may have a different classification, as determined by the network or machine learning algorithm 402.
  • the heat map 506 may indicate that a first type of particle is located at a first location and that a second type of particle is located at a second location.
  • the section of the image 502 may be analyzed using an additional sliding window of a different size. For example, larger sliding windows may be use to detect particles and/or particle types of different sizes.
  • the sample may be classified has having the types of particles identified during the classification process.
  • cropped images including the regions 508 may be generated and stored as examples of specific types of particles.
  • the cropped images may be reviewed by a human reviewer or expert for quality control or may be used on a report to illustrate the particles that are present in an image or sample.
  • the classification system 104 may generate bounding boxes and then execute classification/detection algorithms on the content of the bounding boxes. For example, the classification system 104 may use a sliding window to detect regions where particles are present to generate a plurality of bounding boxes. Since some of the bounding boxes may be duplicates or overlap, the classification system 104 may remove or combine duplicate/overlapping boundary boxes to form a subset or new set of bounding boxes. Those bounding boxes (or the pixels in those bounding boxes) may be fed separately into neural networks for processing and for identification/detection of particles. [0071] Referring now to FIG. 6, a schematic flow chart diagram of a method 600 for classifying a sample is illustrated. The method 600 may be performed by a classification system, such as the classification system 104 of FIG. 1.
  • the method 600 begins and a lab receives 602 a sample.
  • An imaging system 102 digitizes 604 the sample into one or more images at one or more zoom levels.
  • a classification system processes 606 the digital image using a machine learning prediction model to classify or detect one or more particulates or materials of the sample.
  • the classification system stores 608 the digital image with an indication of a classification for the one or more particulates or materials.
  • a report component generates 610 a report for the sample based on the classification for the one or more particulates or materials.
  • Computing device 700 may be used to perform various procedures, such as those discussed herein.
  • Computing device 700 can function as a server, a client, or any other computing entity.
  • Computing device 700 can perform various monitoring functions as discussed herein, and can execute one or more application programs, such as the application programs described herein.
  • Computing device 700 can be any of a wide variety of computing devices, such as a desktop computer, a notebook computer, a server computer, a handheld computer, tablet computer and the like.
  • Computing device 700 includes one or more processor(s) 702, one or more memory device(s) 704, one or more interface(s) 706, one or more mass storage device(s) 708, one or more Input/Output (I/O) device(s) 710, and a display device 730 all of which are coupled to a bus 712.
  • Processor(s) 702 include one or more processors or controllers that execute instructions stored in memory device(s) 704 and/or mass storage device(s) 708.
  • Processor(s) 702 may also include various types of computer-readable media, such as cache memory.
  • Memory device(s) 704 include various computer-readable media, such as volatile memory (e.g., random access memory (RAM) 714) and/or nonvolatile memory (e.g., read-only memory (ROM) 716).
  • volatile memory e.g., random access memory (RAM) 714
  • nonvolatile memory e.g., read-only memory (ROM) 716
  • Memory device(s) 704 may also include rewritable ROM, such as Flash memory.
  • Mass storage device(s) 708 include various computer readable media, such as magnetic tapes, magnetic disks, optical disks, solid-state memory (e.g., Flash memory), and so forth. As shown in FIG. 7, a particular mass storage device is a hard disk drive 724. Various drives may also be included in mass storage device(s) 708 to enable reading from and/or writing to the various computer readable media. Mass storage device(s) 708 include removable media 726 and/or non-removable media.
  • I/O device(s) 710 include various devices that allow data and/or other information to be input to or retrieved from computing device 700.
  • Example I/O device(s) 710 include cursor control devices, keyboards, keypads, microphones, monitors or other display devices, speakers, printers, network interface cards, modems, and the like.
  • Display device 730 includes any type of device capable of displaying information to one or more users of computing device 700. Examples of display device 730 include a monitor, display terminal, video projection device, and the like.
  • Interface(s) 706 include various interfaces that allow computing device 700 to interact with other systems, devices, or computing environments.
  • Example interface(s) 706 may include any number of different network interfaces 720, such as interfaces to local area networks (LANs), wide area networks (WANs), wireless networks, and the Internet.
  • Other interface(s) include user interface 718 and peripheral device interface 722.
  • the interface(s) 706 may also include one or more user interface elements 718.
  • the interface(s) 706 may also include one or more peripheral interfaces such as interfaces for printers, pointing devices (mice, track pad, or any suitable user interface now known to those of ordinary skill in the field, or later discovered), keyboards, and the like.
  • Bus 712 allows processor(s) 702, memory device(s) 704, interface(s) 706, mass storage device(s) 708, and I/O device(s) 710 to communicate with one another, as well as other devices or components coupled to bus 712.
  • Bus 712 represents one or more of several types of bus structures, such as a system bus, PCI bus, IEEE bus, USB bus, and so forth.
  • programs and other executable program components are shown herein as discrete blocks, although it is understood that such programs and components may reside at various times in different storage components of computing device 700, and are executed by processor(s) 702.
  • the systems and procedures described herein can be implemented in hardware, or a combination of hardware, software, and/or firmware.
  • one or more application specific integrated circuits (ASICs) can be programmed to carry out one or more of the systems and procedures described herein.
  • samples and identification of particles or materials within tissue may be performed.
  • Tissue samples may be obtained from biopsies, tissue scraping, brushing, liquid
  • Tissues samples may include human tissue samples or animal tissue samples.
  • the tissue samples may be applied to a slide for imaging, or may be imaged in any other manner for a respective sampling or biopsy method.
  • the classification system 104 may use one or more neural networks for classifying or detecting particles or materials in the images.
  • the classification system 104 may include neural networks that have been trained to identify particles or materials that may be present in a tissue sample from a specific region or location of a body or organ.
  • the neural networks may include neural networks configured to identify cells, particles, or materials that indicate the presence of a specific type of cancer.
  • a first neural network may identify cells corresponding to a specific type of cancer and a second neural network may identify other molecules or cells corresponding to the same or different type of cancer.
  • the types of particles searched for, and thus, the neural networks used for classification may depend on a specific type of examination.
  • the images may be provided to the classification system 104 with an indication that an examination is to be performed to detect a cancerous or benign nature of the tissue.
  • the classification system 104 may output an indication of classifications for particles within the images in the form of a heat map, table, or the like.
  • the indication of classification or identification may correspond to a specific region of an image, bounding box, or other location within the digitized images.
  • a report may be generated indicating the presence of particular particle types and their number, a diagnoses for a medical condition (e.g., such as cancer or other disease), recommended medical procedures or treatment, or the like.
  • the report may be provided to a medical professional or patient for review.
  • FIG. 8 is a schematic flow chart diagram illustrating a method 800 for classifying or detecting a tissue sample.
  • the method 800 may be performed by a classification system, such as the classification system 104 of FIG. 1.
  • the method 800 begins and a tissue sample is received 802.
  • An imaging system 102 digitizes 804 the tissue sample into one or more images at one or more zoom levels.
  • a classification system processes 806 the digital image using a machine learning prediction model to classify or detect one or more particulates or materials of the tissue sample.
  • the classification system stores 808 the digital image with an indication of a classification for the one or more particulates or materials.
  • a report component generates 810 a report for the tissue sample based on the classification for the one or more particulates or materials.
  • Bone samples may include bone matter such as bone, bone marrow, or the like from humans or any other animal. For example, bone samples from pets, wildlife, farm animals, or other animals may be analyzed for health, bone density, or disease indicators.
  • the bone samples may be applied to a slide for imaging, or may be imaged in any other manner for a respective sampling method. Once images of the bone sample are obtained, the images may be provided to a classification system 104.
  • the classification system 104 may use one or more neural networks for classifying or detecting particles or materials in the images.
  • the classification system 104 may include neural networks that have been trained to identify particles or materials that may be present in a bone sample from a human or other specific animal type.
  • the neural networks may include neural networks configured to identify blood cells such as red blood cells or white blood cells.
  • the neural networks may include neural networks configured to classify the bone as having a specific bone density.
  • the neural networks may be trained to categorize a particle in the bone sample as corresponding to a bone density above or below a specific bone density threshold.
  • the bone samples may be processed using a plurality of neural networks.
  • a first neural network may identify the presence of red blood cells and/or white blood cells and a second neural network may determine a bone density for the particles in an image.
  • the types of particles searched for, and thus the neural networks used for classification, may depend on a specific type of examination.
  • the images may be provided to the classification system 104 with an indication that an examination is to be performed to detect particles or density of the bone sample that indicate that the source patient or animal may have a specific medical condition or disease.
  • the classification system 104 may output an indication of classifications for particles within the images of the bone sample in the form of a heat map, table, or the like.
  • the indication of classification or identification may correspond to a specific region of an image, bounding box, or other location within the digitized images.
  • a report may be generated indicating the presence of particular particle types and their count, a diagnosis for a medical condition, recommended medical procedures or treatment, or the like. The report may be provided to a medical professional or patient for review.
  • FIG. 9 is a schematic flow chart diagram illustrating a method 900 for classifying a bone sample.
  • the method 900 may be performed by a classification system, such as the classification system 104 of FIG. 1.
  • the method 900 begins and a bone sample is received 902.
  • An imaging system 102 digitizes 904 the bone sample into one or more images at one or more zoom levels.
  • a classification system processes 906 the digital image using a machine learning prediction model to classify or detect one or more particulates or materials of the bone sample.
  • the classification system stores 908 the digital image with an indication of a classification for the one or more particulates or materials.
  • a report component generates 910 a report for the bone sample based on the classification for the one or more particulates or materials.
  • Fecal samples may include fecal matter from humans or any other animal. For example, fecal matter from pets, wildlife, farm animals, or other animals may be analyzed for health or disease indicators.
  • the fecal samples may be applied to a slide for imaging, or may be imaged in any other manner for a respective sampling method. Once images of the fecal sample are obtained, the images may be provided to a classification system 104.
  • the classification system 104 may use one or more neural networks for classifying or detecting particles or materials in the images.
  • the classification system 104 may include neural networks that have been trained to identify particles or materials that may be present in a fecal sample from a human or other specific animal type.
  • the neural networks may include neural networks configured to identify one or more different types of parasites.
  • the neural networks may include neural networks configured to identify blood or blood particulates.
  • the neural networks may include neural networks configured to identify ovum (egg cells) in the fecal matter.
  • a first neural network may identify the presence of blood
  • a second neural network may identify the presence of ovum
  • a third neural network may identify one or more types of parasites.
  • the types of particles searched for, and thus, the neural networks used for classification, may depend on a specific type of examination.
  • the images may be provided to the classification system 104 with an indication that an examination is to be performed to detect particles in the fecal sample that indicate that the source patient or animal may have a specific medical condition or disease.
  • the classification system 104 may output an indication of classifications for particles within the images in the form of a heat map, table, or the like.
  • the indication of classification or identification may correspond to a specific region of an image, bounding box, or other location within the digitized images.
  • a report may be generated indicating the presence of particular particle types and their count, a diagnosis for a medical condition, recommended medical procedures or treatment, or the like. The report may be provided to a medical professional or patient for review.
  • FIG. 10 is a schematic flow chart diagram illustrating a method 1000 for classifying a fecal sample. The method 1000 may be performed by a classification system, such as the classification system 104 of FIG. 1.
  • the method 1000 begins and a fecal sample is received 1002.
  • An imaging system 102 digitizes 1004 the fecal sample into one or more images at one or more zoom levels.
  • a classification system processes 1006 the digital image using a machine learning prediction model to classify or detect one or more particulates or materials of the fecal sample.
  • the classification system stores 1008 the digital image with an indication of a classification for the one or more particulates or materials.
  • a report component generates 1010 a report for the fecal sample based on the classification for the one or more particulates or materials.
  • Blood samples may include blood samples drawn from humans or any other animal. For example, blood samples from pets, wildlife, farm animals, or other animals may be analyzed for health or disease indicators.
  • the blood samples may be applied to a slide for imaging, or may be imaged in any other manner for a respective sampling method. Once images of the blood sample are obtained, the images may be provided to a classification system 104.
  • the classification system 104 may use one or more neural networks for classifying or detecting particles or materials in the images.
  • the classification system 104 may include neural networks that have been trained to identify particles or materials that may be present in a blood sample from a human or specific animal type.
  • the neural networks may include neural networks configured to identify blood cells such as red blood cells or white blood cells.
  • the neural networks may include neural networks configured to classify or detect blood cells as having abnormal shapes, such as sickle cells.
  • the neural networks may be trained to categorize a particle in the blood sample as a sickle cell or normal red blood cell.
  • the blood samples may be processed using a plurality of neural networks.
  • a first neural network may identify the presence of red blood cells
  • a second neural network may identify the presence of white blood cells
  • a third neural network may identify or detect abnormally shaped cells, such as sickle cells.
  • the types of particles searched for, and thus the neural networks used for classification, may depend on a specific type of examination.
  • the images may be provided to the classification system 104 with an indication that an examination is to be performed to detect blood cell count, blood cell abnormality, or the like.
  • the classification system 104 may output an indication of classifications for particles within the images of the blood sample in the form of a heat map, table, or the like.
  • the indication of classification or identification may correspond to a specific region of an image, bounding box, or other location within the digitized images.
  • a report may be generated indicating the presence of particular particle types and their count (e.g., red or white blood cell count), a diagnosis for a medical condition, recommended medical procedures or treatment, or the like.
  • the report may be provided to a medical professional or patient for review.
  • FIG. 11 is a schematic flow chart diagram illustrating a method 1100 for classifying a blood sample.
  • the method 1100 may be performed by a classification system, such as the classification system 104 of FIG. 1.
  • the method 1100 begins and a blood sample is received 1102.
  • An imaging system 102 digitizes 1104 the blood sample into one or more images at one or more zoom levels.
  • a classification system processes 1106 the digital image using a machine learning prediction model to classify or detect one or more particulates or materials of the blood sample.
  • the classification system stores 1108 the digital image with an indication of a classification for the one or more particulates or materials.
  • a report component generates 1110 a report for the blood sample based on the classification for the one or more particulates or materials.
  • Urine samples may include urine from humans or any other animal. For example, urine samples from pets, wildlife, farm animals, or other animals may be analyzed for health, urine density, or disease indicators. The urine samples may be applied to a slide for imaging, or may be imaged in any other manner for a respective sampling method. Once images of the urine sample are obtained, the images may be provided to a classification system 104.
  • the classification system 104 may use one or more neural networks for classifying or detecting particles or materials in the images.
  • the classification system 104 may include neural networks that have been trained to identify particles or materials that may be present in a urine sample from a human or other specific animal type.
  • the neural networks may include neural networks configured to identify blood particles or cells, such as red blood cells or white blood cells.
  • the neural networks may include neural networks configured to identify sediment (such as protein, leukocytes, blood cells, or bacteria) in the urine. For example, sediment in urine may indicate a higher likelihood of a urinary tract infection for a patient.
  • a neural network is configured to detect a crystal, mineral, kidney stone, or other object or particle in the urine.
  • the neural network for processing images of urine samples is configured to detect any particulates and may classify or detect the particulate types.
  • the urine samples may be processed using a plurality of neural networks.
  • a first neural network may identify the presence of red blood cells and/or white blood cells and a second neural network may detect crystal, mineral, or kidney stone particles.
  • the types of particles searched for, and thus the neural networks used for classification, may depend on a specific type of examination.
  • the images may be provided to the classification system 104 with an indication that an examination is to be performed to detect symptoms of a urinary tract infection or kidney stones.
  • the classification system 104 may output an indication of classifications for particles within the images of the urine sample in the form of a heat map, table, or the like.
  • the indication of classification or identification may correspond to a specific region of an image, bounding box, or other location within the digitized images.
  • a report may be generated indicating the presence of particular particle types and their count, a diagnosis for a medical condition, recommended medical procedures or treatment, or the like.
  • the report may indicate the presence of particles or sediment that correlates with an infection or kidney stones.
  • the report may be provided to a medical professional or patient for review.
  • FIG. 12 is a schematic flow chart diagram illustrating a method 1200 for classifying a urine sample.
  • the method 1200 may be performed by a classification system, such as the classification system 104 of FIG. 1.
  • the method 1200 begins and a urine sample is received 1202.
  • An imaging system 102 digitizes 1204 the urine sample into one or more images at one or more zoom levels.
  • a classification system processes 1206 the digital image using a machine learning prediction model to classify or detect one or more particulates or materials of the urine sample.
  • the classification system stores 1208 the digital image with an indication of a classification for the one or more particulates or materials.
  • a report component generates 1210 a report for the urine sample based on the classification for the one or more particulates or materials.
  • Sputum samples may include saliva, mucus, or other material from an oral cavity or respiratory tract from a human or any other animal.
  • Sputum samples may include saliva, mucus, or other material from an oral cavity or respiratory tract from a human or any other animal.
  • sputum samples from a human patient, pets, wildlife, farm animals, or other animals may be analyzed for health or disease indicators.
  • the sputum samples may be applied to a slide for imaging, or may be imaged in any other manner for a respective sampling method. Once images of the sputum sample are obtained, the images may be provided to a classification system 104.
  • the classification system 104 may use one or more neural networks for classifying or detecting particles or materials in the images of the sputum sample.
  • the classification system 104 may include neural networks that have been trained to identify particles or materials that may be present in a sputum sample from a human or other specific animal type.
  • the neural networks may include neural networks configured to identify tuberculosis (TB), streptococcal pharyngitis (i.e. strep), saliva, nasal mucus, or other particles or materials present in a sputum sample.
  • the neural networks may be configured to identify cells, bacteria, or other particles or materials that may be indicators of a disease or other medical condition of a patient.
  • the sputum samples may be processed using a plurality of neural networks that have been specifically trained or are being trained to detect a specific type of cell, bacteria, particle, or material.
  • the types of particles searched for, and thus the neural networks used for classification, may depend on a specific type of examination.
  • the images may be provided to the classification system 104 with an indication that an examination is to be performed to detect evidence of a specific disease, mucosis, or other condition.
  • the classification system 104 may output an indication of classifications for particles within the images of the sputum sample in the form of a heat map, table, or the like.
  • the indication of classification or identification may correspond to a specific region of an image, bounding box, or other location within the digitized images.
  • a report may be generated indicating the presence of particular particle types and their count, a diagnosis for a medical condition, recommended medical procedures or treatment, or the like. The report may be provided to a medical professional or patient for review.
  • FIG. 13 is a schematic flow chart diagram illustrating a method 1300 for classifying a sputum sample. The method 1300 may be performed by a classification system, such as the classification system 104 of FIG. 1.
  • the method 1300 begins and a sputum sample is received 1302.
  • An imaging system 102 digitizes 1304 the sputum sample into one or more images at one or more zoom levels.
  • a classification system processes 1306 the digital image using a machine learning prediction model to classify or detect one or more particulates or materials of the sputum sample.
  • the classification system stores 1308 the digital image with an indication of a classification for the one or more particulates or materials.
  • a report component generates 1310 a report for the sputum sample based on the classification for the one or more particulates or materials.
  • infectious disease samples may include liquid, tissue, blood, waste, or any other material from the body of a human, animal, or plant.
  • infectious disease samples may be analyzed for health, infectious disease density, or disease indicators.
  • the infectious disease samples may be applied to a slide for imaging, or may be imaged in any other manner for a respective sampling method. Once images of the infectious disease sample are obtained, the images may be provided to a classification system 104.
  • the classification system 104 may use one or more neural networks for classifying or detecting particles or materials in the images.
  • the classification system 104 may include neural networks that have been trained to identify particles or materials that may be present in an infectious disease sample.
  • the neural networks may include neural networks configured to identify particles, cells or bacteria indicating the presence of one or more of malaria, tuberculosis, bone tuberculosis, red blood cell abnormalities, gonorrhea, chlamydia, yeast, trichomoniasis, babesia, bartonella, Howell-Jolly bodies,
  • the infectious disease samples may be processed using a plurality of neural networks that each search for a different type or classification of bacteria or particle.
  • the types of particles searched for, and thus the neural networks used for classification, may depend on a specific type of examination.
  • the images may be provided to the classification system 104 with an indication that an examination is to be performed to detect particles that indicate that the source patient, animal, or plant may have a specific medical condition or disease.
  • the classification system 104 may output an indication of classifications for particles within the images of the infectious disease sample in the form of a heat map, table, or the like.
  • the indication of classification or identification may correspond to a specific region of an image, bounding box, or other location within the digitized images.
  • a report may be generated indicating the presence of particular particle types and their count, a diagnosis for a medical condition or infectious disease, recommended medical procedures or treatment, or the like. The report may be provided to a medical professional or patient for review.
  • FIG. 14 is a schematic flow chart diagram illustrating a method 1400 for classifying an infectious disease sample.
  • the method 1400 may be performed by a classification system, such as the classification system 104 of FIG. 1.
  • the method 1400 begins and an infectious disease sample is received 1402.
  • An imaging system 102 digitizes 1404 the infectious disease sample into one or more images at one or more zoom levels.
  • a classification system processes 1406 the digital image using a machine learning prediction model to classify or detect one or more particulates or materials of the infectious disease sample.
  • the classification system stores 1408 the digital image with an indication of a classification for the one or more particulates or materials.
  • a report component generates 1410 a report for the infectious disease sample based on the classification for the one or more particulates or materials.
  • Cervical mucus samples may include cervical mucus samples obtaining using a pap smear from humans or any other animal.
  • the cervical mucus samples may be applied to a slide for imaging, or may be imaged in any other manner for a respective sampling method. Once images of the cervical mucus sample are obtained, the images may be provided to a classification system 104.
  • the classification system 104 may use one or more neural networks for classifying or detecting particles or materials in the images.
  • the classification system 104 may include neural networks that have been trained to identify particles or materials that may be present in a cervical mucus sample from a human or other specific animal type.
  • the neural networks may identify cells that display potentially pre-cancerous changes, such as cervical intraepithelial neoplasia (CIN) or cervical dysplasia, the squamous intraepithelial lesion system (SIL), or the like.
  • CIN cervical intraepithelial neoplasia
  • SIL squamous intraepithelial lesion system
  • the neural networks may include neural networks configured to identify cancerous cell or particles or cells indicating the presence of cancer.
  • the cervical mucus samples may be processed using a plurality of neural networks to check for one or more different types of cancerous cells.
  • the classification system 104 may output an indication of classifications for particles within the images of the cervical mucus sample in the form of a heat map, table, or the like.
  • the indication of classification or identification may correspond to a specific region of an image, bounding box, or other location within the digitized images.
  • a report may be generated indicating the presence of particular particle types and their count, a diagnosis for a medical condition (e.g., such as the presence of cancer), recommended medical procedures or treatment, or the like.
  • the report may be provided to a medical professional or patient for review.
  • FIG. 15 is a schematic flow chart diagram illustrating a method 1500 for classifying a cervical mucus sample.
  • the method 1500 may be performed by a classification system, such as the classification system 104 of FIG. 1.
  • the method 1500 begins and a cervical mucus sample is received 1502.
  • An imaging system 102 digitizes 1504 the cervical mucus sample into one or more images at one or more zoom levels.
  • a classification system processes 1506 the digital image using a machine learning prediction model to classify or detect one or more particulates or materials of the cervical mucus sample.
  • the classification system stores 1508 the digital image with an indication of a classification for the one or more particulates or materials.
  • a report component generates 1510 a report for the cervical mucus sample based on the classification for the one or more particulates or materials.
  • Vaginal fluid samples may include fluid from vaginal discharge of a human or any other animal.
  • the vaginal fluid samples may be applied to a slide for imaging, or may be imaged in any other manner for a respective sampling method. Once images of the vaginal fluid sample are obtained, the images may be provided to a classification system 104.
  • the classification system 104 may use one or more neural networks for classifying or detecting particles or materials in the images.
  • the classification system 104 may include neural networks that have been trained to identify particles or materials that may be present in a vaginal fluid sample from a human or other specific animal type.
  • the neural networks may include neural networks configured to identify cells, bacteria, spores or other material or particles indicating the presence of bacterial vaginosis, Candida, and/or gardenella.
  • the vaginal fluid samples may be processed using a plurality of neural networks. For example, a first neural network may identify the presence of bacterial vaginosis, a second neural network may identify the presence of Candida, and a third neural network may identify the presence of gardenalla.
  • the types of particles searched for, and thus the neural networks used for classification, may depend on a specific type of examination.
  • the images may be provided to the classification system 104 with an indication that an examination is to be performed to detect particles or density of the vaginal fluid sample that indicate that the source patient or animal may have a specific medical condition or disease.
  • the classification system 104 may output an indication of classifications for particles within the images of the vaginal fluid sample in the form of a heat map, table, or the like.
  • the indication of classification or identification may correspond to a specific region of an image, bounding box, or other location within the digitized images.
  • a report may be generated indicating the presence of particular particle types and their count, a diagnosis for a medical condition, recommended medical procedures or treatment, or the like. The report may be provided to a medical professional or patient for review.
  • FIG. 16 is a schematic flow chart diagram illustrating a method 1600 for classifying a vaginal fluid sample.
  • the method 1600 may be performed by a classification system, such as the classification system 104 of FIG. 1.
  • the method 1600 begins and a vaginal fluid sample is received 1602.
  • An imaging system 102 digitizes 1604 the vaginal fluid sample into one or more images at one or more zoom levels.
  • a classification system processes 1606 the digital image using a machine learning prediction model to classify or detect one or more particulates or materials of the vaginal fluid sample.
  • the classification system stores 1608 the digital image with an indication of a classification for the one or more particulates or materials.
  • a report component generates 1610 a report for the vaginal fluid sample based on the classification for the one or more particulates or materials.
  • Milk samples may include breast milk liquid or samples from humans or any other animal.
  • the milk samples may be applied to a slide for imaging, or may be imaged in any other manner for a respective sampling method. Once images of the milk sample are obtained, the images may be provided to a classification system 104.
  • the classification system 104 may use one or more neural networks for classifying or detecting particles or materials in the images.
  • the classification system 104 may include neural networks that have been trained to identify particles or materials that may be present in a milk sample from a human or other specific animal type.
  • the neural networks may include neural networks configured to identify parasites, somatic cells (e.g., white blood cells), or the like.
  • the milk samples may be processed using a plurality of neural networks. For example, a first neural network may identify the presence of a first type of parasite, a send neural network may identify the presence of a second type of parasite, and at third neural network may identify the presence of white blood cells.
  • the types of particles searched for, and thus the neural networks used for classification, may depend on a specific type of examination.
  • the images may be provided to the classification system 104 with an indication that an examination is to be performed to detect particles or density of the milk sample that indicate that the source patient or animal may have a specific medical condition or disease.
  • the classification system 104 may output an indication of classifications for particles within the images of the milk sample in the form of a heat map, table, or the like.
  • the indication of classification or identification may correspond to a specific region of an image, bounding box, or other location within the digitized images.
  • a report may be generated indicating the presence of particular particle types and their count, a diagnosis for a medical condition, recommended medical procedures or treatment, or the like. The report may be provided to a medical professional or patient for review.
  • FIG. 17 is a schematic flow chart diagram illustrating a method 1700 for classifying a milk sample.
  • the method 1700 may be performed by a classification system, such as the classification system 104 of FIG. 1.
  • the method 1700 begins and a milk sample is received 1702.
  • An imaging system 102 digitizes 1704 the milk sample into one or more images at one or more zoom levels.
  • a classification system processes 1706 the digital image using a machine learning prediction model to classify or detect one or more particulates or materials of the milk sample.
  • the classification system stores 1708 the digital image with an indication of a classification for the one or more particulates or materials.
  • a report component generates 1710 a report for the milk sample based on the classification for the one or more particulates or materials.
  • Semen samples may include semen matter from humans or any other animal. For example, semen samples from pets, wildlife, farm animals, or other animals may be analyzed for health, semen count, or disease indicators. The semen samples may be applied to a slide for imaging, or may be imaged in any other manner for a respective sampling method. Once images of the semen sample are obtained, the images may be provided to a classification system 104. [0136] The classification system 104 may use one or more neural networks for classifying or detecting particles or materials in the images. For example, the classification system 104 may include neural networks that have been trained to identify particles or materials that may be present in a semen sample from a human or other specific animal type.
  • the neural networks may include neural networks configured to identify sperm morphology or other indications of fertility of the source patient or animal.
  • the semen samples may be processed using a plurality of neural networks. For example, a first neural network may identify the presence of normal sperm and a second neural network may determine a presence of abnormal sperm.
  • the types of particles searched for, and thus the neural networks used for classification, may depend on a specific type of examination.
  • the images may be provided to the classification system 104 with an indication that an examination is to be performed to detect particles or sperm count that may indicate that the source patient or animal may have a specific medical condition or disease.
  • the classification system 104 may output an indication of classifications for particles within the images of the semen sample in the form of a heat map, table, or the like.
  • the indication of classification or identification may correspond to a specific region of an image, bounding box, or other location within the digitized images.
  • a report may be generated indicating the presence of particular particle types and their count (e.g., sperm count or percentage of abnormal sperm), a diagnosis for a medical condition, recommended medical procedures or treatment, or the like.
  • the report may be provided to a medical professional or patient for review.
  • FIG. 18 is a schematic flow chart diagram illustrating a method 1800 for classifying a semen sample.
  • the method 1800 may be performed by a classification system, such as the classification system 104 of FIG. 1.
  • the method 1800 begins and a semen sample is received 1802.
  • An imaging system 102 digitizes 1804 the semen sample into one or more images at one or more zoom levels.
  • a classification system processes 1806 the digital image using a machine learning prediction model to classify or detect one or more particulates or materials of the semen sample.
  • the classification system stores 1808 the digital image with an indication of a classification for the one or more particulates or materials.
  • a report component generates 1810 a report for the semen sample based on the classification for the one or more particulates or materials.
  • identification of particles or materials within geological samples may be performed.
  • Geological samples may include dirt, soil, rock, oil or other liquid or other material from the earth or other geological or ecological location.
  • the geological samples may be applied to a slide for imaging, or may be imaged in any other manner corresponding to a respective sampling method. Once images of the geological sample are obtained, the images may be provided to a classification system 104.
  • the classification system 104 may use one or more neural networks for classifying or detecting particles or materials in the images.
  • the classification system 104 may include neural networks that have been trained to identify particles or materials that may be present in a geological sample.
  • the neural networks may include neural networks configured to identify dinoflagellates, specific mineral or crystal types, oil, types of soil particles (sand, clay, or biological matter), or the like.
  • the geological samples may be processed using a plurality of neural networks. For example, a first neural network may identify the presence of oil particles or molecules and a second neural network may determine the presence of specific types of minerals or crystals. The types of particles searched for, and thus the neural networks used for classification, may depend on a specific type of examination.
  • the classification system 104 may output an indication of classifications for particles within the images of the geological sample in the form of a heat map, table, or the like.
  • the indication of classification or identification may correspond to a specific region of an image, bounding box, or other location within the digitized images.
  • a report may be generated indicating the presence of particular particle types and their count, a soil health condition or a specific purpose (e.g., crops), a recommended remediation procedure, or the like.
  • the report may be provided to a farmer or scientist for review.
  • FIG. 19 is a schematic flow chart diagram illustrating a method 1900 for classifying a geological sample.
  • the method 1900 may be performed by a classification system, such as the classification system 104 of FIG. 1.
  • the method 1900 begins and a geological sample is received 1902.
  • An imaging system 102 digitizes 1904 the geological sample into one or more images at one or more zoom levels.
  • a classification system processes 1906 the digital image using a machine learning prediction model to classify or detect one or more particulates or materials of the geological sample.
  • the classification system stores 1908 the digital image with an indication of a classification for the one or more particulates or materials.
  • a report component generates 1910 a report for the geological sample based on the classification for the one or more particulates or materials. Forensics
  • identification of particles or materials within forensic samples may be performed.
  • Forensic samples may include forensic matter from a crime scene or other forensic scene or investigation.
  • forensic samples from carpet, furniture, floors, clothes, shoes, tires, jewelry, hair, mail, or the like may be obtained for legal, historical, or other purposes.
  • the forensic samples may be applied to a slide for imaging, or may be imaged in any other manner for a respective sampling method.
  • camera images using a conventional camera may be obtained. For example, camera images of tire tracks, fingerprints, shoe footprints, tracks, or other evidence may be taken and analyzed. Once images of the forensic sample are obtained, the images may be provided to a classification system 104.
  • the classification system 104 may use one or more neural networks for classifying or detecting particles or materials in the images.
  • the classification system 104 may include neural networks that have been trained to identify particles or materials that may be present in a forensic sample.
  • the neural networks may include neural networks configured to identify drugs, anthrax, blood, minerals, fibers, clothing, shoe rubber, crystals, semen, hair, plant matter, biological mater, or other particles or substances.
  • the neural networks may identify a type of particle or material.
  • neural networks or matching algorithms may be used to match drugs, anthrax, blood, minerals, fibers, clothing, shoe rubber, crystals, semen, hair, plant matter, biological mater, or other particles or substances at one location with samples or material obtained from another location.
  • shoeprints, shoe patterns, fingerprints, tires (tire tracks), fibers, clothing, hair, semen, blood, or other mater may be matched with those of a subject or suspect.
  • an image of a forensic sample and a reference image (e.g., of known material gathered from another location or suspect) may be fed into a neural network for detecting the similarity or matching of the particles, materials, or items.
  • the types of particles searched for, and thus the neural networks used for classification, may depend on a specific type of examination.
  • the images may be provided to the classification system 104 with an indication that an examination is to be performed to detect particles or density of the forensic sample that indicate that the source patient or animal may have a specific medical condition or disease.
  • the classification system 104 may output an indication of classifications for particles within the images of the forensic sample in the form of a heat map, table, or the like.
  • the indication of classification or identification may correspond to a specific region of an image, bounding box, or other location within the digitized images.
  • a report may be generated indicating the presence of particular particle types and their count, a similarity between particles, materials, or items, or the like. The report may be provided to an investigator or customer for review.
  • FIG. 20 is a schematic flow chart diagram illustrating a method 2000 for classifying a forensic sample.
  • the method 2000 may be performed by a classification system, such as the classification system 104 of FIG. 1.
  • the method 2000 begins and a forensic sample is received 2002.
  • An imaging system 102 digitizes 2004 the forensic sample into one or more images at one or more zoom levels.
  • a classification system processes 2006 the digital image using a machine learning prediction model to classify or detect one or more particulates or materials of the forensic sample.
  • the classification system stores 2008 the digital image with an indication of a classification for the one or more particulates or materials.
  • a report component generates 2010 a report for the forensic sample based on the classification for the one or more particulates or materials.
  • identification of particles or materials within agriculture samples may be performed.
  • Agriculture samples may include agriculture matter from soils (e.g., soil samples), dirt, fertilizers, or other materials used during the planting and growing of crops.
  • the agriculture samples may be applied to a slide for imaging, or may be imaged in any other manner for a respective sampling method. Once images of the agriculture sample are obtained, the images may be provided to a classification system 104.
  • the classification system 104 may use one or more neural networks for classifying or detecting particles or materials in the images.
  • the classification system 104 may include neural networks that have been trained to identify particles or materials that may be present in an agriculture sample.
  • the neural networks may include neural networks configured to particles (or organisms) such as nematodes, pollen, bacteria, protozoa, cancerous cells, or the like in a soil sample or other agriculture sample.
  • the agriculture samples may be processed using a plurality of neural networks. For example, a first neural network may identify the presence of nematodes, a second neural network may identify the presence of pollen, and a third neural network may identify the presence of bacteria, or the like..
  • the types of particles searched for, and thus the neural networks used for classification, may depend on a specific type of examination.
  • the classification system 104 may output an indication of classifications for particles within the images of the agriculture sample in the form of a heat map, table, or the like.
  • the indication of classification or identification may correspond to a specific region of an image, bounding box, or other location within the digitized images.
  • a report may be generated indicating the presence of particular particle types and their count, a diagnosis for soil or plant health, a recommended remediation procedure or soil treatment, or the like.
  • FIG. 21 is a schematic flow chart diagram illustrating a method 2100 for classifying an agriculture sample.
  • the method 2100 may be performed by a classification system, such as the classification system 104 of FIG. 1.
  • the method 2100 begins and an agriculture sample is received 2102.
  • An imaging system 102 digitizes 2104 the agriculture sample into one or more images at one or more zoom levels.
  • a classification system processes 2106 the digital image using a machine learning prediction model to classify or detect one or more particulates or materials of the agriculture sample.
  • the classification system stores 2108 the digital image with an indication of a classification for the one or more particulates or materials.
  • a report component generates 2110 a report for the agriculture sample based on the classification for the one or more particulates or materials. Insect or Small Animal
  • identification of particles or materials within insect or small animal samples may be performed.
  • Insect or small animal samples may include full bodies, portions of bodies, larvae, eggs, or the like of insects or small animals found in soil, air, water, food, plant, ecological environment, or other location.
  • the samples may include adult, egg, larvae, or other life stages of insects or small animals.
  • the insect or small animal samples may be applied to a slide for imaging, or may be imaged in any other manner for a respective sampling method.
  • the insect/animal bodies, eggs, larvae or the like may be (knowingly or unknowingly) gathered separately or within another material, such as a dirt, liquid, or other material.
  • the images may be provided to a classification system 104.
  • the classification system 104 may use one or more neural networks for classifying or detecting particles or materials in the images.
  • the classification system 104 may include neural networks that have been trained to identify the body, the portions of a body of an insect or small animal, or the eggs or larvae within the insect or small animal sample.
  • the neural networks may include neural networks configured to identify adult insects, eggs, larvae.
  • neural networks may be used to identify an adult mosquito, a mosquito egg or larvae, or the like.
  • a neural network may distinguish between a male and female mosquito (or other insect or animal).
  • the neural networks may include neural networks configured to identify adult insects (or their larvae or eggs) of different types. For example, mosquitoes, bed bugs, tics, or other insects may be identified and classified.
  • the neural networks may include neural networks configured to identify and classify or detect small animals, such as nematodes, and/or their eggs.
  • the neural networks may include neural networks configured to classify or detect the insect or small animal as of a specific type so that different types of undesirable insects or animals can be distinguished from benign or beneficial insects or animals.
  • the insect or small animal samples may be processed using a plurality of neural networks.
  • a first neural network may identify the presence of a specific type of mosquito or insect
  • second neural network identifies the presence of a specific type mosquito or insect larva or egg
  • a third neural network may identify the presence of a specific type of animal.
  • the types of particles searched for, and thus the neural networks used for classification, may depend on a specific type of examination.
  • the images may be provided to the classification system 104 with an indication that an examination is to be performed to detect mosquito eggs or larvae.
  • the classification system 104 may output an indication of classifications for particles within the images of the insect or small animal sample in the form of a heat map, table, or the like.
  • the indication of classification or identification may correspond to a specific region of an image, bounding box, or other location within the digitized images.
  • a report may be generated indicating the presence of particular particle types and their count (e.g., mosquito larvae count), recommended remediation procedures or treatment for a location, or the like.
  • FIG. 22 is a schematic flow chart diagram illustrating a method 2200 for classifying an insect or small animal sample.
  • the method 2200 may be performed by a classification system, such as the classification system 104 of FIG. 1.
  • the method 2200 begins and an insect or small animal sample is received 2202.
  • An imaging system 102 digitizes 2204 the insect or small animal sample into one or more images at one or more zoom levels.
  • a classification system processes 2206 the digital image using a machine learning prediction model to classify or detect one or more particulates or materials of the insect or small animal sample.
  • the classification system stores 2208 the digital image with an indication of a classification for the one or more particulates or materials.
  • a report component generates 2210 a report for the insect or small animal sample based on the classification for the one or more particulates or materials.
  • Particles suspended in the air can have significant health impacts.
  • mold spores, pet dander, construction fibers or dust, pollen or other plant matter, insects, microorganisms, or other items or materials may become suspended in the air and may come in contact with or be inhaled by a human or pet.
  • detection of air particulates may be of great importance in diagnosing safety of a building or the health of a patient.
  • Samples may be obtained where particulates such as mold spores, pet dander, construction fibers or dust, pollen or other plant matter, insects, microorganisms, or the like are located.
  • a single mold spore may be much smaller than insects or dust or fibers from construction.
  • Air particulate samples may be obtained and associated with health symptoms or health outcomes of a patient.
  • the air quality collection samples may include air collection samples, tape samples, or any other air particulate sample.
  • the air quality collection samples may include particulates that were suspended in the air and were captured in a filter, on tape, or on a surface of an input or output of an air duct.
  • the particulates may include one or more of a mold spore, insect, dust or fiber material, pollen, pet dander, plant matter, microorganisms, or the like. Reports may be generated for mold, pet dander, insects, allergens, or the like.
  • Sample media for slides or air particulate samples are available from Air-o-CellTM or AllergencoTM, for example.
  • the systems may distinguish between different mold or sprore types, such as aseospore, basidiospore, or the like.
  • FIG. 23 illustrates a screenshot where jobs for imaging or classification of air particulate may be displayed, according to one embodiment. For example, an entry for a slide to be imaged, or for images of a slide to be classified, may be shown in the interface of the screenshot.
  • FIG. 24 illustrates an interface for viewing and/or classification of mold spore samples, according to one embodiment.
  • the images may show cropped images of particles classified as a particular type of mold spore.
  • the list of spore type names on the left may indicate the classifications of particles (or spores) in the sample.
  • FIG. 25 illustrates an interface for viewing and/or classification of a mold spore, according to one embodiment.
  • a user may be able to select a name of a mold spore and see example images of that type of spore.
  • a user may be able to view a mold spore (a series of images of mold spores) and provide a classification for each one.
  • an expert may be able to provide a classification for a particulate to error check and/or train a machine learning algorithm.
  • FIG. 26 illustrates an interface for viewing and/or classification of a mold spore, according to one embodiment.
  • FIG. 27 illustrates an interface for displaying and viewing images or air particulates at a wide zoom, according to one embodiment.
  • the wide zoom may show an image of a whole sample.
  • FIG. 28 illustrates an interface for displaying and viewing images or air particulates at a narrower zoom, according to one embodiment.
  • the narrow zoom may show an image of a single particle or spore within the sample.
  • the zoom level of a narrow zoom may vary significantly depending on the size of the particle to be viewed. For example, spores, dust particulates, construction fibers, insects, or the like may vary significantly in size.
  • Applicants have recognized that online tools and infrastructure combined into a platform will make it possible for domain experts who have limited or no knowledge of artificial intelligence programming to use their image data, analysis data, and domain expertise to create and monetize trained models for use in image and/or data processing.
  • the platform may have a built-in storage, analytic models, training algorithms, viewers, training labeling tools, and data change tracking tools that facilitate preparation and shared usage of tools for training models and developing analytics.
  • sorting tools for sorting data in preparation for or during training can significantly speed up the training and data preparation process.
  • Current machine learning solutions require humans to manually mark every feature that they want to go into the training set. Presumably, other people have or will write or develop computer vision programs or tools that mark some of the features to try to speed this process.
  • no solutions are known that allow the sorting of this output to speed the classification and elimination of improperly classified markings.
  • Sorting the output on many different features allows at least an order of magnitude increase in speed for creating training data. Re-using existing algorithms to create a first-pass data set allows multiple orders of magnitude increase of creating training data. This increased performance on a platform can allow ideas to go-to-market faster, drastically reduce time taken to build training data, and/or increase sales by allowing other groups to reuse and resell algorithms.
  • Embodiments of machine learning platforms and included tools may be directed to particle classification or detection.
  • microscopy images or other images of particles may be obtained for and examined for extremely fast particle classification.
  • detecting or classifying particles based on images can be extremely fast (e.g., from sub second to minutes) when compared to other detection techniques, such as chemical or DNA testing for biological particles (which may take hours, days, or weeks and require a physical sample at the testing site).
  • FIG. 29 is a schematic diagram illustrating usage, operation, and components of a machine learning platform 2900.
  • the platform 2900 may include infrastructure for storing and/or processing microscopy images or other data using machine learning algorithms and/or computer vision.
  • the platform 2900 may include data storage, processing devices, and software tools to allow a user to access the platform 2900 and its services via a network, such as the Internet.
  • the platform 2900 may store and execute computer code for receiving, storing, and processing microscopy images.
  • the platform 2900 may also include specialized hardware for deep learning.
  • the platform 2900 may include hardware neural networks and nodes which may facilitate improved speed and versatility for machine learning and neural network models.
  • the platform 2900 includes viewing tools for viewing and navigating microscopy images.
  • the platform 2900 may include storage or special purpose for hardware for storing or accessing machine learning algorithms or models.
  • the platform 2900 may be accessible to remote computing or communication devices via an application program interface (API).
  • API application program interface
  • the API may allow users to upload data, trigger analytics or classifications, or perform any other function or service provided by the platform 2900.
  • the platform 2900 includes storage 2902, 2904 for storing data for training or for processing.
  • the storage 2902, 2904 may be used to store image data such as microscopy image data of slides or other images of particles.
  • users of the platform may upload image data for training or classification.
  • the storage 2902, 2904 may be used to store classification data, such as annotations for images.
  • the annotations may include an indication of a known class for an image or particle within an image.
  • the storage 2902, 2904 may be also be used to store other data, such as numerical, text, spreadsheet, field, settings, or other data. Processing and training may use not only image data but any of the available stored data. Thus, users may be able to upload all associated data and not just images of particles.
  • the storage 2902, 2904 may be used to store trained models. For example, the results of a training procedure may be stored for use by the same user or may be shared by the user for usage by other users.
  • the platform 2900 may also include tools for the processing, training, browsing, or usage of data.
  • the platform includes data creation tools.
  • the data creation tools may include training data labeling tools for generating or assisting in the generation of labels or annotations for data that is to be used in a training set.
  • the data creation tools provide semi- autonomous creation of training data. For example, a user may upload their data to be processed by existing algorithms that find and mark many features available in a microscopy image.
  • the algorithms may include platform included algorithms and/or algorithms generated by a third-party, such as by a user of the platform 2900.
  • the semi- autonomous creation tools may automatically determine locations of particles (e.g., using the blob detector tools), as opposed to blank or background locations where no particles are present.
  • the semi-autonomous creation tools generate preliminary classifications or bounding boxes and present them to a user for confirmation or modification. This can significantly reduce labeling or annotating time as a user may be able to quickly see and confirm/modify without searching through an image for a particle to label or annotate.
  • the semi-autonomous data creation tools may sort training data.
  • the data creation tools may perform preliminary processing to detect attributes or features of different particles and then provide a sorted output based on the attributes or features.
  • the data creation tools may find particles (e.g., locations where there is something that is not background), detect attributes or features in the locations that contain the particles, and then sort the found particles based on the attributes or features. For example, automatically box all the particles (non-background) in an image, and then allow for sorting based on any of the above.
  • Example attributes or features may include a size, saturation, hue, density, or any other feature or attribute of the pixels or image of the particle. Even these example attributes or features can significantly improve the training process because they reduce the amount of time a user must spend classifying particles as related or similar particles are grouped before labeling/annotation.
  • the data creation tools may also include manual training tools.
  • the manual training tools may provide an image of a particle to a user for viewing and receive an indication of a class or type of the particle from the user.
  • the user may also be able to manually create new classifications which may be parent or child classifications for other classifications.
  • a user may modify a previously provided classification (e.g., a classification provided by the system or by a different human user) to reclassify or correct the classification of a feature or particle.
  • the classification process may be guided by the platform 2900 to reduce the amount of human input or thought needed to create training data.
  • the platform 2900 may provide image cropper tools for cropping images, such as to remove background space or to adjust a size of a cropped image. For example, a user may be able to view a particle within a bounding box and adjust the size of the bounding box to include or exclude a desired amount of background.
  • the platform 2900 may provide blob detector tools which detect the presence of "blobs" or particles in an image, e.g. anything that is not background, such as the white background in a slide image where not particles or material is present.
  • the platform 2900 provides a bounding box and data annotation suite that creates bounding boxes around a detected particle and stores an annotation or label for what is inside the bounding box.
  • the bounding box (e.g., the pixels of an image that belong to the bounding box) and the annotation may be human provided or computer generated/classified.
  • the bounding box and data annotation suite may provide tools for adjusting, creating, or deleting bounding boxes as well as modifying, creating, or deleting annotations or labels for the contents of the bounding box.
  • the platform 2900 may also include model creation tools that may be used to generate or train a model.
  • the model creation tools may include stored training algorithms, neural network layouts, or algorithm templates that may be selected by a user or analytic as a model for performing a desired purpose.
  • the model creation tools may include an indication of input parameters and desired output parameters. These input and output parameters may be provided a human user, training data, or the like. Based on this data, the model creation tools may suggest or receive a selection of a base model or structure (e.g., a neural network structure) that best fits the classification problem.
  • the model creation tools may also include training algorithms for training a selected model.
  • the model creation tools may allow a user to select a training algorithm, parameters for the training algorithm, and/or training data to be used to train a model.
  • the model creation tools may also perform the training algorithm on a selected model to produce a trained model.
  • a resulting trained model (and/or the parameters for nodes or like in the trained model) may be stored for later usage in a data store 2902, 2904.
  • the platform 2900 also includes payment processing tools.
  • the payment processing tools may track usage and/or sharing by users to obtain payment from and/or provide payment to users. For example, users may pay based on units for processing time, storage, images processed, or the like.
  • the payment processing tools may track the utilization of a customer of the platform and then bill the customer for the usage.
  • the payment processing tools may identify the source or owner of a resource, such as of a trained model, training set, or the like, and forward a percentage of revenue for use of that resource to the owner or source.
  • users may be able to generate trained models and share them to receive payment. This may provide another source of revenue for users or entities with domain knowledge, large training sets, accurate data models, or the like.
  • customers of the platform 2900 may use platform tools to generate trained models, training data, algorithms, analytics, or the like. After generating the models, training data, algorithms, and analytics, the generated information may reside in the platform and thus may be shared with other users.
  • the platform 2900 may include sharing/publishing tools.
  • the platform 2900 may provide an online marketplace where users can search or publish machine learning tools.
  • User may select a machine learning tool, such as a training data set, trained model, or the like to publish for access/purchase by other users.
  • the user may see a list of available machine learning tools and an indication of which are private and which are shared.
  • the user can manage what is private and shared and easily publish machine learning tools for profit.
  • the platform 2900 may include workflow and reporting tools.
  • Blocks 2906-2914 represent a method, using the platform 2900, that may be performed by a user (3 rd Party Provider) to provide a trained model for use by other users of the platform 2900.
  • the method begins and a user uploads 2906 images to the platform 2900.
  • the user may upload 2906 images owned or gathered by the user.
  • the images may be stored in a storage 2902, 2904.
  • the user uploads 2908 other data, such as text, spreadsheet, or other data to be stored in storage 2902, 2904.
  • the user creates 2910 a training set using platform tools, such as any of those discussed herein.
  • the user creates 2912 a trained model using platform tools, such as any of those discussed herein.
  • the trained model may include a neural network that has trained to include values at nodes to perform classification on images similar to the training data.
  • the user may use his/her domain knowledge to train the model until it performs well and performs satisfactory in classifying particles of a certain type.
  • the user may publish 2914 the trained model for use by other users. For example, the user may provide the training data and use platform tools to get good results, but the underlying neural network or model may still reside in the platform. Other users may use their own images in an on-line or production environment to determine a classification or particles using the trained model created by the other use. Thus, the other users may benefit from the trained model created by other users or 3 rd party providers. Models, training set data, or other data or algorithms may be selectively published to a marketplace. Thus, a user can selectively keep data or algorithms private or publish for selling/subscription to others.
  • Blocks 2916-2922 represent a method, using the platform 2900, that may be performed by a user (Customer) to use a trained model created by a different user (e.g., 3 rd Party Provider) using the platform 2900.
  • the method begins and a user uploads 2916 images to the platform 2900.
  • the user may upload 2906 images owned or gathered by the user.
  • the images may be stored in a storage 2902, 2904.
  • the images may include particles for detection/classification.
  • the user a customer
  • the user selects 2918 an image processing model for classifying particles in the images.
  • the user may browser a marketplace where models available on the platform 2900 or produced by 3 rd parties are available for usage.
  • the user may select 2918 a model that best matches the need of the user.
  • the platform processes 2920 images in the uploaded images (2916) using the selected model (2918).
  • the selected model may classify particles within the images and provide reports or results to the user.
  • the user may pay 2922 per image processed. A portion of the amount paid 2922 by the user may be provided to the 3 rd party provider that provided/created the trained model.
  • users may create, share, purchase, and use trained models, training sets, or other machine learning tools or data sets using the platform 2900.
  • users may benefit from the domain knowledge of other users and well as profit from their own domain knowledge.
  • the location of the tools, storage, infrastructure, marketplace, and the like in the platform 2900 may also allow users to easily combine models, algorithms, or the like within a single environment (e.g., on the platform).
  • users may chain output from one analytic or model into another analytic or model. This significantly improves efficiency and usage because it may not be necessary to move data between different locations and set up different processing environments to perform different process on the data.
  • a user may chain training models, algorithms, and analytics available on the platform 2900 (including a marketplace) to generate a training set, a trained model, or other algorithm.
  • a user may chain trained models, algorithms, and analytics available on the platform 2900 (including a marketplace) to perform classification or particles, images, or other data, all without require the data be moved or connected to a different processing environment.
  • the platform 2900 also tracks feedback or changes to training or production data. For example, the platform 2900 tracks user changes made to training or production data that can be used to augment the training data. For example, if the user corrects a classification of a particle, the platform 2900 may store an indication that the classification has changed and perform a training algorithm to update/improve a trained model. In one embodiment, the platform 2900 tracks a logged in user so that only specific users are able to augment training data or production data. The platform 2900 may also include tools for consciences based acceptance into the training data. In one embodiment, the platform 2900 further includes or stores algorithms that automatically decide what information should be added to the training data. For example, the platform 2900 may determine that new data from a specific user or account should be added to the training data because the user or account is trusted to have expert knowledge with respect to the training data.
  • Example 1 is a method for digital microscopy of samples.
  • the method includes receiving a digital image of a sample.
  • the method includes processing the digital image using a machine learning prediction model to classify or detect one or more particulates of the sample.
  • the method includes storing the digital image with an indication of a classification for the one or more particulates.
  • Example 2 the method of Example 1 further includes generating a report indicating one or more of: the particulates present in the sample; a particle count; and a health diagnosis.
  • receiving the digital image in any of Examples 1-2 includes receiving over a network from a remote lab.
  • Example 4 the method in any of Examples 1-3 further include aggregating digital images and related data from a plurality of sources.
  • Example 5 the machine learning prediction model in any of Examples 1-4 is trained using one or more of supervised machine learning and unsupervised machine learning.
  • Example 6 the machine learning prediction model in any of Examples 1-5 includes a deep neural network.
  • Example 7 storing the digital image in any of Examples 1-6 includes storing in a database a plurality of digital images of samples comprising the digital image. The method further includes providing a digital image for the plurality of digital images to one or more users for review or classification and receiving an indication of the classification provided by the one or more users.
  • Example 8 the method in an Example 7 further includes retraining the machine learning prediction model if the classification provided by the one or more users is different than a previous classification.
  • Example 9 is an apparatus or system including means to perform a method as in any of Examples 1- 8.
  • Example 10 is a machine readable storage including machine-readable instructions, when executed, to implement a method or realize an apparatus or system as in any of Examples 1-10.
  • Various techniques, or certain aspects or portions thereof, may take the form of program code (i.e., instructions) embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, a non-transitory computer readable storage medium, or any other machine readable storage medium wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the various techniques.
  • the computing device may include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device.
  • the volatile and non- volatile memory and/or storage elements may be a RAM, an EPROM, a flash drive, an optical drive, a magnetic hard drive, or another medium for storing electronic data.
  • One or more programs that may implement or utilize the various techniques described herein may use an application programming interface (API), reusable controls, and the like. Such programs may be implemented in a high-level procedural or an object-oriented programming language to communicate with a computer system. However, the program(s) may be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language, and combined with hardware implementations.
  • API application programming interface
  • a component may be implemented as a hardware circuit comprising custom very large scale integration (VLSI) circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components.
  • VLSI very large scale integration
  • a component may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, or the like.
  • Components may also be implemented in software for execution by various types of processors.
  • An identified component of executable code may, for instance, include one or more physical or logical blocks of computer instructions, which may, for instance, be organized as an object, a procedure, or a function.
  • executables of an identified component need not be physically located together, but may include disparate instructions stored in different locations that, when joined logically together, include the component and achieve the stated purpose for the component.
  • a component of executable code may be a single instruction, or many instructions, and may even be distributed over several different code segments, among different programs, and across several memory devices.
  • operational data may be identified and illustrated herein within components, and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be collected as a single data set, or may be distributed over different locations including over different storage devices, and may exist, at least partially, merely as electronic signals on a system or network.
  • the components may be passive or active, including agents operable to perform desired functions.
  • Implementations of the disclosure can also be used in cloud computing environments.
  • “cloud computing” is defined as a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned via virtualization and released with minimal management effort or service provider interaction, and then scaled accordingly.
  • configurable computing resources e.g., networks, servers, storage, applications, and services
  • a cloud model can be composed of various characteristics (e.g., on-demand self-service, broad network access, resource pooling, rapid elasticity, measured service, or any suitable characteristic now known to those of ordinary skill in the field, or later discovered), service models (e.g., Software as a Service (SaaS), Platform as a Service (PaaS), Infrastructure as a Service (IaaS)), and deployment models (e.g., private cloud, community cloud, public cloud, hybrid cloud, or any suitable service type model now known to those of ordinary skill in the field, or later discovered). Databases and servers described with respect to the disclosure can be included in a cloud model.
  • service models e.g., Software as a Service (SaaS), Platform as a Service (PaaS), Infrastructure as a Service (IaaS)
  • deployment models e.g., private cloud, community cloud, public cloud, hybrid cloud, or any suitable service type model now known to those of ordinary skill in the field, or later discovered.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Chemical & Material Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Dispersion Chemistry (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne des systèmes, des procédés et des dispositifs pour obtenir et classer ou détecter des échantillons par imagerie numérique, telle que par microscopie numérique. Dans un mode de réalisation, des systèmes, des procédés et des dispositifs de microscopie numérique peuvent consister à recevoir une image numérique d'un échantillon. La mise en œuvre peut consister à traiter l'image numérique à l'aide d'un modèle de prédiction par apprentissage machine pour classer ou détecter une ou plusieurs matières particulaires de l'échantillon. La mise en œuvre peut consister à stocker l'image numérique avec une indication d'une classification pour lesdites une ou plusieurs matières particulaires.
PCT/US2017/033876 2016-05-20 2017-05-22 Classification par apprentissage machine de particules ou de substances dans des images de microscopie numérique Ceased WO2017201540A1 (fr)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US201662339627P 2016-05-20 2016-05-20
US62/339,627 2016-05-20
US201662361913P 2016-07-13 2016-07-13
US62/361,913 2016-07-13
US201762447336P 2017-01-17 2017-01-17
US62/447,336 2017-01-17

Publications (1)

Publication Number Publication Date
WO2017201540A1 true WO2017201540A1 (fr) 2017-11-23

Family

ID=60326591

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2017/033876 Ceased WO2017201540A1 (fr) 2016-05-20 2017-05-22 Classification par apprentissage machine de particules ou de substances dans des images de microscopie numérique

Country Status (1)

Country Link
WO (1) WO2017201540A1 (fr)

Cited By (40)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108171274A (zh) * 2018-01-17 2018-06-15 百度在线网络技术（北京）有限公司 用于识别动物的方法和装置
CN108596038A (zh) * 2018-03-28 2018-09-28 电子科技大学 一种结合形态学分割和神经网络的粪便中红细胞识别方法
CN108683535A (zh) * 2018-05-18 2018-10-19 南京大学 基于机器学习的高速数据分配模块故障诊断系统及方法
CN108680473A (zh) * 2018-05-09 2018-10-19 南开大学 一种应用单颗粒气溶胶质谱仪进行颗粒物源解析的方法
CN108760590A (zh) * 2018-03-08 2018-11-06 佛山市云米电器科技有限公司 一种基于图像处理的厨房油烟浓度检测与干扰排除方法
CN109063908A (zh) * 2018-07-30 2018-12-21 浙江鸿程计算机系统有限公司 一种基于深度多任务学习的城市aqi预测与空间细粒度aqi等级估计方法
CN109359725A (zh) * 2018-10-24 2019-02-19 北京周同科技有限公司 卷积神经网络模型的训练方法、装置、设备及计算机可读存储介质
CN109543532A (zh) * 2018-10-19 2019-03-29 兰波(苏州)智能科技有限公司 一种基于分类标准和人类行为的纤维分类的方法
JP2019148473A (ja) * 2018-02-27 2019-09-05 シスメックス株式会社 画像解析方法、画像解析装置、プログラム、学習済み深層学習アルゴリズムの製造方法および学習済み深層学習アルゴリズム
WO2019169432A1 (fr) * 2018-03-07 2019-09-12 Verdict Holdings Pty Ltd Procédés d'identification de matériel biologique par microscopie
CN110309609A (zh) * 2019-07-06 2019-10-08 桂林电子科技大学 一种基于粗糙集和小波神经网络的建筑室内空气品质评价方法
WO2020161585A1 (fr) * 2019-02-08 2020-08-13 Polygone Technologies Inc. Caractérisation de contamination de fluides par des plastiques à l'aide d'une imagerie de milieux filtrants
CN111739658A (zh) * 2020-07-20 2020-10-02 医渡云（北京）技术有限公司 基于输入病例预测传染病趋势的方法及装置
CN112308093A (zh) * 2020-11-24 2021-02-02 腾讯科技（深圳）有限公司 基于图像识别的空气质量感知方法、模型训练方法及系统
JP2021067468A (ja) * 2019-10-17 2021-04-30 五洋建設株式会社 土質特性推定方法、装置、プログラム、およびシステム
CN112868027A (zh) * 2018-10-18 2021-05-28 莱卡微系统Cms有限责任公司 显微镜工作流程的优化
CN112889088A (zh) * 2018-10-31 2021-06-01 合同会社予幸集团中央研究所 程序、学习模型、信息处理装置、信息处理方法、信息显示方法和学习模型的制造方法
WO2021141764A1 (fr) * 2020-01-06 2021-07-15 Becton, Dickinson And Company Procédé d'apprentissage profond dans l'aide à un diagnostic de patient et une identification de population cellulaire aberrante en cytométrie de flux
WO2021150284A1 (fr) * 2020-01-22 2021-07-29 Gracenote, Inc. Recadrage automatique d'images au moyen d'un prédicteur d'apprentissage automatique
CN113228049A (zh) * 2018-11-07 2021-08-06 福斯分析仪器公司 用于将奶分类的奶分析仪
US11145065B2 (en) 2020-01-22 2021-10-12 Gracenote, Inc. Selection of video frames using a machine learning predictor
WO2021252502A1 (fr) * 2020-06-08 2021-12-16 Rodriguez Diaz Eladio Procédés et systèmes de détermination d'une carte d'objet
CN114303050A (zh) * 2019-07-10 2022-04-08 贝克顿·迪金森公司 用于调整细胞分选分类的可重构集成电路
EP3992850A1 (fr) * 2020-10-30 2022-05-04 Friedrich-Alexander-Universität Erlangen-Nürnberg Procédé et système de détection de caractéristiques dans des applications d'imagerie
US11392797B2 (en) 2020-01-31 2022-07-19 Here Global B.V. Method, apparatus, and system for filtering imagery to train a feature detection model
WO2023278692A1 (fr) * 2021-07-02 2023-01-05 Thrive Bioscience, Inc. Procédé de détection de plaque et appareil d'imagerie de cellules
WO2022266208A3 (fr) * 2021-06-16 2023-01-19 Microtrace, Llc Classification utilisant des stratégies d'intelligence artificielle qui reconstruisent des données à l'aide de transformations de compression et de décompression
CN115908948A (zh) * 2023-01-05 2023-04-04 北京霍里思特科技有限公司 一种在线调整模型的智能分选系统及其控制方法
US11676221B2 (en) 2009-04-30 2023-06-13 Patientslikeme, Inc. Systems and methods for encouragement of data submission in online communities
ES2948438A1 (es) * 2022-02-16 2023-09-12 Woom Fertility S L Método implementado por computador para obtener el estado de fertilidad de una mujer
US11894139B1 (en) * 2018-12-03 2024-02-06 Patientslikeme Llc Disease spectrum classification
US11915429B2 (en) 2021-08-31 2024-02-27 Gracenote, Inc. Methods and systems for automatically generating backdrop imagery for a graphical user interface
US11941816B2 (en) 2020-01-22 2024-03-26 Gracenote, Inc. Automated cropping of images using a machine learning predictor
US12038969B2 (en) 2019-05-03 2024-07-16 Verily Life Sciences Llc Predictive classification of insects
US12038372B2 (en) 2015-10-13 2024-07-16 Becton, Dickinson And Company Multi-modal fluorescence imaging flow cytometry system
US12055476B2 (en) 2014-03-18 2024-08-06 The Regents Of The University Of California Parallel flow cytometer using radiofrequency multiplexing
US12078587B2 (en) 2016-03-17 2024-09-03 Becton, Dickinson And Company Cell sorting using a high throughput fluorescence flow cytometer
US12136484B2 (en) 2021-11-05 2024-11-05 Altis Labs, Inc. Method and apparatus utilizing image-based modeling in healthcare
US12158422B2 (en) 2013-01-09 2024-12-03 The Regents Of The University Of California Apparatus and methods for fluorescence imaging using radiofrequency-multiplexed excitation
US12417825B2 (en) 2017-09-15 2025-09-16 Alden Scientific, Inc. Systems and methods for collecting and analyzing comprehensive medical information

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4092014A (en) * 1976-10-13 1978-05-30 Hughes Steel Products Pty. Ltd. Feed mixer
US7369955B2 (en) * 2006-02-23 2008-05-06 Homesafe Inspection, Inc. Method for residential indoor environmental quality inspection and monitoring
US7792353B2 (en) * 2006-10-31 2010-09-07 Hewlett-Packard Development Company, L.P. Retraining a machine-learning classifier using re-labeled training samples
WO2011002272A1 (fr) * 2009-07-01 2011-01-06 Universiti Sains Malaysia Système de mesure et d'avertissement de pollution de l'air
US8386401B2 (en) * 2008-09-10 2013-02-26 Digital Infuzion, Inc. Machine learning methods and systems for identifying patterns in data using a plurality of learning machines wherein the learning machine that optimizes a performance function is selected
US8817253B2 (en) * 2007-02-15 2014-08-26 Green Vision Systems Ltd. Hyper-spectral imaging and analysis of a sample of matter, for identifying and characterizing an object of interest therein
US20140247984A1 (en) * 2013-03-01 2014-09-04 Colormodules Inc. Methods for color correcting digital images and devices thereof
US8907803B2 (en) * 2012-01-09 2014-12-09 Intwine Energy Networked air quality monitoring
WO2015153505A1 (fr) * 2014-03-31 2015-10-08 Ingrain, Inc. Détermination de volume élémentaire représentatif au moyen de statistiques basées sur le regroupement

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4092014A (en) * 1976-10-13 1978-05-30 Hughes Steel Products Pty. Ltd. Feed mixer
US7369955B2 (en) * 2006-02-23 2008-05-06 Homesafe Inspection, Inc. Method for residential indoor environmental quality inspection and monitoring
US7792353B2 (en) * 2006-10-31 2010-09-07 Hewlett-Packard Development Company, L.P. Retraining a machine-learning classifier using re-labeled training samples
US8817253B2 (en) * 2007-02-15 2014-08-26 Green Vision Systems Ltd. Hyper-spectral imaging and analysis of a sample of matter, for identifying and characterizing an object of interest therein
US8386401B2 (en) * 2008-09-10 2013-02-26 Digital Infuzion, Inc. Machine learning methods and systems for identifying patterns in data using a plurality of learning machines wherein the learning machine that optimizes a performance function is selected
WO2011002272A1 (fr) * 2009-07-01 2011-01-06 Universiti Sains Malaysia Système de mesure et d'avertissement de pollution de l'air
US8907803B2 (en) * 2012-01-09 2014-12-09 Intwine Energy Networked air quality monitoring
US20140247984A1 (en) * 2013-03-01 2014-09-04 Colormodules Inc. Methods for color correcting digital images and devices thereof
WO2015153505A1 (fr) * 2014-03-31 2015-10-08 Ingrain, Inc. Détermination de volume élémentaire représentatif au moyen de statistiques basées sur le regroupement

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CHEMIMAGE CORPORATION.. WHAT IS HYPERSPECTRAL IMAGING?, 2014, pages 1 - 2, Retrieved from the Internet <URL:http://www.chemimage.com/markets/forensics/hsi.aspx> [retrieved on 20170710] *

Cited By (68)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11676221B2 (en) 2009-04-30 2023-06-13 Patientslikeme, Inc. Systems and methods for encouragement of data submission in online communities
US12158421B2 (en) 2013-01-09 2024-12-03 The Regents Of The University Of California Apparatus and methods for fluorescence imaging using radiofrequency-multiplexed excitation
US12158422B2 (en) 2013-01-09 2024-12-03 The Regents Of The University Of California Apparatus and methods for fluorescence imaging using radiofrequency-multiplexed excitation
US12055476B2 (en) 2014-03-18 2024-08-06 The Regents Of The University Of California Parallel flow cytometer using radiofrequency multiplexing
US12038372B2 (en) 2015-10-13 2024-07-16 Becton, Dickinson And Company Multi-modal fluorescence imaging flow cytometry system
US12313521B1 (en) 2016-03-17 2025-05-27 Becton, Dickinson And Company Cell sorting using a high throughput fluorescence flow cytometer
US12078587B2 (en) 2016-03-17 2024-09-03 Becton, Dickinson And Company Cell sorting using a high throughput fluorescence flow cytometer
US12417825B2 (en) 2017-09-15 2025-09-16 Alden Scientific, Inc. Systems and methods for collecting and analyzing comprehensive medical information
CN108171274B (zh) * 2018-01-17 2019-08-09 百度在线网络技术（北京）有限公司 用于识别动物的方法和装置
CN108171274A (zh) * 2018-01-17 2018-06-15 百度在线网络技术（北京）有限公司 用于识别动物的方法和装置
JP2019148473A (ja) * 2018-02-27 2019-09-05 シスメックス株式会社 画像解析方法、画像解析装置、プログラム、学習済み深層学習アルゴリズムの製造方法および学習済み深層学習アルゴリズム
JP7769732B2 (ja) 2018-02-27 2025-11-13 シスメックス株式会社 画像解析方法、画像解析装置、およびプログラム
JP2024038396A (ja) * 2018-02-27 2024-03-19 シスメックス株式会社 画像解析方法、画像解析装置、およびプログラム
JP7217589B2 (ja) 2018-02-27 2023-02-03 シスメックス株式会社 画像解析方法、画像解析装置、プログラム、学習済み深層学習アルゴリズムの製造方法および学習済み深層学習アルゴリズム
WO2019169432A1 (fr) * 2018-03-07 2019-09-12 Verdict Holdings Pty Ltd Procédés d'identification de matériel biologique par microscopie
EP3762934A4 (fr) * 2018-03-07 2022-04-06 Verdict Holdings Pty Ltd Procédés d'identification de matériel biologique par microscopie
CN112204674A (zh) * 2018-03-07 2021-01-08 维迪控股私人有限公司 通过显微术识别生物材料的方法
JP2021517255A (ja) * 2018-03-07 2021-07-15 ヴァーディクト ホールディングス プロプライエタリー リミテッド 顕微鏡による生物学的物質の同定方法
US11798662B2 (en) 2018-03-07 2023-10-24 Verdict Holdings Pty Ltd Methods for identifying biological material by microscopy
CN108760590A (zh) * 2018-03-08 2018-11-06 佛山市云米电器科技有限公司 一种基于图像处理的厨房油烟浓度检测与干扰排除方法
CN108596038B (zh) * 2018-03-28 2022-03-15 电子科技大学 一种结合形态学分割和神经网络的粪便中红细胞识别方法
CN108596038A (zh) * 2018-03-28 2018-09-28 电子科技大学 一种结合形态学分割和神经网络的粪便中红细胞识别方法
CN108680473A (zh) * 2018-05-09 2018-10-19 南开大学 一种应用单颗粒气溶胶质谱仪进行颗粒物源解析的方法
CN108683535A (zh) * 2018-05-18 2018-10-19 南京大学 基于机器学习的高速数据分配模块故障诊断系统及方法
CN109063908A (zh) * 2018-07-30 2018-12-21 浙江鸿程计算机系统有限公司 一种基于深度多任务学习的城市aqi预测与空间细粒度aqi等级估计方法
CN112868027A (zh) * 2018-10-18 2021-05-28 莱卡微系统Cms有限责任公司 显微镜工作流程的优化
US12374131B2 (en) 2018-10-18 2025-07-29 Leica Microsystems Cms Gmbh Optimization of workflows for microscopes
CN109543532B (zh) * 2018-10-19 2023-07-28 兰波(苏州)智能科技有限公司 一种基于分类标准和人类行为的纤维分类的方法
CN109543532A (zh) * 2018-10-19 2019-03-29 兰波(苏州)智能科技有限公司 一种基于分类标准和人类行为的纤维分类的方法
CN109359725A (zh) * 2018-10-24 2019-02-19 北京周同科技有限公司 卷积神经网络模型的训练方法、装置、设备及计算机可读存储介质
CN112889088A (zh) * 2018-10-31 2021-06-01 合同会社予幸集团中央研究所 程序、学习模型、信息处理装置、信息处理方法、信息显示方法和学习模型的制造方法
CN113228049A (zh) * 2018-11-07 2021-08-06 福斯分析仪器公司 用于将奶分类的奶分析仪
CN113228049B (zh) * 2018-11-07 2024-02-02 福斯分析仪器公司 用于将奶分类的奶分析仪
US11721094B2 (en) 2018-11-07 2023-08-08 Foss Analytical A/S Milk analyser for classifying milk
US11894139B1 (en) * 2018-12-03 2024-02-06 Patientslikeme Llc Disease spectrum classification
WO2020161585A1 (fr) * 2019-02-08 2020-08-13 Polygone Technologies Inc. Caractérisation de contamination de fluides par des plastiques à l'aide d'une imagerie de milieux filtrants
US12038969B2 (en) 2019-05-03 2024-07-16 Verily Life Sciences Llc Predictive classification of insects
CN110309609B (zh) * 2019-07-06 2022-06-28 桂林电子科技大学 一种基于粗糙集和wnn的建筑室内空气品质评价方法
CN110309609A (zh) * 2019-07-06 2019-10-08 桂林电子科技大学 一种基于粗糙集和小波神经网络的建筑室内空气品质评价方法
US11978269B2 (en) * 2019-07-10 2024-05-07 Becton, Dickinson And Company Reconfigurable integrated circuits for adjusting cell sorting classification
CN114303050A (zh) * 2019-07-10 2022-04-08 贝克顿·迪金森公司 用于调整细胞分选分类的可重构集成电路
US20230334883A1 (en) * 2019-07-10 2023-10-19 Becton, Dickinson And Company Reconfigurable Integrated Circuits for Adjusting Cell Sorting Classification
JP7292176B2 (ja) 2019-10-17 2023-06-16 五洋建設株式会社 土質特性推定方法、装置、プログラム、およびシステム
JP2021067468A (ja) * 2019-10-17 2021-04-30 五洋建設株式会社 土質特性推定方法、装置、プログラム、およびシステム
US11662295B2 (en) 2020-01-06 2023-05-30 Becton, Dickinson And Company Deep learning method in aiding patient diagnosis and aberrant cell population identification in flow cytometry
WO2021141764A1 (fr) * 2020-01-06 2021-07-15 Becton, Dickinson And Company Procédé d'apprentissage profond dans l'aide à un diagnostic de patient et une identification de population cellulaire aberrante en cytométrie de flux
US11941816B2 (en) 2020-01-22 2024-03-26 Gracenote, Inc. Automated cropping of images using a machine learning predictor
US12347157B2 (en) 2020-01-22 2025-07-01 Gracenote, Inc. Selection of video frames using a machine learning predictor
WO2021150284A1 (fr) * 2020-01-22 2021-07-29 Gracenote, Inc. Recadrage automatique d'images au moyen d'un prédicteur d'apprentissage automatique
US12272069B2 (en) 2020-01-22 2025-04-08 Gracenote, Inc. Automated cropping of images using a machine learning predictor
US11080549B1 (en) 2020-01-22 2021-08-03 Gracenote, Inc. Automated cropping of images using a machine learning predictor
US11145065B2 (en) 2020-01-22 2021-10-12 Gracenote, Inc. Selection of video frames using a machine learning predictor
US11776234B2 (en) 2020-01-22 2023-10-03 Gracenote, Inc. Selection of video frames using a machine learning predictor
US11392797B2 (en) 2020-01-31 2022-07-19 Here Global B.V. Method, apparatus, and system for filtering imagery to train a feature detection model
WO2021252502A1 (fr) * 2020-06-08 2021-12-16 Rodriguez Diaz Eladio Procédés et systèmes de détermination d'une carte d'objet
CN111739658A (zh) * 2020-07-20 2020-10-02 医渡云（北京）技术有限公司 基于输入病例预测传染病趋势的方法及装置
CN111739658B (zh) * 2020-07-20 2021-02-02 医渡云（北京）技术有限公司 基于输入病例预测传染病趋势的方法及装置
EP3992850A1 (fr) * 2020-10-30 2022-05-04 Friedrich-Alexander-Universität Erlangen-Nürnberg Procédé et système de détection de caractéristiques dans des applications d'imagerie
CN112308093A (zh) * 2020-11-24 2021-02-02 腾讯科技（深圳）有限公司 基于图像识别的空气质量感知方法、模型训练方法及系统
CN112308093B (zh) * 2020-11-24 2023-05-30 腾讯科技（深圳）有限公司 基于图像识别的空气质量感知方法、模型训练方法及系统
WO2022266208A3 (fr) * 2021-06-16 2023-01-19 Microtrace, Llc Classification utilisant des stratégies d'intelligence artificielle qui reconstruisent des données à l'aide de transformations de compression et de décompression
WO2023278692A1 (fr) * 2021-07-02 2023-01-05 Thrive Bioscience, Inc. Procédé de détection de plaque et appareil d'imagerie de cellules
US12198353B2 (en) 2021-08-31 2025-01-14 Gracenote, Inc. Methods and systems for automatically generating backdrop imagery for a graphical user interface
US11915429B2 (en) 2021-08-31 2024-02-27 Gracenote, Inc. Methods and systems for automatically generating backdrop imagery for a graphical user interface
US12136484B2 (en) 2021-11-05 2024-11-05 Altis Labs, Inc. Method and apparatus utilizing image-based modeling in healthcare
ES2948438A1 (es) * 2022-02-16 2023-09-12 Woom Fertility S L Método implementado por computador para obtener el estado de fertilidad de una mujer
CN115908948A (zh) * 2023-01-05 2023-04-04 北京霍里思特科技有限公司 一种在线调整模型的智能分选系统及其控制方法
CN115908948B (zh) * 2023-01-05 2024-04-26 北京霍里思特科技有限公司 一种在线调整模型的智能分选系统及其控制方法

Similar Documents

Publication Publication Date Title
US10255693B2 (en) Machine learning classification and training for digital microscopy images
WO2017201540A1 (fr) Classification par apprentissage machine de particules ou de substances dans des images de microscopie numérique
US10552663B2 (en) Machine learning classification and training for digital microscopy cytology images
US10311573B2 (en) Training and machine learning classification of mold in digital microscopy images
Kers et al. Deep learning-based classification of kidney transplant pathology: a retrospective, multicentre, proof-of-concept study
Caldeira et al. Identification of cotton leaf lesions using deep learning techniques
Qiu et al. Detection of fusarium head blight in wheat using a deep neural network and color imaging
Trevisan et al. Macroepidemiological aspects of porcine reproductive and respiratory syndrome virus detection by major United States veterinary diagnostic laboratories over time, age group, and specimen
Moscalu et al. Histopathological images analysis and predictive modeling implemented in digital pathology—current affairs and perspectives
Maruthai et al. Hybrid vision GNNs based early detection and protection against pest diseases in coffee plants
Kaur et al. A novel framework for semi-automated system for grape leaf disease detection
Böckmann et al. Rapid and low-cost insect detection for analysing species trapped on yellow sticky traps
Smith et al. Internet of animal health things (IoAHT) opportunities and challenges
CN111247551A (zh) 受精卵质量评估方法、受精卵质量评估系统、程序和信息处理设备
Mignoni et al. Soybean images dataset for caterpillar and Diabrotica speciosa pest detection and classification
Hundekari et al. Design and implementation of deep learning method for disease identification in plant leaf
Gedeon et al. Identification and counting of European Souslik burrows from UAV images by pixel-based image analysis and random forest classification: a simple, semi-automated, yet accurate method for estimating population size
Klingström et al. Beyond the hype: using AI, big data, wearable devices, and the internet of things for high-throughput livestock phenotyping
Araaf et al. Coffee leaf rust disease detection and implementation of an edge device for pruning infected leaves via deep learning algorithms
Wang et al. Forage height and above-ground biomass estimation by comparing UAV-based multispectral and RGB imagery
Feng et al. Wheat fusarium head blight automatic non-destructive detection based on multi-scale imaging: A technical perspective
Madhanakumar Lumpy skin disease prediction using machine learning
Yu et al. Extracting fractional vegetation cover from digital photographs: a comparison of in situ, SamplePoint, and image classification methods
Alzubi Lumpy Skin Disease Detection in Cattle by a Robust Approach using Advanced Convolutional Neural Networks.
Li et al. Development of a target-to-sensor mode multispectral imaging device for high-throughput and high-precision touch-based leaf-scale soybean phenotyping

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17800341

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC - FORM 1205A (05.03.2019)

122 Ep: pct application non-entry in european phase

Ref document number: 17800341

Country of ref document: EP

Kind code of ref document: A1