EP4272155A1 - Apprentissage machine pour la détection précoce de changements morphologiques cellulaires - Google Patents

Apprentissage machine pour la détection précoce de changements morphologiques cellulaires

Info

Publication number
EP4272155A1
EP4272155A1 EP22750503.9A EP22750503A EP4272155A1 EP 4272155 A1 EP4272155 A1 EP 4272155A1 EP 22750503 A EP22750503 A EP 22750503A EP 4272155 A1 EP4272155 A1 EP 4272155A1
Authority
EP
European Patent Office
Prior art keywords
images
cells
infected
virus
models
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP22750503.9A
Other languages
German (de)
English (en)
Other versions
EP4272155A4 (fr
Inventor
Ilya GOLDBERG
Dmitry FEDOROV
Christian Lang
Kristian KVILEKVAL
Katherine YEUNG
Henry DODKINS
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Viqi Inc
Original Assignee
Viqi Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Viqi Inc filed Critical Viqi Inc
Priority claimed from PCT/US2022/015379 external-priority patent/WO2022170145A1/fr
Publication of EP4272155A1 publication Critical patent/EP4272155A1/fr
Publication of EP4272155A4 publication Critical patent/EP4272155A4/fr
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/2433—Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06V10/7747—Organisation of the process, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/809—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of classification results, e.g. where the classifiers operate on the same input data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/87—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using selection of the recognition techniques, e.g. of a classifier in a multiple classifier system
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/693—Acquisition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/698—Matching; Classification
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H15/00—ICT specially adapted for medical reports, e.g. generation or transmission thereof
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/67—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for remote operation
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images

Definitions

  • the disclosed embodiments relate generally to machine learning about biological cells from digital images for early detection of changes in cellular structure, such as the early detection of COVID-19 plaques in cells.
  • a plaque assay is one of the most important assays in virology. It measures the number of infectious viral particles in a sample by observing the effects of infection on a culture of susceptible cells. Currently, it takes about two to fourteen days to process a plaque assay because several rounds of infection are necessary to ensure an accurate read-out. It is desirable to reduce the time to get results of infectivity.
  • Figure 1 A is a block diagram of a client-server computer system with multiple client computers communicating with one or more computer servers in a server center (or the cloud) over a computer network, such as a wide area network of the internet, and the services offered.
  • Figure 1 B illustrates a block diagram of a biological cell analysis system implementing an early detection system for cellular morphological changes.
  • Figure 2A illustrates a user interface window generated by the system including a slide viewer illustrating a brightfield image of biological cells (center) and annotations (right panel) of some of the cells.
  • Figure 2B illustrates the detail of the right-side panel of the user interface shown in Figure 2A.
  • Figure 3 illustrates a user interface window generated by the system with an example output of machine learning classification of classified cells for detecting morphological changes in brightfield images.
  • Figure 4A illustrates another user interface viewer window provided by the system.
  • Figure 4B illustrates the viewer window of brightfield image of cells overlaid with the classification results from classification algorithms using a generated Al model.
  • Figure 5A illustrates workflow diagrams to compare a traditional assay workflow for a standard viral plaque assay with that of an Al enhanced assay workflow, including a machine learning/model training phase with brightfield images of cells with known cell conditions and a diagnostic phase with cell classification of cells to be tested for their cell condition by an Al model.
  • Figure 5B illustrates a workflow diagram of the two phases, training and classification for Al powered cell assays according to embodiments of the invention.
  • Figure 5C illustrates a workflow diagram and timeline comparing a traditional plaque assay with an Al enhanced assay.
  • Figure 5D illustrates image capture of digital images of wells in a plate with infected biological cells and the formation of rectangular tiles of pixels from a digital image.
  • Figure 5E illustrates a validation chart with plots to compare infectivity determined with an Al model and machine learning algorithms versus infectivity determined with traditional assay methods.
  • Figure 6 illustrates a block diagram of a client-server computer system with multiple client computers communicating with one or more computer servers in a server center (or the cloud) over a computer network, such as a wide area network of the internet.
  • Figure 7 illustrates a block diagram of a computer system for use as a server computer and client computers (devices) in the system shown in Figure 6.
  • the disclosed embodiments can speed the development of vaccines and antivirals, gene therapy vectors, and oncolytic viral therapies using machine learning to develop a rapid automated infectivity assay based on identifying virus-infected cells in brightfield microscopy within hours of infection.
  • Machine learning can detect cell morphologies associated with virus infection even when they cannot be detected by human observers.
  • the disclosed embodiments do not rely on detecting cell death after multiple rounds of infection like in traditional plaque or TCID50 assays, instead relying on much earlier infection events.
  • the use of brightfield microscopy obviates cell fixation and staining which allows the technique to be more easily automated as well as making it faster and more cost effective.
  • other digital images of virus-infected cells can be used, such as darkfield images from darkfield microscopy, phase contrast images from phase contrast microscopy, and differential interference contrast (DIG) images from DIG microscopy.
  • Benefits of the disclosed embodiment include:
  • HCS High Content Screening
  • HCS high content screening
  • Rapid accurate identification of a virus can be used to implement effective means of contact tracing, quarantine, and the deployment of effective antiviral agents.
  • HCA High Content Analysis
  • the disclosed embodiments use artificial intelligence (machine learning) to recognize patterns without the need for human intervention.
  • the broad principals disclosed can be applied to plaque assays as detailed below. Equally, the disclosed embodiments are also applicable to drug dosage response of cells, cell toxicity, and detection of live/dead cell detection without the use of fluorescence labelling.
  • the disclosed embodiments use machine learning to accelerate testing of anti-viral agents.
  • the disclosed embodiments take a novel approach to viral assays by training modern artificial intelligence (Al) models (Als) to detect individual infected cells or small clusters of infected cells using brightfield (phase contrast or differential interference contrast) microscopy earlier than they can be detected manually, and without the use of viable stains.
  • Al artificial intelligence
  • the Al models disclosed herein differentiate infected and uninfected cells based on morphological changes in the cells.
  • the Al models process images of cells in biological samples and can also be referred to herein as imaging artificial intelligence (Al) models.
  • Al imaging artificial intelligence
  • the use of Al obviates the need for several slow elements of the assay protocol, resulting in increased throughput for evaluating vaccines and antiviral drugs.
  • Al can also facilitate automation of high-content analysis (HCA) platforms and thus more easily fit into existing drug discovery pipelines.
  • HCA high-content analysis
  • HCA High-Content Analysis
  • Automation will permit the screening of many more compounds under many more conditions, leading to faster development of antiviral remedies.
  • This type of plaque assay development is applicable to other cell types and viruses, and so this could also be used in influenza studies, for example.
  • experts have warned that as with influenza we can expect more outbreaks of COVID-19 due to seasonal variations; therefore, rapid identification of effective antiviral agents are critical to our future.
  • the plaque assay is one of the most important assays in virology. It measures the number of infectious viral particles in a sample by observing the effects of infection on a culture of susceptible cells.
  • This assay counts the number of infectious viral particles in a sample (plaque-forming units or PFUs) which is in contrast to assays based on amplifying genetic material (e.g., rtPCR), which only count the number of viral genomes.
  • amplifying genetic material e.g., rtPCR
  • the typical assay takes two to fourteen days to develop because several rounds of infection are necessary to ensure that there are localized clusters of dead cells defining each plaque indicating an infection, rather than single dead cells which could occur for any number of reasons.
  • the clusters of dead cells are typically detected with an end-point viability stain such as crystal violet.
  • the disclosed embodiments detect viral plaques, in an automated manner, by training modern artificial intelligence models (Als) to detect individual infected cells or small clusters of infected cells using brig htfield (phase contrast or differential interference contrast) microscopy earlier than they can be detected manually, and without the use of viable stains.
  • Als artificial intelligence models
  • the disclosed embodiments detect individual infected cells, thus obviating the need for several elements of the current protocol that make it slow, e.g., multiple rounds of infection, agarose overlay to limit viral diffusion, and end-point viability staining.
  • the final outcome can be an increased throughput for evaluating vaccines and antiviral drugs.
  • Modern artificial intelligence models have not been previously used to identify individual virally infected cells without labels or stains.
  • the disclosed embodiments can use single-cell image-based assays with Al models to discern differences in cells and tissues that cannot be differentiated by highly trained human observers.
  • Artificial intelligence models (Als) can readily differentiate sub-cellular patterns that even trained human experts could not with immuno-fluorescence assays on individual cells.
  • DIG differential interference contrast
  • phase contrast artificial intelligence models (Als) can readily differentiate samples of unstained (stain free) cells and tissues. Human observers cannot do so without labels or stains. Moreover, human observers cannot quickly observe and count samples with large quantities of infected biological cells to determined infection concentrations.
  • the disclosed embodiments use the best available algorithms applied to the traditional plaque assay that is currently a bottleneck in the search for antiviral therapies.
  • a specific challenge overcome by the disclosed embodiments is the experimental design to collect an appropriate training set of infected cells, in a state prior to them developing visible changes (which are typically lysis and death), but without using stains, markers or other means to highlight the cells undergoing early stages of infection.
  • the disclosed embodiments can do more than just applying machine learning approaches to existing plaque assays.
  • the Al models do not just count plaque forming units (PFU).
  • PFU plaque forming units
  • the Al models that are developed can differentiate subcellular patterns and display them in a user interface.
  • the disclosed embodiments perform an analysis of a viral assay, generate models to recognize infections cells, and disseminate the resulting Al models rapidly, without requiring a significant amount of capital investment for custom software development related to microscopy image format readers, scalable slide viewers, data processing for machine learning, model evaluation and sharing, and a web-based interface to minimize time-consuming installation and configuration.
  • FIG. 1 A a web-based scalable image analysis platform 10 is shown.
  • the web-based scalable image analysis platform 10 is usable for a variety of image management and analysis tasks in life sciences.
  • a vast majority of images processed by the web-based scalable image analysis platform are high resolution microscopy images.
  • the web-based scalable image analysis platform 10 modeled after a BisQue system, can be used to analyze biological cells.
  • BisQue is an open-source image database and analysis system built for biologists working with digital artifacts as a fundamental tool for gathering evidence. The system is built to manage and analyze both data artifacts and metadata for large-scale datasets. BisQue has gained increasing adoption by several large bio-imaging labs world-wide. For example, the CyVerse cyberinfrastructure serves thousands of users using BisQue for its imaging needs.
  • the web-based scalable image analysis platform 10 being a cloud-based platform, is built with a scalable service-oriented architecture.
  • the web-based scalable image analysis platform 10 provides access to services through web browsers 14 and client tools 12.
  • the web browsers 14 access client services 19 including web pages with viewers and statistics and can be used for general control, such as file import and file export.
  • Web browsers 14 and client tools 12 can interface to a plurality of micro-services 16-18, each performing a specialized task.
  • a web browser can also act as a client tool (Javascript) and interface to the microservices 16-18 (e.g., running analysis modules from a webpage).
  • a metadata service 17 allows metadata to be stored in storage 24 and queried as nested tag/value documents. Examples for metadata include sample preparation, experimental conditions, imaging parameters, and results.
  • a blob and image service 16 provides access to binary datasets such as images and provides methods to perform pixel operations for example.
  • An analysis service 18 allows running complex analysis modules written in many languages (e.g., Python, Matlab).
  • the analysis modules of the analysis service 18 typically interact with the other services (e.g., to fetch image tiles or metadata records) during computation.
  • the microservices 16-18 and the client services 19 can interface to the backend services of storage services 24 associated with storage devices and execution services 26 associated with computer processors or compute clusters.
  • the platform supports all popular microscopy image formats (including channels and time series).
  • the platform has a slide viewer that allows visualization and annotation of images of any size, with multiple channels and of any bit depth.
  • Figure 2A shows a user interface 200 with an example high- resolution image of a portion of a biological sample in a viewer window 201 .
  • the example high-resolution image is a five (5) channel, sixteen (16) data bit-width image taken from a high resolution digital microscope or other high resolution digital imaging device (digital imager).
  • the digital imaging device is an imaging robot that performs robotic microscopy. Additional robots, such as a fluid handling robot, can be used to process the plates for imaging in order to provide a more automated process.
  • custom software can be used to interrogate the client software for the needed data.
  • Toolkits and services can be deployed locally to automatically and securely transfer images and metadata as acquired.
  • raw image data can be manually or automatically imported or exported.
  • a toolkit can be structured so that collection of raw image data can be imported with multiple strategies: from application database, from filesystem export, etc.
  • Experimental meta-data is usually not reported by imaging devices unless they are part of a LIMS (laboratory information management system). The meta data is acquired with the raw image data.
  • Experimental layouts are represented on a grid using a spreadsheet.
  • Spreadsheet templates for experimental layouts can be provided to clients to facilitate the uniform importation of data.
  • Parsers and web user interface (Ul) components can also be used on user-provided spreadsheets.
  • images and metadata can be transferred directly through the web browser via a proprietary web portal.
  • local storage and processing can be made available for large-scale users.
  • FIG. 1 B a high-level block diagram of a biological cell analysis system 100 is shown that utilizes machine learning to detect early-stage cellular morphological changes.
  • the biological cell analysis system 100 includes an image processing system 102, a machine learning (Al) model 104 that can be either trained or used for classification/analysis, and a user interface 106.
  • the machine learning (Al) model 104 can be trained for use with one or more machine learning algorithms (including image processing algorithms) and then used with the one or more machine learning algorithms for classification/analysis of objects within images of biological samples (infected, uninfected, treated or untreated), including biological cells and their sub-cellular structure.
  • the image processing system 102 can read digital images of cells stored in a storage device 124 and associated metadata from a database 101.
  • the raw image data and associated metadata can be respectively transferred from a user’s storage device/database via the internet into the storage device 124 and the database 101 for cloud image processing.
  • the database 101 is stored in the storage device 124 with the images while in another embodiment, the database 101 is stored in another storage device (e.g., memory 720, SSD 730, Disk Drive 740 shown in Figure 7) separate from the storage device 124 storing the raw image data.
  • the database 101 and the storage device 124 can be local and local processing by one or more local processors with local software can be utilized for clients having sufficient hardware and volume of stored assays.
  • the image processing system 102 can be used to process raw image data.
  • the raw image data can be processed to obtain specific image information (e.g., resolution, size, background filtering, etc.)
  • the image processing system 102 can generate feature vectors from the captured images of the biological cells.
  • the feature vectors can be used to train the machine learning (Al) model 104.
  • the machine learning (Al) model 104 can be trained to provide useful and accurate information about the biological cells and other objects within a biological sample.
  • the machine learning (Al) model 104 is used with one or more classifier algorithms to classify biological cells and to detect morphological changes of the classified biological cells over time.
  • the image data can be used to train an Al model 104 in a training mode 105A.
  • the Al model 104 can be validated with additional image data that is excluded from the training process. If the Al model 104 has been previously trained (pretrained), it can be used in an analytical mode 105B to analyze biological cell assays.
  • images of biological cell assays can be analyzed to recognize and classify biological cells. Counts of the number of classified cells can be made within the given area of the images. From the counts, results and accuracy of predicted concentrations can be generated.
  • a user can interact with the system 100 through the user interface 106.
  • the user interface 106 can be used to build one or more Al models for a new sample of biological cells.
  • the user interface 106 can be used to seed the recognition/classification of one or more objects in the new sample.
  • the user interface 106 can receive the report generated by the use of the Al model 104 and algorithms in analyzing the one or more images of a biological sample.
  • the report and analytical results can be viewed in various windows generated by the user interface 106 on a display device.
  • the report may include diagnosis information of a patient associated with the biological sample, or the sample can be from environmental monitoring.
  • the report can include percentages of likelihood of the presence of one or more infectious viruses, which can inform possible treatment information for the patient.
  • Figure 2A illustrates a user interface 200 with a display window 201 displaying a brightfield image of biological cells in the brightfield channel.
  • Figure 2B illustrates the right panel 202 of annotations of different tags or labels 221-224 having different colored circles on a few objects recognized by a user within the image 201 shown in Figure 2A.
  • the objects tagged or labeled in the image can be recognized by machine learning with annotations (type of object, location) added as part of the metadata 206 of an image.
  • the tags or labels objects in the image can be colored differently, such as by different colored circles, and can be overlaid onto the objects (e.g., cells, debris) such as shown in the image of Figure 2A.
  • the different colors can be used to emphasize the annotation and provide information about the tagged object, such as a cell or a subcellular component (e.g., nucleus, lysosomes, peroxisomes, mitochondria, endoplasmic reticulum, golgi apparatus) and its state (live, dead, infected, uninfected).
  • a cell or a subcellular component e.g., nucleus, lysosomes, peroxisomes, mitochondria, endoplasmic reticulum, golgi apparatus
  • its state live, dead, infected, uninfected.
  • the annotations associated with the tags or labels 221-224 can include Debris 204A in a first color, CellXJive 204B in a second color, CellX_dead 204C in a third color, Cell Y Ji ve 204D in a fourth color, and CellY_dead 204E in a fifth color, for example.
  • the annotations shown in Figure 2B can be colored to match that of the colored circles overlaid onto the objects (e.g., cells, debris) shown in the image of Figure 2A.
  • the tags or labels 221-224 on the objects and associated annotations are used to train one or more Al models of supervised machine learning algorithms so that the same classes of objects can be recognized throughout the image at different locations.
  • the web-based scalable image analysis platform 10 shown in Figure 1A includes a machine learning service as part of the Analysis Services 18 to build Al models and classify cells and other objects.
  • Figure 3 illustrates a deep learning model builder 300 as part of the platform.
  • An example imaging Al cell classification model shown in Figure 3 is used for classifying live/dead cells in a sample of cells.
  • the model builder 300 illustrates a sample 302 (sample two of one through five) of a training dataset that is shown in the right side of the frame.
  • a row of control buttons 303 at the top of each page, including Create, Upload, Download, Analyze, and Browse, can be use for creation, uploading, downloading, analyzing, and browsing.
  • a slider and display 304A can be used to filter out classes based on minimum number of samples per class.
  • a slider and display 304B can be used to select a minimum accuracy required for a class to be used in classification.
  • a slider and display 304C can be used to select a goodness threshold under which classification results for individual samples will be discarded during classification.
  • a progress bar 306 of the steps in building the Al model is provided, including selecting the dataset, the filter classes, creating samples, training of Al model, and validating the Al model.
  • the user interface guides the user through each of these steps of the process, shown by the progress bar 306. If something changes in one or more of the settings, a revalidate button 307 is provided to re-validate a previously trained and validated Al model.
  • the builder includes a pie chart 308 that visibly shows, with different colors, the size of the different object classes found in the training dataset.
  • the builder 300 further shows a plurality of tabs 310 that can be selected to show information about the Al builder including available classes of object that can be recognized, a plot of the classes of objects used during model building, model performance table (shown), a performance plot of objects, a summary table, and a comparison table.
  • the selected tab shows a model performance table 312 of the object classes recognized by the Al model for the selected training dataset. It is information used during training of the Al model.
  • the Al model and classifier included 145 cells from the training dataset, resulting in a 93.8 percent accuracy, an 8.6 percent error rate, an 85.7 percent F1 % rate (a metric quantifying the prediction accuracy), with 32 cells from the training dataset used for validation, and 0.0 percent of the cells discarded during training.
  • the machine learning service allows a user to build machine learning models on large-scale training data collected from images and metadata with a variety of features and learning frameworks.
  • the generated models can be used to predict properties over large image collections and visualize the results as overlays for easy digestion by scientific users.
  • the machine learning service can train an Al model that can distinguish brightfield images of virus-infected cells from uninfected cells at a subcellular level. Training Al models to find initial infection events in brightfield imaged cells, enables the creation of a fully automatable virus infection assay with same-day turnaround.
  • FIGs 4A-4B show a user interface window 400 with an example resultant output of a machine learning classification with classified (e.g., live/dead) cells and other objects recognized in brightfield images and color coded using an object outline overlay.
  • tags or labels 221-224 were added on top of objects and associated annotations were created.
  • these tags and labels 221-224 are used to train one or more Al models of supervised machine learning algorithms to train on the training objects 421T-424T.
  • the supervised machine learning algorithms can recognize the same classes of objects as recognized objects 421 R-424R throughout the image at different locations during analysis after training.
  • Al models can be trained using fluorescent tags (fluorescent protein genes, fluorescently labeled antibodies, etc.) to mark cells of interest.
  • Multi-channel imaging with a fluorescence channel and a brightfield channel can be used to train Al models.
  • Viruses containing fluorescent protein genes, or staining the cells with fluorescently- labeled antibodies can be used to detect viral proteins and distinguish infected from uninfected cells in the fluorescence channel.
  • the Al model can be trained by looking exclusively at the brightfield channel at cells identified as infected vs. uninfected by using the coincident infection signal in the fluorescent channel.
  • a trained Al model can be validated by having it make predictions on the cells using the brightfield channel alone, and using the coincidence of the Al's infected predictions with the signal in the fluorescence channel to determine Al model accuracy. After training and validation, the Al model can then be used solely on the brightfield channel, without the fluorescent channel, during analysis of other brightfield images of infected cells.
  • the viewer overlays the classification results on top of the image as color- coded masked regions.
  • the color red can be overlaid onto certain cells to indicate the dead cells to the user.
  • the color yellow for example, can be overlaid onto certain cells to indicate live cells.
  • Other colors can be used instead and other cell states can be indicated by more colors as well.
  • the visualizer platform slide viewer
  • the user interface window 400 includes a viewer window 401 and a side bar 402.
  • a magnified portion 410 of the sample is shown.
  • the view window 401 includes an overview window 404 of the sample in a well 405.
  • the portion 410 of the sample shown in the viewer window 401 is indicated in the overview 404 by a rectangle 406 in the well 405.
  • Magnification information 407 is overlaid onto the portion 410 in the viewer window 401 .
  • a scale 408 is overlaid onto the portion 410 displayed in the viewer window 401 .
  • Magnification adjustment controls 409,411 U,411 D are overlaid on the portion 410 to allow adjustment of the portion 410 of the sample displayed in the viewer window 401 .
  • a slider 409 can be selected by a user input device and adjusted up or down to magnify or demagnify the portion 410 of the sample displayed.
  • An up button 411 U can be pressed by the user input device to zoom in on the sample portion displayed in the viewer window 401 .
  • a down button 411 U can be pressed by a user input device to zoom out on the sample portion displayed in the viewer window 401 .
  • the side bar 402 can indicate the various objects (e.g., infected cells, uninfected cells, subcells) recognized and displayed in the viewer window 401 .
  • the side bar can also indicate the Al model that is used to generate the objects and the date and time the analysis is run with the Al model.
  • the side bar can also indicate the user provided metadata that was used to train the Al model.
  • FIG. 4B The magnified portion 410 of the sample shown by the viewer 401 is illustrated in Figure 4B. Over that of the viewer window of Figure 2A, objects are detected, classified and shown by the different colors after an analysis.
  • the web based platform has built-in support for sharing and collaborating among scientific teams. Because it is substantially web-based, no complex software installations are required to get started analyzing biological cells. The capabilities of the webbased scalable image platform 10 make it easy to use the system for the early detection of changes in cellular structure.
  • the disclosed embodiments train imaging artificial intelligence models (Als) to detect individual, unlabeled cells infected with virus. This is innovative because machine learning directly on brightfield images without stains or other additional preparations has never been accomplished for virus infectivity assays. This allows such assays to be conducted on platforms compatible with HCA automation workflows, without the need of costly and time-consuming additional staining, sample preparation or development of transgenic viruses carrying fluorescent protein genes. Furthermore, the insights gained during the development of the models are useful for other cell-based assays.
  • the developed models are shareable worldwide using a web-based platform.
  • This is innovative as model sharing in the past meant setting up complex software systems to read such models and then performing classifications based on them.
  • sharing of models merely requires a web-browser.
  • Efficient sharing of models among scientific teams is crucial for the rapid development of urgently needed vaccines or anti-virals.
  • One main objective of the embodiments is to reduce the time to obtain plaque assay results by an order of magnitude (e.g., from days to hours) via the use of machine learning.
  • panels 501-503 are shown to discuss different assay workflows.
  • Panel 501 (top panel) of Figure 5A shows a traditional assay workflow.
  • Panels 502-503 (bottom two panels) of Figure 5A show an artificial intelligence (Al) enhanced workflow with a training workflow and a classification workflow.
  • Training of Al models can use images of wells of cells infected with a virus at a multiplicity of infection (MOI) of two or greater, and images of wells of cells that are mock-infected. Subsets of these images can serve as the positive and negative classes for training the Al model. These images are captured at different times after infection ranging from a few minutes (zero hours) to forty-eight hours after infection. Images of wells with cells infected at MOIs ranging from 0.8 - 0.025 can be used for the infectivity assay validation dataset.
  • the multiplicity of infection (MOI) can be defined as the ratio of the number of live infectious virus particles per cell.
  • Both traditional and Al workflows start with an array of a plurality of incubated wells 505XY containing infected cells.
  • the plurality of wells 505XY can be in a tray (plate) 504 and organized by column X and row Y.
  • a serial dilution of infected cells can be placed in the wells of each column along a row.
  • the serial dilution of infected cells in the wells can be repeated in one or more rows so that a plurality of rows of serial diluted cells are present in the wells of the tray at the start of incubation 504A.
  • a plurality of digital images 513A of each well in the tray 504 is captured by an image capture device 512, such as a microscope or automated imager.
  • an image capture device 512 such as a microscope or automated imager.
  • a plurality of digital images 513B of each well in the tray 504 is captured by the image capture device 512.
  • additional incubation periods (hours, days) of the infected cells in the tray 504 a plurality of images can be captured at each incubation period by the image capture device 512.
  • a final set of a plurality of images 513N can be captured to the plurality of wells 505XY at the final incubation period by the image capture device 512.
  • the plurality of images 513A-513N of the wells in the tray at each period can be used to train machine learning models 514A-514N.
  • one or more of the machine learning models 514A-514N can be selected as the analysis Al models 514’ and used to classify cells in one or more wells of a tray (plate) 504 that are infected with an unknown concentration of infectious agent.
  • results are provided in a readout in the form of “holes” in the culture where plaques have formed after two to fifteen (2-15) days.
  • Results taking days to generate can be a significant bottleneck, especially when time is of the essence, such as in COVID-19 antivirus/vaccine development for example.
  • the standard viral plaque assay begins with serial dilutions of the virus sample to achieve an infection rate of five to one hundred (5 - 100) plaque forming units (PFUs) per plate or well of a multi-well plate.
  • the diluted sample is applied to a nearly confluent lawn of susceptible cells, and the cells are incubated for two to fourteen (2-14) days to allow the infection to proceed through several rounds of re-infection.
  • the plates can be stained with a viable stain (usually crystal violet) to expose the patches of dead cells that constitute each plaque.
  • Each plaque represents a single infection event followed by multiple rounds of localized virus production and re-infection.
  • the virus produced in the first round of infection is kept from diffusing throughout the well by overlaying the cells with an agarose gel.
  • the disclosed embodiments use machine learning techniques to reduce the timeline to receive assay results.
  • panel 502 of Figure 5A during a training phase, brightfield microscopy images are taken at one or more time points. These brightfield images contain thousands to millions of tissue cells and are used to train machine learning models (optionally including the final assay result as training data). It is expected that the earliest models (e.g., model 514A) will have less predictive power because the cells have not yet transformed enough from the viral infection. As part of this training phase, the earliest model is selected that provides prediction quality similar to the traditional assay.
  • Panel 503 of Figure 5A illustrates an Al powered prediction phase.
  • the selected model is then used to classify cells at a time point corresponding to the selected model.
  • the result is a per-well prediction of the degree of infection (predicted infectivity), being made within a much shorter time period (such as within hours, instead of days).
  • one disclosed embodiment can perform the following functional steps:
  • the disclosed embodiment can collect images of several hundred to a few thousand cells at each timepoint that are unambiguously positive for infection and the same number of images of uninfected cells under similar conditions.
  • Als artificial intelligence models
  • the disclosed embodiment can use artificial intelligence models (Als) to measure live virus titers with at least the same accuracy as the standard plaque assay, but in a much shorter time.
  • One or more disclosed embodiments includes a platform to automate plaque assays, use brightfield images without viable stains or molecular markers, and get an infectious virus titter in hours instead of days.
  • Figure 5B is a block diagram of the artificial intelligence (Al) workflow (process) 550, including a training workflow (process) 551 and a classification (analytical) workflow (process) 552, for predicting viral infectivity of samples of biological cells under test.
  • the training workflow 551 After images of the biological cell samples are captured, the training workflow 551 generates a background artificial intelligence model 562 and a cell ensemble artificial intelligence model 568 that can be used by the classification workflow 552 to estimate or predict viral infectivity in a biological cell sample under test.
  • the ensemble model is a machine learning technique that combines several base Al models together in order to produce one optimal predictive Al model. The goal is to find a single model that will best predict the desired outcome. Rather than making one model, ensemble models take a plurality of models into account, and average those models to produce one final model.
  • the training and validation images 560 are preferably processed from raw image data to form rectangular image tiles 555 of M by N pixels.
  • the M pixels by N pixels of a tile can number in the range inclusively between thirty-two pixels by thirty-two pixels and the entire pixel width and pixel height of the one or more captured images.
  • a tile can be 128 pixels by 128 pixels or 256 pixels by 256 pixels.
  • Figure 5D illustrates rectangular or square tiles 555 for the image 554 of infected cells in a well 505XY.
  • cells can be isolated in an image area referred to as an isolated cell image.
  • tiles are preferable, the disclosed principles are applicable to both isolated cell images and tile images.
  • the Als can be trained to recognize viral infectivity on a cell-by-cell basis or a tile-by-tile basis.
  • a tile of 256 X 256 pixels is slightly larger than a typical cell, while a tile of 128 X 128 pixels (128px) is smaller than individual cells.
  • Using larger tiles can result in more accurate predictions, despite the 4-fold lower quantity of tiles in training.
  • embodiments can interchangeably use both 128px tiles as well as 256px tiles.
  • Blocks 561 and 562 train Al to prefilter background from the training and validation images 560. Because of plating limitations, some tiles can contain background that will decrease accuracy. To alleviate this problem, a “prefilter” network is used to eliminate tiles that do not have cellular material. Background filter training is performed at block 561 to produce a background model Al at block 562 which can perform a background filter step at block 572 to discard tiles that have no cellular material. An advantage of the “prefilter” network and background filter step 572, it is not necessary for training and validation images 560 to be of individually isolated cells.
  • the other workflow path of process 551 involves training one or more Als to detect morphological changes at the cellular level.
  • training an artificial intelligence model that can recognize whatever you want it to recognize in images involves exposing the Al to a dataset of images and telling the Al which subset of those images of what you want it to recognize. For example, to distinguish between dogs, cats, and stop signs, the Al would have to be told which of the images in the data set are dogs, cats, and stop signs.
  • the Al in the disclosed invention, is being trained to distinguish between infected and uninfected cells in a brightfield image.
  • no human observer can tell the difference between infected and uninfected cells at this level of magnification and time of infection.
  • an Al can distinguish morphological changes in a cell that would elude a human observer even when both are ‘looking” at the same images.
  • Brightfield images, without the use of stains and dyes, contain too much information for a human observer to recognize changes.
  • Als are better at synthesizing images to recognize the differences.
  • the Al training has to be established in a specific way to train the Al to distinguish what a human observer cannot.
  • the method of training one or more artificial intelligence (Al) models to differentiate between infected cells and uninfected cells begins by exposing the Al to a dataset of images of stain free cells infected with a known virus stock and images of stain free uninfected cells.
  • the Als have to be trained in a specific way that is different from traditional transfer learning methods where different types of images are already known.
  • cats, dogs, and stop signs are already known to the observer, thus a traditionally trained Al can simply be told which images are dogs, cats, and stop signs.
  • Human observers cannot distinguish between infected and uninfected cells at the magnification and timepoint after infection utilized in embodiments of the invention.
  • the training dataset consists of images of wells infected at a known high multiplicity of infection (MOI) ratio such that all or nearly all cells in the infected cells are infected with virus and none of the cells in the mock infected sample are infected.
  • MOI multiplicity of infection
  • the dataset used in one embodiment of the invention consisted of images of wells infected at a multiplicity of infection (MOI) of 2 for the infected cells used in training and 0 MOI for the mock infected samples.
  • MOI multiplicity of infection
  • the training images are periodically captured at predetermined times over an incubation period ranging from zero until such time that cytopathic effects (CPEs) can be manually observed after initial infection.
  • CPEs are structural changes in the host cells caused by viral infection, e.g., rounding of the cell. If the virus is known to lyse nearly all of the infected cells in 12 hours, than the training images would not be captured beyond 12 hours.
  • the Al models were trained at 1 , 4, 8, and 12 hours timepoints after initial infection.
  • the preferred method of training an Al model for viral assays is to train level-1 , individual Als independently in parallel, with the results brought together to train a level-2 ensemble Al.
  • the ensemble Al model 568 uses a two-layer stacking approach to training.
  • a plurality (e.g., 12) level-1 cell Al models 565A- 565N are each trained during cell class training 564A-564N using configuration files specifying starting models and their training parameters.
  • the resultant trained cell Al models 565A-565N are used to generate predictions on the images of biological cells withheld from the training images.
  • These predictions of infectivity are used during ensemble training to train the level-2 classifiers, the cell ensemble model 568.
  • the level-2 classifiers are in turn used to make predictions of infectivity on the separate validation images of the biological cells.
  • Disclosed embodiments of the invention have successfully assayed enveloped and nonenveloped RNA and DNA viruses.
  • the virus stock used in embodiments of the invention contains virus of the group of DNA (coated or uncoated) and RNA (coated or uncoated).
  • the table above lists the various types of viruses accurately classified using the disclosed Al assisted assay system and method of the invention.
  • An alternative approach to machine learning involves feature-based classification using CHARM features (a large feature set used successfully in many cell-based Al applications, constituting more than 4,000 numerical image features). These features can be combined with the automated featureclassifier trainer in the final ensemble stage. The trainer can automatically select a pipeline of feature normalization, scoring, selection and classification algorithms from scikit-learn, and automatically optimizes their parameters. Several variants of this technique can be used to supplement the CNNs.
  • the training the Als can be executed manually via a number of scripts.
  • each stage of the trai ni n g/cl assification process is performed as an analysis module in a computer server system cloud platform (e.g., see Figure 6), and automates the overall workflow composed of constituent software modules on a compute cluster. This ensures that the data flow of intermediate results of the training/classification workflow are fed correctly into subsequent analyses in an automatic manner, rather than managing data flow manually.
  • the disclosed embodiments also provide for storage of trained models as outputs of Al training modules and allows them to serve as inputs in subsequent prediction steps. This ensures that the entire dataflow is specified as part of the workflow, which not only aids in organizing various Al models trained using various datasets and parameter sets, but prevents inadvertent mistakes that are possible when these flows consist of potentially thousands of manually handled individual files.
  • An automated feature classifier trainer tries several different feature selection and scoring techniques coupled to several different classifiers, tries a range of appropriate parameters for these algorithms and selects the best-performing model and corresponding parameter set.
  • the ensemble model resulted in an accuracy of 81 % for 128px tiles, compared to an average of 70% and max of 77% for the individual constituent models.
  • the ensemble network achieved an accuracy of 88%, compared to an average of 82% and max of 86% for the individual constituent models. Varying the parameters ensured the constituent Al models are trained as differently as reasonably possible so that their individual results are as different as possible.
  • the ensemble Al model can be validated. Validation compares the Al model prediction of a known virus’ infectivity against the same known virus’ infectivity determined by a traditional plaque assay.
  • the Al models are trained in the context of a high-MOI training set, it is important to validate these models in the context of a real assay where the MOIs are ⁇ 1.0.
  • the predictions of the Al models are compared to the results of traditional assays of cells with MOIs in a series of viral dilutions of a stock with known titer. For 256px tiles, the ensemble Al model can achieve an accuracy of eighty percent or more.
  • Al models background model 562, and cell ensemble model 568, trained, they can be used in a classification process of infected biological cells captured in plate images 570.
  • heretofore unseen images 570 are classified with objects of the samples in the wells being recognized and counted.
  • the images 570 can be from a single infection in a single well of a typical ninety-six well plate.
  • the images are analyzed and predictions are made by the Al model and the associated machine learning algorithms. These plate images 570 are not used in a training process.
  • the plate images 570 are prefiltered at step 572 by the background model 562 to remove background “noise”, such as any non-cellular material for example, and generate prefiltered images 574.
  • the prefiltered images 574 are analyzed by the ensemble model 568 during the cell (object) classification step 578. This can be performed in parallel on tiles of images.
  • the analyzed images are assigned a score for each tile in the image at process step 579.
  • the scores are aggregated together or averaged across all the tiles in the image and all the images in a well for a given dilution within the linear range of MOI determined by the validation step. The aggregated score is then used to predict an MOI estimate at block 585.
  • the analysis or classification process step 552 can be used for viral assays such as an inactivation assay.
  • the Als are trained and validated on a known virus. Cells infected with the known virus can be subjected to an inactivation agent and then analyzed in process step 552. A researcher can determine from the MOI estimate 585 how effective was the inactivation agent against the known virus.
  • embodiments of the invention can be used for oncolytic targeting, clearance assays, diagnostic assays, and creating biocides, vaccines and antivirals.
  • FIG. 5C illustrating a workflow diagram contrasting a traditional plaque assay with an Al assisted assay.
  • a first step 582 of both traditional and Al assisted assays biological cells are plated into wells of a play or tray.
  • the plated cells are infected with a virus to form infected cells.
  • the choice of cell lines and viruses are not particularly material to the principles of the disclosed embodiments.
  • the scope of the invention is not limited to a particular cell line or viruses with the caveat that the cell line is susceptible to infection by the virus.
  • the principles of the disclosed embodiments can be applied to any virus and any cell line susceptible to infection by that virus.
  • plaque assay is a critical tool for viral isolation and quantitation.
  • the number of infectious viral particles in a sample is measured in ‘plaque-forming units’ or pfu. Titration of a viral suspension is carried out by serial dilutions of a known virus. The purpose of making and testing serial dilutions is to achieve a “countable” number of plaques in the cell monolayer. If too many infectious particles are present, individual cells are likely to be infected with more than one virion or individual plaques will merge to form large areas lacking cells.
  • a monolayer of a cell line is plated typically in a soft agar overlay on a petri dish containing the appropriate nutrient agar.
  • the viral suspension titer is added to a monolayer of cells susceptible to infection by the virus used.
  • An agarose overlay is applied to the cell monolayer after infecting the monolayer with the virus. The agarose overlay immobilizes viruses and prevents cross-contamination among plaques.
  • Over the timeline 504A-504N the plated infected cells are then incubated for a period of two to fourteen days.
  • a monolayer of cells is plated into wells 505XY of the imaging plate (tray) 504.
  • the monolayer of cells is infected with a viral titer to form infected cells.
  • the plated cells are infected with a virus to form infected cells.
  • the incubation 518A period 581A-518B can occur. The infected cells need only be incubated for a few hours (e.g., such as between a range of four to forty-eight hours).
  • the infected cells are imaged at step 587 using a plate imager (image capture device, imaging device or digital imager) 512.
  • a plate imager image capture device, imaging device or digital imager
  • the disclosed embodiments can use an automated plate imager capable of producing brig htfield images with 20X or 40X magnification.
  • a preferred image collection strategy is to work with a user group working with COVID-19 that are performing plaque assays as part of their normal operations.
  • the user group typically has a microscope 512 able to image the whole surface of a well at high resolution (subcellular resolution) using phase-contrast with 40x-60x resolution. These types of microscopes are commonly available in imaging facilities, but they may be less available under biosafety level three (BSL-3) conditions.
  • BSL-3 biosafety level three
  • the purpose of imaging each entire well is to use the normal plaque readout at the end-point to localize the plaques at plaque locations 516, and then use those same areas 516 of the well in the previous timepoints to obtain images of infected cells prior to any changes being visible.
  • the plaque locations 516 at the end point can be used as another Al model training input as part of a data collection strategy.
  • the plate images 513A-513N and metatdata are then uploaded to a cloud Al platform 10 where the images are processed.
  • the image processing step 589 can include, downsampling, tiling, filtering background, etc.
  • Image processing 589 and classification steps 590 are preferably performed in the cloud-based (web-based) platform 10.
  • the processed image tiles are given to the trained Al for a classification or analysis step 590.
  • the classification step 590 can be performed in parallel by multiple processors in a computer cluster, such as provided by a server computer (see server computer 604 in Figure 6).
  • the Al scores for the image tiles and the metadata (annotated objects) can be aggregated together.
  • a report of the viral infectivity of the cells is generated.
  • the report can then be sent back to the client or laboratory, such as by email.
  • the report can be reviewed and the user interface with the viewer window can be viewed by the user.
  • the Al based assay timeline can be completed in one day or less.
  • the disclosed embodiments have been developed to receive images from existing COVID-19 user groups. It alters the method for existing plaque assay with additional imaging.
  • the additional imaging posed additional challenges beyond the limited availability of high-resolution whole-well imagers in biosafety level 3 (BSL-3) facilities.
  • Phase contrast and other brightfield methods used to develop contrast in transparent samples rely on structured light transmission through the sample and into the objective.
  • the agarose overlay typically used in a plaque assay, is not normally required to preserve optical properties of structured light, and so its thickness, optical clarity and composition may interfere with the imaging.
  • it is possible to image through agarose overlays as these are used in chemotaxis studies and studying chemical irritants in cell culture models at high resolution. While these imaging challenges were ultimately surmountable, they did impose additional sample handling considerations on the standard plaque assay.
  • Figure 5D illustrates a plate or tray 504 with a plurality of X columns and a plurality of Y rows of wells 505XY.
  • a typical tray may have 12 columns and 8 rows for a total of 96 wells.
  • An imaging device can be used to capture a plurality of high resolution digital images 554 over each of the plurality of wells 505XY in the plate or tray.
  • the digital images 554 of each well can include infected biological cells.
  • digital images 554 of a single well, a slide, or a petri dish may be captured with infected biological cells.
  • a digital image 554 of a portion of a well 505XY can be further partitioned into a plurality of tiles 555.
  • Each tile 555 can be rectangular or square with dimensions of M by N pixels.
  • the M pixels by N pixels of a tile can number in the range inclusively between thirty-two pixels by thirty-two pixels and the entire pixel width and pixel height of the one or more captured digital images 554.
  • the dimensions of a tile can be 128 x 128 pixels or 256 x 256 pixels.
  • the tiles are preferably confluent with no gaps or overlaps.
  • Figure 5E illustrates a validation chart with plots to compare infectivity determined with an Al model and machine learning algorithms versus infectivity determined with traditional assay methods.
  • On the X axis is the known virus with the known MOI determined by traditional assay methods, such as by plaque assays or a median tissue culture infectious dose (TCID50) assay.
  • On the Y axis are the predicted infectivity of the known virus as predicted by the trained Al.
  • the linear region (in green) was determined from the intercept of the linear function extending from the inflection point and the upper and lower asymptotes.
  • the linear range of the MOI was determined to lie between 0.05 and 0.60 PFU/Cell for this exemplary validation step.
  • the linear range can be used by the researcher to determine which dilution of the known virus sample can be used to get a good linear response in the actual viral assay.
  • the validation step determines an effective MOI range of the assay by validating the one or more artificial intelligence (Al) models against a series of dilutions of a known titer of the known virus stock at an incubation time point determined during the training period.
  • Al artificial intelligence
  • An alternative disclosed embodiment for collecting images of early stages of infection is disclosed herein. Multi-well plates of cells are infected with a sample of pre-tittered virus such that the infection rate is 1-3 PFU/cell.
  • the disclosed embodiments are inherently automated. Previous viral assays included many manual processing steps that could potentially introduce human error into the results. For example, overlaying an agarose layer to prevent unwanted viral infection, is typically done manually in standard plaque assays. Overlaying the agarose layer is a complicated process step, requiring precision temperature control. By eliminating the need for an agarose layer, the disclosed embodiments can be end- to-end automated, increasing throughput and decreasing human error.
  • image data is annotated into annotated image data.
  • the annotated image data is then collated into collated annotated image data.
  • the collated annotated image data can be used to train artificial intelligence models (Als).
  • the artificial intelligence models (Als) are trained at various timepoints from the collated annotated image data.
  • the performance of the Al models is compared to the results of standard plaque assays to validate one or more of the Al models.
  • the training data is collated into labeled training sets of infection-positive cells and infection negative cells. All of the cells in a given well infected at a multiplicity of infection (MOI) of approximately one or greater (MOI of > 1) are assumed to be infection positive cells (positive). Cells in wells that were mock-infected have a multiplicity of infection (MOI) of zero (MOI of 0) and are assumed to be infection negative cells (negative).
  • MOI multiplicity of infection
  • a training set of infected cells can be collected prior to plaques being visible or detectable with a viability stain.
  • candidate cells are identified that will subsequently lyse or be dead in the final timepoint with the viability stain. This requires first aligning all of the timepoints at a cellular level. While the multi-well plate format can keep the timepoints somewhat aligned as the plate is moved from the incubator to the microscope multiple times, this typically is not sufficient to identify the same individual cells in subsequent timepoints without processing.
  • the embodiments can use several cell segmentation algorithms that work with brig htfield images.
  • affine transforms were used on the regions of interest (ROIs) defined using the visible plaques in the final timepoint to produce ROIs in previous timepoints where the progressing infection is not visible. Other areas outside of these ROIs are used to identify infection-negative cells for training.
  • ROIs regions of interest
  • Training artificial intelligence models (Als) on each timepoint independently allows us to identify timepoints that contain a sufficient number of infected cells to accurately differentiate them from uninfected cells.
  • Shrinking the ROIs at earlier timepoints can be used to mimic the spread of the infection and can reduce the number of uninfected cells in the ROI to improve the training at earlier timepoints.
  • the disclosed embodiments use multiple independent replicates, multiple timepoints post-infection, and several infectivity ratios (PFU/well for a first data collection strategy (strategy 1) and a PFU/cell or MOI for a second data collection strategy (strategy 2).
  • PFU/well for a first data collection strategy
  • PFU/cell or MOI for a second data collection strategy
  • Al performance can increase as the infection progresses with later timepoints.
  • the assay’s practical value is maximized at the earlier timepoints. For this reason, Al performance is evaluated separately at each timepoint.
  • six to eight (6-8) timepoints, with three to five (3-5) virus dilutions and three to five (3-5) replicates, is sufficient for Al model training.
  • Each timepoint and dilution is used to train a separate Al model (18-40 total), using all but one replicate for training, and the remaining replicate for testing.
  • Round-robin leave-one-out testing multiplies the total number of Al models by a factor of three to five (3-5) and can produce a measure for the variance in Al performance.
  • CNNs convolutional neural nets
  • Als artificial intelligence models
  • Similar results have been obtained with CNNs.
  • Data can be reduced to train CNN artificial intelligence models by the use of transfer learning from an unrelated imaging problem, or the use of data augmentation.
  • the Al based assay can be validated in its final intended form to see if the accuracy is maintained.
  • the new assay can be tested on standard multi-well plates without agarose overlays to see if the Al model training holds across this change in conditions.
  • a new infection at an MOI much less than 1 is performed to mimic a standard plaque assay. Because the plaques are now single cells, the system is able to tolerate a much wider range of PFUs per well than a standard assay. This can be demonstrated by finding the individual infected cells in a lawn of otherwise healthy cells, and that the infected cell count can substitute for a standard plaque count.
  • the web based platform was originally based on BisQue. However, web based platform has been rewritten to make significant improvements over BisQue in many areas including: scalable viewers for data of any size, proprietary storage formats supporting millions of identified objects per image, and scalable deployments. Furthermore, several platform enhancements permit better utilization of resources and faster adoption of novel assays in order to speed the generation of results.
  • the web based platform has been designed to support multiple machine learning frameworks for both comparison and future-proofing.
  • Caffe deep learning framework is supported.
  • Added support for both TensorFlow and PyTorch/Caffe2 is provided in order to compare the effect of the framework training on result quality.
  • transfer learning based on each deep learning framework preconfigured neural networks and datasets. Given the levelling of use that the web based permits, different frameworks can be quickly compared. Different neural network topologies can be further used as data becomes available.
  • Al assisted viral assays can simplify and speed up the process of determining viral infectivity.
  • the principals disclosed can also be applied to specific types of viral assays, such as diagnostic assays or inactivation assays.
  • Diagnostic assays are infectivity assays where the virus being sampled is an unknown virus.
  • the virus can be a sample taken from a swab of an infected person. The identity of the virus can be suspected but is not definitively known. If the virus sample is suspected to be a known virus type than the infectivity of the virus sample can be classified by Al trained on the known virus type. For a virus sample that is not suspected to be a known virus, a more generalized Al trained on a plurality of viral stocks can be used to classify the unknown viral sample. Similarly, a plurality of Als can be trained on a plurality of viruses.
  • Each Al model can report out the probabilities that the cells were infected with the one virus to which the Al model was trained for based upon a comparison of the predicted MOI of the unknown sample and MOI of the virus the Al was trained for.
  • Another variant of a diagnostic assay can be performed in conjunction with biopsies to determine where the viral load is in a patient’s body by sampling various tissues and fluids from the patient. In this case the virus is known or suspected, and an Al assisted infectivity assay can determine the location where the virus is localized.
  • inactivation assays seek to determine the effects of a treatment or procedure on the infectivity of a known virus sample.
  • a treatment or procedure is conducted on the known virus sample or the infected cell which will affect the infectivity of the viral sample.
  • Treatments and procedures can be with antivirals, biocides, gene modification of the infected cell or virus, drugs, inactivating antibodies in patient sera, etc.
  • a treatment can modify the genes of the virus to express genes for gene therapies.
  • a treatment can modify the genes of the virus to affect the virus's targeting of cells for infection (e.g., oncolytic therapies).
  • a treatment can modify the genes of the virus to modulate the virus's infectivity.
  • a treatment can modify the genes of the virus to modulate the immune response in a host patient (e.g., vaccine development and oncolytic immunotherapies).
  • the MOI of the viral sample is determined before and after the treatment or procedure to quantify the effects on the infectivity of the viral sample.
  • Embodiments of the disclosed invention can determine the MOI of the viral sample using Al to differentiate infected and non-infected cells before and after treatment.
  • Subsets of inactivation assays include clearance assays, where a pharmaceutical manufacturing process step is likened to the procedure or treatment in question.
  • the inactivation agent is the process step.
  • a known virus is introduced into the pharmaceutical manufacturing process step and the effects of the process on the amount of the virus before and after the process step must be determined.
  • Determination of viral infectivity without an inactivation step is required when characterizing new preparations of a known virus for experimental or therapeutic use, or when characterizing new strains developed as candidates for use in gene therapy, or for determining the effectiveness of targeting of a virus to specific cells, such as used for oncolytic viral therapy.
  • diagnostic assays and inactivation assays are disclosed embodiments, it is to be understood that the disclosed embodiments are not limited to these two types of assays.
  • an Al assisted viral assay is a disclosed embodiment and the principles disclosed are applicable to using Al and Al models to quantify viral infectivity in a viral sample for any purpose.
  • the client-server computer system 600 includes a plurality of client computers 602A-602N in communication with one or more computer servers 604 in a server center (or the cloud) 606 over a computer network 608, such as a wide area network of the internet.
  • the web-based scalable image analysis platform 610 for biological cells can be executed on the one or more computer servers 604 for access by the plurality of client computers 602A-602N.
  • FIG. 7 a block diagram of a computing system 700 is shown that can execute the software instructions for the web-based scalable image analysis platform 610 for biological cells.
  • the computing system 700 can be an instance of the one or more servers executing stored software instructions to perform the functional processes described herein.
  • the computing system 700 can also be an instance of a plurality of instances of the client computers in the wide area network executing stored software instructions to perform the functional processes described herein of a client computer to provide and display a web browser with the various window viewers described herein.
  • the computing system 700 can include a computer 701 coupled in communication with a graphics monitor 702 with or without a microphone.
  • the computer 701 can further be coupled to a loudspeaker 790, a microphone 791 , and a camera 792 in a service area with audio video devices.
  • the computer 701 can include one or more processors 710, memory 720; one or more storage drives (e.g., solid state drive, hard disk drive) 730,740; a video input/output interface 750A; a video input interface 750B; a parallel/serial input/output data interface 760; a plurality of network interfaces 761 A-761 N; a plurality of radio transmitter/receivers (transceivers) 762A- 762N; and an audio interface 770.
  • the graphics monitor 702 can be coupled in communication with the video input/output interface 750A.
  • the camera 792 can be coupled in communication with the video input interface 750B.
  • the speaker 790 and microphone 791 can be coupled in communication with the audio interface 770.
  • the camera 792 can be used to view one or more audio-visual devices in a service area, such as the monitor 702.
  • the loudspeaker 790 can be used to communicate out to a user in the service area while the microphone 791 can be used to receive communications from the user in the service area.
  • the data interface 760 can provide wired data connections, such as one or more universal serial bus (USB) interfaces and/or one or more serial input/output interfaces (e.g., RS232).
  • the data interface 760 can also provide a parallel data interface.
  • the plurality of radio transmitter/receivers (transceivers) 762A-762N can provide wireless data connections such as over WIFI, Bluetooth, and/or cellular.
  • the one or more audio video devices can use the wireless data connections or the wired data connections to communicate with the computer 701 .
  • the computer 701 can be an edge computer that provides for remote logins and remote virtual sessions through one or more of the plurality of network interfaces 761 A-761 N. Additionally, each of the network interfaces support one or more network connections. Network interfaces can be virtual interfaces and also be logically separated from other virtual interfaces. One or more of the plurality of network interfaces 761 A-761 N can be used to make network connections between client computers and server computers.
  • One or more computing systems 700 and/or one or more computers 701 can be used to perform some or all of the processes disclosed herein.
  • the software instructions that performs the functionality of servers and devices are stored in the storage device 730,740 and loaded into memory 720 when being executed by the processor 710.
  • the processor 710 executes instructions residing on a machine-readable medium, such as the hard disk drive 730,740, a removable medium (e.g., a compact disk 799, a magnetic tape, etc.), or a combination of both.
  • a machine-readable medium such as the hard disk drive 730,740, a removable medium (e.g., a compact disk 799, a magnetic tape, etc.), or a combination of both.
  • one or more of the video interfaces 750A-750B include graphical processing units (GPUs) that can be used to executes instructions and perform functions of the disclosed embodiments.
  • the instructions can be loaded from the machine- readable medium into the memory 720, which can include Random Access Memory (RAM), dynamic RAM (DRAM), etc.
  • the processor 710.750A-750B can retrieve the instructions from the memory 720 and execute the instructions to perform operations described herein.
  • any or all of the components and the associated hardware illustrated in Figure 7 can be used in various embodiments of a computer system 700. However, it should be appreciated that other configurations of the computer system 700 can include more or less devices than those shown in Figure 7.
  • Examples of the processor readable medium include an electronic circuit, a semiconductor memory device, a read only memory (ROM), a flash memory, an erasable programmable read only memory (EPROM), a floppy diskette, a CD-ROM, an optical disk, a hard disk, a fiber optic medium, a radio frequency (RF) link, etc.
  • the computer data signal may include any signal that can propagate over a transmission medium such as electronic network channels, optical fibers, air, electromagnetic, RF links, etc.
  • the code segments may be downloaded using a computer data signal via computer networks such as the Internet, Intranet, etc. and stored in a storage device (processor readable medium).

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • General Physics & Mathematics (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Public Health (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Epidemiology (AREA)
  • Molecular Biology (AREA)
  • Primary Health Care (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Pathology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Biophysics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • General Business, Economics & Management (AREA)
  • Quality & Reliability (AREA)
  • Business, Economics & Management (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Investigating Or Analysing Biological Materials (AREA)
  • Measuring Or Testing Involving Enzymes Or Micro-Organisms (AREA)

Abstract

Sont divulgués des procédés et des systèmes d'apprentissage machine pour la détection précoce de changements morphologiques dans un état cellulaire de cellules biologiques. Selon un mode de réalisation divulgué, la mise au point de vaccins et d'agents antiviraux est accélérée à l'aide d'un apprentissage machine pour identifier des plaques virales plus rapidement qu'avec une détection à l'aide d'une observation humaine seule. Selon le mode de réalisation divulgué, la détection de changements morphologiques dans des cellules infectées par un virus peut être réalisée avant que les plaques provoquées par la mort cellulaire ne soient observables (mort cellulaire typique en 2 à 14 jours). L'apprentissage machine fournit des techniques à haute densité/haut débit pour l'étude de la virologie pour la mise au point de nouveaux composés antiviraux. L'apprentissage machine peut également être utilisé pour caractériser l'efficacité de nouveaux composés antiviraux sur des souches virales à mutation rapide, telles que la grippe et le SARS-CoV-2.
EP22750503.9A 2021-02-05 2022-02-04 Apprentissage machine pour la détection précoce de changements morphologiques cellulaires Pending EP4272155A4 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US202163146541P 2021-02-05 2021-02-05
US202163228093P 2021-07-31 2021-07-31
PCT/US2022/015379 WO2022170145A1 (fr) 2021-02-05 2022-02-04 Apprentissage machine pour la détection précoce de changements morphologiques cellulaires

Publications (2)

Publication Number Publication Date
EP4272155A1 true EP4272155A1 (fr) 2023-11-08
EP4272155A4 EP4272155A4 (fr) 2024-11-27

Family

ID=88208325

Family Applications (1)

Application Number Title Priority Date Filing Date
EP22750503.9A Pending EP4272155A4 (fr) 2021-02-05 2022-02-04 Apprentissage machine pour la détection précoce de changements morphologiques cellulaires

Country Status (1)

Country Link
EP (1) EP4272155A4 (fr)

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020112722A1 (fr) * 2018-11-30 2020-06-04 Amgen Inc. Systèmes et procédés de détection d'effet cytopathogène dans des cellules

Also Published As

Publication number Publication date
EP4272155A4 (fr) 2024-11-27

Similar Documents

Publication Publication Date Title
US20220261990A1 (en) Machine learning for early detection of cellular morphological changes
Cimini et al. Optimizing the Cell Painting assay for image-based profiling
Abels et al. Computational pathology definitions, best practices, and recommendations for regulatory guidance: a white paper from the Digital Pathology Association
EP3153588B1 (fr) Détermination rapide de la croissance microbienne et de la sensibilité antimicrobienne
Keiser et al. Automation of infectious focus assay for determination of filovirus titers and direct comparison to plaque and TCID50 assays
Taube et al. Society for Immunotherapy of Cancer: updates and best practices for multiplex immunohistochemistry (IHC) and immunofluorescence (IF) image analysis and data sharing
Ain et al. Machine learning approach towards quality assurance: Challenges and possible strategies in laboratory medicine
WO2024006308A1 (fr) Criblage automatique haut contenu utilisant l'intelligence artificielle pour le développement de composés médicamenteux
Zhao et al. Multiplex imaging in immuno-oncology
Wang et al. A clinical bacterial dataset for deep learning in microbiological rapid on-site evaluation
Yahuza et al. Recent advancements in detection and quantification of malaria using artificial intelligence
Chernomoretz et al. Using Cell‐ID 1.4 with R for microscope‐based cytometry
McMahon et al. A novel framework for the automated characterization of Gram-stained blood culture slides using a large-scale vision transformer
Hu et al. Multisite assessment of reproducibility in high‐content cell migration imaging data
Vulliard et al. BioProfiling. jl: profiling biological perturbations with high-content imaging in single cells and heterogeneous populations
Ahmadi et al. A benchmarked comparison of software packages for time-lapse image processing of monolayer bacterial population dynamics
Liu et al. Highly‐efficient differentiation of reactive lymphocytes in peripheral blood using multi‐object detection network with large kernels
Mace et al. Extraction and comparison of gene expression patterns from 2D RNA in situ hybridization images
Pengo et al. A novel automated microscopy platform for multiresolution multispectral early detection of lung cancer cells in bronchoalveolar lavage samples
Thirumal et al. TITAN: An end‐to‐end data analysis environment for the Hyperion™ imaging system
Stockwell et al. Workflow for high-content, individual cell quantification of fluorescent markers from universal microscope data, supported by open source software
Lo et al. Enhancing quantitative imaging to study DNA damage response: A guide to automated liquid handling and imaging
Kroll et al. High‐Content Microscopy Analysis of Subcellular Structures: Assay Development and Application to Focal Adhesion Quantification
Shiina et al. Importance of dataset design in developing robust U-Net models for label-free cell morphology evaluation
Ribeiro et al. Instance segmentation in medical imaging: A comparative study of cnn and transformer-based models in a teledermatology study-case

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20230804

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
A4 Supplementary search report drawn up and despatched

Effective date: 20241029

RIC1 Information provided on ipc code assigned before grant

Ipc: G16H 50/70 20180101ALI20241023BHEP

Ipc: G16H 50/20 20180101ALI20241023BHEP

Ipc: G16H 40/67 20180101ALI20241023BHEP

Ipc: G16H 30/40 20180101ALI20241023BHEP

Ipc: G16H 15/00 20180101ALI20241023BHEP

Ipc: G06N 3/08 20230101ALI20241023BHEP

Ipc: G06N 3/045 20230101ALI20241023BHEP

Ipc: G06V 20/69 20220101ALI20241023BHEP

Ipc: G06V 10/70 20220101ALI20241023BHEP

Ipc: G06V 10/82 20220101ALI20241023BHEP

Ipc: G06V 10/80 20220101ALI20241023BHEP

Ipc: G06V 10/774 20220101ALI20241023BHEP

Ipc: G06F 18/2433 20230101ALI20241023BHEP

Ipc: G06T 7/00 20170101AFI20241023BHEP