WO2024258873A2 - Analyse entraînée par ia et explication d'images médicales - Google Patents

Analyse entraînée par ia et explication d'images médicales Download PDF

Info

Publication number
WO2024258873A2
WO2024258873A2 PCT/US2024/033449 US2024033449W WO2024258873A2 WO 2024258873 A2 WO2024258873 A2 WO 2024258873A2 US 2024033449 W US2024033449 W US 2024033449W WO 2024258873 A2 WO2024258873 A2 WO 2024258873A2
Authority
WO
WIPO (PCT)
Prior art keywords
image
interest
region
model
images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/US2024/033449
Other languages
English (en)
Other versions
WO2024258873A3 (fr
Inventor
Alma Gregory Sorensen
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
DeepHealth Inc
Original Assignee
DeepHealth Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by DeepHealth Inc filed Critical DeepHealth Inc
Publication of WO2024258873A2 publication Critical patent/WO2024258873A2/fr
Publication of WO2024258873A3 publication Critical patent/WO2024258873A3/fr
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H15/00—ICT specially adapted for medical reports, e.g. generation or transmission thereof
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/30—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10116—X-ray image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10132—Ultrasound image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30028—Colon; Small intestine
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30068—Mammography; Breast
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/67—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for remote operation

Definitions

  • Tire effectiveness of screening mammography depends on the ability of radiologists to provide quality interpretations by recalling patients with cancer (sensitivity) while limiting the number of patients recalled without cancer (specificity). Close to 50% of radiologists have unacceptable interpretation perfonnance either in terms of sensitivity or specificity.
  • the Breast Cancer Surveillance Consortium evaluated 323 MQSA-qualified radiologists: separately, the National Mammography Database (Lee et al. 2021) evaluated 1223 MQSA-qualified radiologists. Both evaluations showed unacceptable cancer detection rates (low' sensitivity) and/or unacceptable recall rates (low specificity) among more than 40% percent of practitioners. The impact of this is that patients receive highly variable care.
  • Cancerous lesions are rare and often indicated by small, subtle features (e.g.. microcalcifications, lesions partially masked by dense tissue, lesions at the edge of the image). Further contributing to variability is the practical reality that there is limited expertise available to interpret mammograms effectively. Estimates that, at most, 30% of screening mammograms are interpreted by breast imaging experts continue to mirror national practice. And even when experts are available, such experts can miss cancers. In many locations, especially rural locations, interpretation quality may be even worse given that screening mammograms are interpreted by general radiologists who only spend a limited amount of time interpreting mammography despite being MQSA-qualified. This problem of limited expertise is growing as the number of experts is not keeping pace with tire need and the general shortage of radiologists is growing.
  • CDR cancer detection rate
  • RR recall rate
  • Al artificial intelligence
  • LLMs large language models
  • a user to submit a prompt in the form of a question on various topics, or even submit an image, and receive a text-based output as a response.
  • use of LLMs present challenges.
  • One such challenge is the ability’ for an LLM to “hallucinate” or otherwise create an erroneous or inaccurate response.
  • Such responses may appear accurate or legitimate and the inaccuracy may in some cases only be detected upon further analysis by a trained individual in the topic at issue.
  • the present disclosure addresses the aforementioned drawbacks by providing methods, systems, and non-transitory computer readable media for use of an Al model (c.g., an LLM) to generate an explanation of an output of an abnonnality -detecting Al model that processes medical images.
  • an Al model c.g., an LLM
  • the abnormality-detecting Al model may be a proven model (e.g., FDA approved) for detecting abnonnalities, the risk of a misdiagnosis by a model not specifically trained for such functions is avoided. Further, by using the Al model (e.g., an LLM) to generate an explanation of the output of the abnormality-detecting Al model, an individual (e.g., a patient, a doctor, radiologist, pathologist, or another medical professional) may obtain a better understanding of the output of the abnormality-detecting Al model.
  • a proven model e.g., FDA approved
  • the methods and systems include a memory and a processor configured to access the memory and to receive one or more images of tissue for a patient; apply the one or more images to a first artificial intelligence (Al) model to obtain a malignancy score for a region of interest on a first image; apply the first image and the malignancy score for the region of interest on the synthetic image to a second Al model to obtain a probability output; convert the probability output to an explanation text; and provide a report including the explanation text for the region of interest on the synthetic image corresponding an abnonnality in the tissue for the patient.
  • Al artificial intelligence
  • the methods and systems include a memory and a processor configured to access the memory and to receive one or more training images of tissue; receive a ground truth malignancy score for tire region of interest in a first image, the first image being a synthetic image or an image of the one or more training images; train a first Al model based on the first image and the ground truth malignancy score by adjusting first hyperparameters of the first Al model to produce a first output closer to the ground truth malignancy score for the region of interest in the first image; receive a ground truth explanation for a malignant tumor in the tissue corresponding to the ground truth malignancy score for the region of interest; and train a second Al model based on the first image and the ground truth explanation by adjusting second hyperparameters of the second Al model to produce a second output closer to the ground truth explanation.
  • the methods and systems include a memory configured to store a plurality of images of the breast tissue and a processor configured to access the memory and to provide each image of tire breast tissue to a first artificial intelligence (Al) model comprising a first trained neural network that is trained to identify regions of interest; receive a first identified region of interest from the first Al model corresponding to a first image of the plurality of images of the breast tissue; receive a second identified region of interest from the first Al model corresponding to a second image of the plurality of images of the breast tissue; generate a synthetic image by populating a pixel array of the synthetic image based on at least the first identified region of interest and the second identified region of interest; provide the synthetic image to a second Al model comprising a second trained neural network; determine a malignancy likelihood score using tire second Al model; provide at least one of the plurality of images of the beast tissue, the first region of
  • Al artificial intelligence
  • FIG. 1 is a block diagram of an example system for abnonnality analysis and explanation.
  • FIG. 2 is a block diagram of example components that can implement the system of FIG. 1
  • FIG. 3 is a block diagram that shows an example x-ray imaging system.
  • FIG. 4 is a block diagram that shows an example embodiment of a model for generating regions of interest for a two-dimensional slice of three-dimensional tomosynthesis data.
  • FIG. 5 is a further block diagram that shows an example secondary model for generating an Al score based on a malignancy likelihood score.
  • FIG. 6 is a diagram of an example workstation used by a clinician.
  • FIG. 7 is a block diagram of an example processor for implementing the instructions of the malignancy imagery evaluation and explanation method, according to aspects of tire present disclosure.
  • FIG. 8 is an example of a flowchart describing an example method for abnormality analysis and explanation according to aspects of the present disclosure.
  • FIG. 9 is an example of a flowchart describing an example method for abnormality analysis and explanation model training according to aspects of the present disclosure.
  • FIG. 10 is another example of a flowchart describing an example method for malignancy analysis and explanation according to aspects of the present disclosure.
  • FIG. 11 illustrates an example of a report that may be generated using tire flowchart of FIG. 8 according to aspects of the present disclosure.
  • the present disclosure provides methods, systems and non-transitory computer readable media for use of an Al model (e.g., an LLM) to generate an explanation of an output of an abnormality-detecting Al model that processes medical images.
  • an Al model e.g., an LLM
  • tire abnormality-detecting Al model may be a proven model (e.g., approved by the U.S. Food and Drug Administration (FDA)) for detecting abnormalities in patients, the risk of a misdiagnosis by a model not specifically trained for such functions is avoided.
  • FDA U.S. Food and Drug Administration
  • an individual e.g., a patient, a doctor, radiologist, pathologist, or another medical professional
  • the Al model e.g., an LLM
  • an individual e.g., a patient, a doctor, radiologist, pathologist, or another medical professional
  • one or more computing devices 150 can receive one or more types of patient medical imaging data (e.g., fluoroscopic imaging data, x-ray imaging data, computerized tomography (CT) imaging data, magnetic resonance imaging (MRI) data, ultrasound imaging data) from data source 102.
  • patient medical imaging data e.g., fluoroscopic imaging data, x-ray imaging data, computerized tomography (CT) imaging data, magnetic resonance imaging (MRI) data, ultrasound imaging data
  • computing device 150 can execute at least a portion of a malignancy detection evaluation system 104 to classify tissue imaging data (e.g., breast tissue imaging data, etc.) received from the data source 102 and/or to generate feature data, maps, or synthetic images based on the tissue imaging data received from the data source 102.
  • tissue imaging data e.g., breast tissue imaging data, etc.
  • computing device 150 can execute at least a portion of a malignancy explanation system 106 to provide an explanation based on an output (e.g., feature data, maps, synthetic images, regions of interest in images, or malignancy scores for regions of interest in images) produced from the malignancy detection evaluation system 104.
  • the computing device 150 can communicate infomiation about data received from the data source 102 to a server 152 over a communication network 154, which can execute at least a portion of the malignancy detection evaluation system 104 and/or the malignancy explanation system 106.
  • the server 152 can return information to the computing device 150 (and/or any other suitable computing device) indicative of an output of the malignancy detection evaluation system 104 and/or the malignancy explanation system 106.
  • computing device 150 and/or server 152 can be any suitable computing device or combination of devices, such as one or more desktop computers, laptop computers, smartphones, tablet computers, wearable computers, server computers, virtual machines being executed by a physical computing device, and so on.
  • the computing device 150 may be a workstation operated or accessed by a clinician.
  • the data source 102 can be any suitable source of data (e.g., measurement data, x-ray imaging data, computerized tomography (CT) imaging data, fluoroscopic imaging data, MRI data, ultrasound imaging data, images or maps reconstructed from such data), such as an X-ray system or other suitable imaging or functional measurement device, another computing device (e.g., a server storing data), and so on.
  • data source 102 can be local to computing device 150.
  • data source 102 can be incorporated with computing device 150 (e.g., computing device 150 can be configured as part of a device for capturing, scanning, and/or storing data).
  • data source 102 can be connected to computing device 150 by a cable, a direct wireless link, and so on. Additionally or alternatively, in some embodiments, data source 102 can be located locally and/or remotely from computing device 150, and can communicate data to computing device 150 (and/or server 152) via a communication network (e.g., communication network 154).
  • a communication network e.g., communication network 154
  • communication network 154 can be any suitable communication network or combination of communication networks.
  • communication network 154 can include a WiFi network (which can include one or more wireless routers, one or more switches, etc.), a peer-to-peer network (e.g., a Bluetooth network), a cellular network (e.g., a 3G network, a 4G network, etc., complying with any suitable standard, such as CDMA, GSM, LTE. LTE Advanced, WiMAX, etc.), a wired network, and so on.
  • a WiFi network which can include one or more wireless routers, one or more switches, etc.
  • a peer-to-peer network e.g., a Bluetooth network
  • a cellular network e.g., a 3G network, a 4G network, etc., complying with any suitable standard, such as CDMA, GSM, LTE. LTE Advanced, WiMAX, etc.
  • wired network and so on.
  • communication network 154 can be a local area network, a wide area network, a public network (e.g., the Internet), a private or semiprivate network (e.g., a corporate or university intranet), any other suitable type of network, or any suitable combination of networks.
  • Communications links shown in FIG. 1 can each be any suitable communications link or combination of communications links, such as wired links, fiber optic links, Wi-Fi links, Bluetooth links, cellular links, and so on.
  • one or more computing devices 150 can include a processor 202, a display 204, one or more inputs 206, one or more communication systems 208, and/or memory 210.
  • processor 202 can be any suitable hardware processor or combination of processors, such as a central processing unit ("CPU’), a graphics processing unit (“GPU"), and so on.
  • display 204 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, and so on.
  • inputs 206 can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.
  • communications systems 208 can include any suitable hardware, firmware, and/or software for communicating information over communication network 154 and/or any other suitable communication networks.
  • communications systems 208 can include one or more transceivers, one or more communication chips and/or chip sets, and so on.
  • communications systems 208 can include hardware, firmware and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.
  • memory 210 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 202 to present content using display 204, to communicate with server 152 via communications system(s) 208, and so on.
  • Memory 210 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof.
  • emory 210 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on.
  • memory 210 can have encoded thereon, or otherwise stored therein, a computer program for controlling operation of computing device 150.
  • processor 202 can execute at least a portion of the computer program to present content (e.g., images, heat maps, synthetic images, explanation, user interfaces, graphics, tables), receive content from server 152 and/or data source 102, transmit information to server 152 and/or data source 102, and so on.
  • the processor 202 executes instructions stored in the memon' 210 to implement the functionality of the computing device 150 described herein.
  • server 152 can include a processor 212, a display 214, one or more inputs 216, one or more communications systems 218, and/or memory 220.
  • processor 212 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on.
  • display 214 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, and so on.
  • inputs 216 can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.
  • communications systems 218 can include any suitable hardware, firmware, and/or software for communicating information over communication network 154 and/or any other suitable communication networks.
  • communications systems 218 can include one or more transceivers, one or more communication chips and/or chip sets, and so on.
  • communications systems 218 can include hardware, firmware and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.
  • memory 220 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 212 to present content using display 214. to communicate with one or more computing devices 150, and so on.
  • Memory- 220 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof.
  • memory' 220 can include RAM, ROM. EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on.
  • memory' 220 can have encoded thereon a server program for controlling operation of server 152.
  • processor 212 can execute at least a portion of the server program to transmit information and/or content (e.g., data, images, a user interface) to one or more computing devices 150 and/or data source 102. receive information and/or content from one or more computing devices 150 and/or data source 102, receive instructions from one or more devices (e.g.. a personal computer, a laptop computer, a tablet computer, a smartphone), and so on.
  • the processor 212 executes instructions stored in the memory 220 to implement the functionality of the server 152 described herein.
  • data source 102 can include a processor 222, one or more data acquisition systcm(s) or inputs 224, one or more communications systems 226, and/or memory 7 228.
  • processor 222 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on.
  • the one or more inputs 224 are generally configured to acquire data and can include a functional lumen imaging probe.
  • one or more inputs 224 can include any suitable hardware, firmware, and/or software for coupling to and/or controlling operations of a functional lumen imaging probe.
  • one or more portions of the one or more inputs 224 can be removable and/or replaceable.
  • the data source 102 can also include additional inputs and/or outputs.
  • data source 102 can include input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, a trackpad, a trackball, and so on.
  • data source 102 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, etc., one or more speakers, and so on.
  • communications systems 226 can include any suitable hardware, firmware, and/or software for communicating information to computing device 150 (and, in some embodiments, over communication network 154 and/or any other suitable communication networks).
  • communications systems 226 can include one or more transceivers, one or more communication chips and/or chip sets, and so on.
  • communications systems 226 can include hardware, firmware and/or software that can be used to establish a wired connection using any suitable port and/or communication standard (e.g., VGA, DVI video, USB, RS-232, etc.), Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.
  • memory 228 can include any suitable storage device or devices that can be used to store instructions, values, data, or tire like, that can be used, for example, by processor 222 to control the one or more inputs 224; to receive data from the one or more inputs 224; to generate images, heat maps, and/or computed parameters from data; to present content (e.g., images, heat maps, a user interface) using a display; to communicate with one or more computing devices 150; and so on.
  • Memory 228 can include any suitable volatile memory', non-volatile memory, storage, or any suitable combination thereof.
  • memory 7 228 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on.
  • memory 7 228 can have encoded thereon, or otherw ise stored therein, a program for controlling operation of data source 102.
  • processor 222 can execute at least a portion of the program to compute parameters, transmit information and/or content (e.g., data, images, maps) to one or more computing devices 150 and/or the server 152, receive information and/or content from one or more computing devices 150 and/orthe server 152, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone, etc.), and so on.
  • the processor 222 executes instructions stored in the memory 228 to implement the functionality of the data source 102 described herein.
  • any suitable computer readable media can be used for storing instructions for perfonning the functions and/or processes described herein.
  • computer readable media can be transitory or non- transitory.
  • non-transitory computer readable media can include media such as magnetic media (e.g., hard disks, floppy disks), optical media (e.g., compact discs, digital video discs, Blu-ray discs), semiconductor media (e.g., random access memory' (“RAM”), flash memory’, electrically programmable read only memory (“EPROM”), electrically erasable programmable read only memory (“EEPROM”)), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and/or any suitable tangible media.
  • RAM random access memory'
  • EPROM electrically programmable read only memory
  • EEPROM electrically erasable programmable read only memory
  • transitory computer readable media can include signals on networks, in wires, conductors, optical fibers, circuits, or any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and/or any suitable intangible media.
  • Hie x-ray imaging system 300 which may be a 3D digital breast tomosynthesis (DBT)
  • DBT digital breast tomosynthesis
  • Hie x-ray imaging system 300 can include an x-raysource assembly 308 coupled to a first end 310 of an arm 302.
  • An x-ray detector assembly 312 can be coupled proximate an opposing end 314.
  • the x-ray source assembly' 308 may extend substantially perpendicular to the arm 302 and be directed toward the x-ray detector assembly 312.
  • the x-ray detector assembly 312 also extends from the arm 302 such that the x-ray detector assembly 312 receives x-rayradiation produced by the x-ray source assembly 308, transmitted through the breast, and incident on the x- ray detector assembly 312.
  • a breast support plate 316, and a breast compression plate 318, are positioned between the x-ray source assembly 308 and tire x-ray detector assembly 312.
  • the x-ray source assembly 308 may be stationary or movable.
  • the x-ray imaging system 300 can generate a reconstructed image including 3D DBT data.
  • the 3D DBT data can include a number of 2D slices.
  • the reconstructed image can include 3D tomography data including a plurality of 2D slices having a thickness of about 1mm.
  • the 2D slices can be used to create a synthetic 2D image, which will be described below.
  • the x-ray imaging system 300 can generate intermediate two-dimensional "slabs," representing, for example, maximum intensity' projections of a subset of 2D slices. For example, a single maximum intensity projection slab can be generated from ten slices, and multiple slabs can be generated from a plurality of slices, such as ten slabs from one hundred slices.
  • the reconstructed image can be applied as an input to a computer 326 which stores tire image in a mass storage device 328, which can include a memory.
  • the computer 326 may also provide commands to the x-ray imaging system 300 in order to control the generation of the reconstructed image.
  • a model 400 for generating regions of interest (ROIs) for a two-dimensional (2D) slice 404 of 3D tomosynthesis data is shown.
  • the ROIs may be referred to as indicators.
  • the model 400 can accept the 2D slice 404 and output any number of ROIs, for example, a first ROI including a first area 408A and a first score 412A, and a second ROI including a second area 408B and a second score 412B.
  • Each ROI can be associated with a slice number indicating the 2D slice that the ROI was generated based on, for example a fourth slice of a set of seventy-five 2D slices.
  • the slice number can be used when selecting and/or combining ROIs to create a synthetic image, as will be explained below.
  • Each ROI can include an area that can be a subregion of the 2D slice 404.
  • each 2D slice can be formatted as an array of pixels.
  • the subregion can be a subset of the array of pixels.
  • the model 400 can include one or more neural networks configured to detect objects within 2D images.
  • the objects can be ROIs.
  • the model 400 can output ROI's that follow a predetermined shape.
  • rectangular bounding boxes can be used to encompass a potential candidate for a tumor or lesion.
  • irregular shapes e.g., a “blob” of pixels
  • the model 400 can then be trained to identify rectangular-shaped ROIs including a subarray of the pixels included in the 2D slice 404.
  • the pixels of the ROI can include one or more color intensity values (e.g., a white intensity value) and a location within the 2D slice 404 (e.g., the pixel at a given (x, y) location in a 2000x1500 pixel slice). While some mammography imaging systems produce greyscale images of breast tissue, it is appreciated that the model can be used with colorized 2D slices. Each 2D slice of 3D tomosynthesis data can be the same size, such as 2000x 1500 pixels.
  • the ROI can include a relevancy score indicating how relevant the subarray of pixels is to determine a malignancy likelihood score.
  • the relevancy score can be used to create a synthetic 2D image using one or more ROIs, as will be explained in detail below.
  • Tire relevancy score can be selected from a range of values such as between 0 and 1.
  • a human practitioner can assign relevancy scores for each ROI within the range of the values.
  • the human practitioner could alternatively assign relevancy scores using a different scale, such as 0-100 (with higher scores indication higher potential for malignancy) which could then be normalized to relevancy score range used by tire model 400.
  • the human practitioner can identify ROIs as benign in order to better train the model 400 to identify potentially malignant ROIs.
  • the model 400 can include a neural network such as a convolutional neural network.
  • a training dataset including 2D data consisting of full-field digital mammography (FFDM) images and/or slices from a set of 3D tomosynthesis images and pre-identified (e.g., by one or more medical practitioners) ROIs can be used to train the model.
  • Human practitioners can identify ROIs by examining a given 2D image, outlining, using a predetermined shape such as a rectangular box, any regions that may be of interest, and assign a relevancy score to the predetermined shape based on their medical expertise and/or experience in evaluating tumors and/or lesions.
  • the relevancy score can be assigned based on pathology results that indicate whether or not a lesion is malignant.
  • a large training database can be generated by having one or more medical practitioners identify (e.g., annotate) ROIs in 2D images taken from a plurality of FFDM images or slices of 3D tomosynthesis images (e.g., images of multiple patients).
  • An advantage of using FFDM images is that there are presently more publicly available annotated FFDM images than annotated 3D tomosynthesis images. Additionally, 2D images are easier to annotate than 3D tomosynthesis images, which can require annotating a large number of individual slices included in each 3D tomosynthesis image.
  • the model 400 can receive an input 2D slice and output one or more ROIs, each ROI including an estimated relevancy score and a subarray of pixels of the input 2D slice.
  • Tire model 400 can include a number of layers such as convolutional layers. It is understood that some embodiments of tire model 400 may have different numbers of layers, a different arrangement of layers or other differences. However, in all embodiments, the model 400 can be capable of receiving an input 2D input slice and outputting any regions of interest associated with the input 2D input slice.
  • the model 400 can be a one-stage detection network including one or more subnetworks.
  • Tire model 400 can include a first subnetwork 416.
  • the first subnetwork 416 can be a feedforward residual neural network (“ResNef ’) with one or more layers 418A-C.
  • a second subnetwork 420 can be built on top of the first subnetwork to effectively create a single neural network, using the first subnetwork 416 as the backbone for the network.
  • the second subnetwork 420 can contain a plurality of layers including a first layer 422A, a second layer 422B, and a third layer 422C, though other numbers of layers (e.g., five layers) can be used, and three layers are shown for simplicity.
  • Each of the first layer 422A, the second layer 422B, and the third layer 422C can be a convolutional layer.
  • Each layer can be made of a number of building blocks (not shown).
  • Each building block can include a number of parameters layers such as three parameter layers, each parameter layer including a number of filters (e.g., 456) with a given filter size (e.g., 3x3).
  • Each of the first layer 422A, the second layer 422B, and the third layer 422C can have an associated output size such as 144x 144, 72x72, and 36x36. The output sizes can vary between input slices based on preprocessing conditions and/or parameters.
  • the second subnetwork can also include a global average pooling layer connected to a final layer (i.e., the third layer 422C), a fully-connected layer connected to the global average pooling layer, and a softmax layer connected to the fully-connected layer and having a 1 x 1 output size (i.e., a single value).
  • a global average pooling layer connected to a final layer (i.e., the third layer 422C)
  • a fully-connected layer connected to the global average pooling layer
  • a softmax layer connected to the fully-connected layer and having a 1 x 1 output size (i.e., a single value).
  • the model 400 can include a plurality of tertiary subnetworks such as a first tertiary network 424A, a second tertiary network 424B, and a third tertiary network 424C.
  • Each of the tertiary networks 424A-C can be connected to a layer of the second subnetwork 420.
  • the first tertiary network 424A can be connected to the first layer 422A
  • the second tertiary network 424B can be connected to the second layer 422B.
  • the third tertiary network 424C can be connected to the third layer 422C.
  • Each tertiary network can receive features from a layer of the second subnetwork 420 in order to detect tumors and/or lesions at different levels of scale.
  • Each tertia network can include a box regression subnetwork 426.
  • the box regression subnetwork 426 can include one or more convolutional layers 428A-B. each followed by rectified linear (ReLU) activations, and a final convolutional layer 430 configured to output regression coordinates corresponding to anchors associated with a portion of one of the layers of the second subnetwork 420 (and corresponding to an array of pixels of the input 2D slice 404).
  • the anchors can be predetermined subarrays of the various layers of the second subnetwork 420.
  • Tire regression coordinates can represent a predicted offset between an anchor and a predicted bounding box. For each bounding box included in an ROI, a set of regression coordinates (e.g., four regression coordinates) and the corresponding anchor can be used to calculate the coordinates of the bounding box.
  • Each tertiary network can include a classification subnetwork 432.
  • the classification subnetwork 432 can include one or more convolutional layers 434A-B, each followed by ReLU activations, and a final convolutional layer 438 followed by sigmoidal activations to output predictions of object presence (i.e., malignant tumor and/or lesion presence).
  • the classification subnetwork 432 can be used to obtain one or more estimations of whether or not a patient has a malignant tumor and/or lesion at various spatial locations of the 2D slice 404. More specifically, each bounding box can be associated with an estimated score output by the classification subnetwork. In some embodiments, the value of each estimated score can range from zero to one.
  • tire spatial locations can include an entire layer, i.e., first layer 422A, of the second subnetwork 420.
  • tire classification subnetwork 432 can output an estimation of whether or not a patient has a malignant tumor and/or lesion based on a 2D slice.
  • the final convolutional layer 438 can be followed by Softmax activations in models that are trained to classify multiple types of malignant regions, for example multiple levels of malignancy (e.g., low risk regions, high risk regions, etc.).
  • Tire model 400 can include an output layer 450 for normalizing data across different scales, calculating bounding box coordinates, and filtering out low scoring bounding box predictions.
  • the output layer 450 can receive outputs from the tertiary subnetworks 424A-C and output one or more ROIs, each ROI including an array of pixels scaled to the array size of the 2D slice 404 and an associated score.
  • the array of pixels can be a bounding box (e.g., a rectangular bounding box) calculated based on the regression coordinates and the anchors.
  • the output layer 450 can filter out any scores below a predetermined threshold, for example, 0.5.
  • the output layer 450 can receive outputs from the tertiary subnetworks 424A-C and output a single malignancy likelihood score.
  • the single malignancy likelihood score can be selected to be the highest scoring bounding box score.
  • a synthetic 2D image may be created using one or more ROIs generated by processing one or more images by the model 400 of FIG. 4.
  • a synthetic 2D image may be a 2D pixel array that corresponds to an x-y area of a patient also covered by each slice of a stack of 2D slices (where each slice corresponds to a different z-axis position).
  • Tire pixels from each ROI generated by the model 400 (c.g., having a relevancy score above a certain threshold) may be added to the 2D pixel array at the x-y coordinates corresponding to tire location of the ROI on the slice from which it was obtained.
  • the ROI having the higher relevancy score may be selected for adding to the synthetic 2D image.
  • the synthetic image may be a composite 2D image formed from tire ROIs of a set of images processed by the model 400.
  • the malignancy likelihood score 554 can indicate a category of risk, i.e., a low risk, medium risk, or high risk category. In some embodiments, the malignancy likelihood score 554 can be selected from a range of values, such as the integers 1-5, with 1 indicating a lowest risk level and 5 indicating a highest risk level.
  • the malignancy Al model 500 can include a neural network such as a residual convolutional neural network.
  • a training dataset including synthetic images or original images labeled as malignant or non-malignant can be used to train the model. Human practitioners can label the synthetic images. For instance, a synthetic image corresponding to a patient known to have cancer could be given a label of " I", whereas a synthetic image corresponding to a patient known to not have cancer could be given a label of “0”.
  • the malignancy Al model 500 can receive an input synthetic image and output a malignancy likelihood score indicating whether or not the breast tissue contains malignant tumors and/or lesions.
  • the malignancy Al model 500 can include a number of layers such as convolutional layers. It is understood that some embodiments of the malignancy Al model 500 may have different numbers of layers, a different arrangement of layers or other differences. However, in all embodiments, the malignancy Al model 500 can be capable of receiving an input 2D synthetic image and outputting a malignancy likelihood score.
  • the malignancy Al model 500 can be a one-stage detection network including one or more subnetworks. [0057] Briefly referring back to FIG. 4 as well as FIG. 5, the malignancy Al model 500 can include a primary subnetwork 516 and a secondary subnetwork 520, which can be the same as the primary subnetwork 416 and the secondary subnetwork 420 of the model 400 described above.
  • the primary subnetwork 516 and the secondary subnetwork 520 can be the same as the primary subnetwork 416 and the secondary subnetwork 420 after the model 400 has been trained.
  • the malignancy Al model 500 can be initialized with weights from the model 400 before training.
  • Tire malignancy Al model 500 is important because the model 400 described above may be able to detect regions of the breast tissue that are of interest but may not be able to accurately determine if the ROIs are actually malignant.
  • the malignancy Al model 500 may be used to more accurately estimate malignancy of the breast tissue using the synthetic images generated by the model 400 described above.
  • the malignancy Al model 500 can include a plurality of tertiary subnetworks, such as a first tertiary network 524A, a second tertiary network 524B, and a third tertiary network 524C.
  • Each of the tertiary networks 524A-C can be connected to a layer of the second subnetwork 520.
  • the first tertiary network 524A can be connected to a first layer 522A
  • the second tertiary network 524B can be connected to a second layer 522B
  • the third tertiary network 524C can be connected to a third layer 522C.
  • Each tertian' network can receive features from a layer of the second subnetwork 520 in order to estimate malignancy of the breast tissue at different levels of scale.
  • Each tertiary network can include a box regression subnetwork 526.
  • Tire box regression subnetwork 526 can include one or more convolutional layers 528A-B. each followed by rectified linear (ReLU) activations, and a final convolutional layer 530 configured to output regression coordinates corresponding to anchors associated with a portion of one of the layers of the second subnetwork 520 (and corresponding to an array of pixels of the input synthetic 2D slice 504).
  • the anchors can be predetermined subarrays of the various layers of the second subnetwork 520.
  • the regression coordinates can represent a predicted offset between an anchor and a predicted bounding box. For each bounding box included in an ROI, a set of regression coordinates (e.g., four regression coordinates) and the corresponding anchor can be used to calculate the coordinates of the bounding box.
  • Each tertiary network can include a classification subnetwork 532.
  • the classification subnetwork 532 can include one or more convolutional layers 534A-B, each follow ed by ReLU activations, and a final convolutional layer 538 followed by sigmoidal activations to output predictions of object presence (i.e., malignant tumor and/or lesion presence).
  • the classification subnetwork 532 can be used to obtain one or more estimations of whether or not a patient has a malignant tumor and/or lesion at various spatial locations of the synthetic 2D slice 504. More specifically, each bounding box can be associated with an estimated score output by the classification subnetwork 532. The bounding box can also be associated with a slice number as described above.
  • the value of each estimated score can range from zero to one.
  • One of the spatial locations can include an entire layer, i.e., first layer 522A, of the second subnetwork 520.
  • the classification subnetwork 532 can output an estimation of whether or not a patient has a malignant tumor and/or lesion based on a 2D slice.
  • the final convolutional layer 538 can be followed by Softmax activations in models that are trained to classify multiple types of malignant regions, for example multiple levels of malignancy (e.g.. low risk regions, high risk regions, etc.).
  • Tire malignancy Al model 500 can include an output layer 550 for normalizing data across different scales, calculating bounding box coordinates, and filtering out low scoring bounding box predictions.
  • the output layer 550 can receive outputs from the tertiary subnetworks 524A-C and output one or more ROIs, each ROI including an array of pixels scaled to the array size of the 2D slice 504 and an associated score.
  • the array of pixels can be a bounding box (e.g., a rectangular bounding box) calculated based on the regression coordinates and the anchors.
  • Tire output layer 550 can filter out any scores below 7 a predetermined threshold, for example, 0.5.
  • the output layer 550 can determine the array of pixels of each ROI based on the one or more anchors associated with the remaining scores.
  • the output layer 550 may resize the anchors in order to match the scale of the 2D slice 504, as may be necessary for anchors associated with smaller layers of the second subnetwork 520, before including the anchor as the array of an output ROI.
  • the output layer 550 can receive outputs from the tertiary subnetworks 524A-C and output the malignancy likelihood score 554.
  • the malignancy likelihood score 554 may be numerical and defined by a range (e.g., from 0 to 10, from 0 to 100, etc.) in which a higher number indicates a malignancy is more likely.
  • the malignancy likelihood score 554 is an example of an Al score output by an Al model. In some embodiments, the malignancy likelihood score 554 can be selected to be the highest scoring bounding box score. In some embodiments, the malignancy Al model 500 can output one or more ROIs 508, each including a score 508A and an array of pixels 508B. The array of pixels 508B can be a rectangular bounding box. The one or more ROIs 508 can provide additional information to a practitioner about potentially malignant regions of the synthetic image 504.
  • the workstation 602 may be operated by a clinician.
  • the workstation 602 will typically include a display 604; one or more input devices 606, such as a keyboard and mouse; and a processor 608.
  • the processor 608 may include a commercially available programmable machine running a commercially available operating system.
  • the workstation 602 provides a clinician interface that can i) receive imaging data (e.g., from the data source 102), ii) receive a request to display imaging data, iii) display received imaging data, iv) receive and output requests for radiologist assessment of imaging data, and/or v) display a report (including an explanation) or other outputs generated by the malignancy detection evaluation system 104 and/or the malignancy explanation system 106.
  • FIG. 7 shows an example of a software architecture design of the malignancy detection evaluation and explanation system.
  • the input receiver module 702 receives input data (c.g., mammogram images) from the clinical IT system (e.g., PACS) 704 and stores them in an internal storage location for subsequent processing by other modules.
  • the input receiver module 702 includes a component called a listener, which can continuously “listen” for new DICOM studies sent via the customer’s IT system (i.e., PACS) and save them to a dedicated folder, pre-specified during installation, which is accessible by a docker container.
  • the input receiver module 702 may place the received studies in a queue to be processed by the input verification & preprocessor module 706.
  • the input verification & preprocessor module 706 can perform a series of acceptance criteria checks to determine whether or not each mammogram is suitable for analysis. The module 706 can then extract the pixel data and relevant DICOM attributes. This step can ensure that the model evaluates mammography studies for which the system is indicated and configured. This step mitigates the risk of the software evaluating an exam that could give unexpected results or an exam that was not intended as a screening mammogram for clinical interpretation.
  • the input verification & preprocessor module 706 may provide a mammogram study to the malignancy detection evaluation model 708.
  • the model 708 may include models 400 and 500 described above. Additionally, the model 708 may correspond to or be implemented by the malignancy detection evaluation system 104 (FIG 1).
  • the malignancy explanation model 710 can receive the output (e.g., a synthetic image including a region of interest and a malignancy score for the region of interest) of the malignancy detection evaluation model 708 as input and produce an explanation probability output to the output generator 712.
  • the output e.g., a synthetic image including a region of interest and a malignancy score for the region of interest
  • Some examples of workflows are described in further detail below' with respect to FIGS. 8-10.
  • Tire output generator 712 can convert tire explanation probability output to an explanation text. In further examples, tire output generator 712 can further output the synthetic image, the region of interest, the malignancy score for the region of interest, and/or any other suitable information. In even further examples, the output generator 712 can output the workflow assignment for each exam, the Mammo CAD SR file for each exam, and the Breast Imaging Repository, and/or Data System (BIRADS) output for exams assigned to a “No Further Review” workflow. If the input is invalid, the outputs may not be generated.
  • BIOS Data System
  • the output sender module 714 is configurable to route the outputs to one or more clinical software systems 704. This configuration may occur at the time of installation to ensure the user outputs correctly interface with the clinical software 704.
  • the clinical software 704 may include worklist software 716, reporting software 718, and PACS software 720.
  • the malignancy explanation model 710, output generator 712, and output sender 714 may correspond to or be implemented by tire malignancy explanation system 106 (FIG. 1).
  • the components of FIG. 7 may be implemented by the system 100 of FIG. 1.
  • the model engine container, including components 702, 706-714 may be an example of one of the computing devices 150 or of the server 152 of FIG. 1.
  • the clinical systems 704 may be implemented by or across one or more of the computer devices 150 and the server 152 of FIG. 1.
  • FIG. 8 shows an example of a flowchart describing an example method for abnormality analysis and explanation, according to aspects of the present disclosure.
  • the process 800 of FIG. 8 may be executed by the system 100 of FIG. 1, or more particularly by one or more processors thereof (e.g., the processor 202 of the computing device 150 and/or the processor 212 of the server 152).
  • the process 800 can be implemented as instructions on the memory 210 in the computing device 150.
  • the computing device 150 can further include the processor 202 in communication with the memory and configured to execute the instructions.
  • the process 800 can be implemented as instructions on the memory 220 in the server 152.
  • the server 152 can further include the processor 212 in communication with tire memory 220 and configured to execute the instructions.
  • tire process of FIG. 8 may be executed by the system 100 implementing the malignancy detection evaluation and explanation system as illustrated in FIG. 7.
  • the system 100 receives one or more images of tissue for a patient.
  • the system 100 can receive the one or more images from tire data source 102, the memory 210 of the computing device, the memory 220 of the server, or any other suitable source.
  • the one or more images of tissue can include three- dimensional tomosynthesis data of the tissue for the patient.
  • the tissue for the patient with the three spatial dimensions e.g., x, y, and z dimensions
  • each image can be a two-dimensional slice image (e.g., on a x-y plane) while the multiple images can be aligned on the z dimension (e.g., a predetermined thickness).
  • the thickness can be 1 mm or any other suitable thickness of the tissue.
  • the multiple images may include about 10-150 two-dimensional slice images or more.
  • Each image can be an array of pixels of a predetermined size, such as 2,000 x 1 ,500 pixels or any other suitable size.
  • the one or more images can include a single image.
  • the one or more images can include a mammogram image, a magnetic resonance imaging (MRI) image, a computerized tomography (CT) image, a pathology slide, a video clip from a colonoscopy (or one or more image frames thereof), a video clip of a surgical procedure (or one or more image frames thereof), an image or video clip from a medical procedure, or any other suitable image.
  • the tissue can be breast tissue, and the 3D tomosynthesis data can be generated by the 3D mammography imaging system (e.g., the x-ray imaging system 300 of FIG. 3).
  • the tissue is not limited to breast tissue and can be any other suitable issue of the patient.
  • the system 100 applies the one or more images to a first artificial intelligence (Al) model to obtain a malignancy score for a region of interest on a first image.
  • tire first image can be a synthetic image (generated from the one or more images).
  • the first image is an image of the one or more images.
  • the first Al model is the malignancy detection evaluation model 708 (FIG. 7).
  • the first image can include a plurality of regions of interest being not less than a number of regions of interest of other images in the one or more images.
  • the system 100 can select the first image that has the most potential malignant regions of interest.
  • the first Al model can include multiple Al models, including, for example, an ROI Al model (e.g., the ROI Al model 400 of FIG. 4) for generating the region of interest on the first image and a malignancy Al model (e.g., the malignancy Al model 500 of FIG. 5) for determining the malignancy score of the region of interest.
  • the ROI Al model may also be referred to as a third Al model and the malignancy Al model may also be referred to as a fourth Al model, to distinguish from the second Al model referenced with respect to block 806 below.
  • the region of interest generated by the ROI Al model can be one or more regions of interest.
  • the first Al model can be a single model or include different sub-models and may receive the one or more images and produce the malignancy score for the region of interest on the first image.
  • the first Al model may be a trained model that has been trained with a training image set having medical images (e.g., mammogram images, magnetic resonance imaging (MRI) images, computerized tomography (CT) images, pathology slides, frames from video clips of a colonoscopy, frames from video clips of a surgical procedure, or any other suitable images) and associated ground truths that specify regions of interest and malignancy scores.
  • the medical images of the training image set may be similar in type to the images to be received and processed by the first Al model.
  • the regions of interest and malignancy scores may correspond to various abnormalities such as, for example, a cancer, a brain hemorrhage, or any other suitable abnormality.
  • the malignancy score can indicate not only the severity of malignancy of the region of interest but also a type of abnormality in the region of interest.
  • the system 100 can apply the one or more images to the ROI Al model to obtain the region of interest on the first image and then apply the first image to the malignancy Al model to determine the malignancy score of the region of interest on the first image.
  • the system 100 may apply an image to the ROI Al model 400 and receive as output one or more ROIs of the image and a relevancy score for each ROI. This image and an ROI of the one or more ROIs may serve as the first image and the region of interest on the first image referenced in block 804.
  • more than one image may be provided to the ROI Al model 400 to determine ROIs for each image (if present), and one of the images and associated ROIs may serve as the first image and the region of interest on the first image referenced in block 804.
  • the image having a region of interest with a highest relevancy score may be selected as the first image.
  • the ROI Al model may process one or more images to identify ROIs in the images, and a synthetic image may be generated with the ROIs from multiple images. The synthetic image having at least one ROI may then serve as the first image referenced in block 804.
  • the system 100 can generate a preliminary synthetic image without using any identified region of interest.
  • the preliminary synthetic image can be a pixel array, which is the same size as an image of the one or more images and is initialized with null values for each of pixel intensity values.
  • Tire system 100 can add one or more identified regions of interest to the preliminary synthetic image to generate the synthetic image. Hie regions of interest can be added based on one or more criteria.
  • the process can determine the region of interest with the highest score for each pixel location.
  • the system 100 may suppress all the regions of interest except for the regions of interest with the highest relevancy score to populate the synthetic image.
  • the system 100 can generate the synthetic image including the region of interest, which has the highest relevancy score in the pixel area corresponding to the region of interest in the one or more images.
  • the system 100 can fill in unpopulated regions of the preliminary synthetic image to generate the synthetic image.
  • the system 100 can perform post-processing (e.g., blending edges of the region of interest) of tire synthetic image including the region of interest.
  • the system 100 can provide the one or more images to the ROI Al model 400, receive a first identified region of interest from the ROI Al model 400 corresponding to a first image of the one or more images; receive a second identified region of interest from the ROI Al model 400 corresponding to a second image of the one or more images of the tissue; determine a region of interest for the first image based on the first identified region of interest of the first image and the second identified region of interest of the second image; and generate tire first image by populating a pixel array of the first image including the region of interest.
  • the system 100 can determine that the first identified region of interest and the second identified region of interest overlap, and determine that the first identified region of interest has a higher relevancy score than the second identified region of interest, and determine that the region of interest is to include at least a portion of the first identified region of interest on the first image in response to determining the first identified region of interest has the higher relevancy score.
  • the synthetic image i.e., the first image in this example
  • the first image can be one of the images without modification.
  • the ROI Al model is the ROI Al model 400 illustrated in FIG. 4, also referenced as the third Al model.
  • the ROI Al model 400 can include a neural network such as a convolutional neural network to identify the region of interest.
  • the ROI Al model 400 can include multiple subnetworks and/or layers (e.g.. a feedforward residual neural network 416, the second subnetwork 420 including one or more layers 422A, 422B, 422C, one or more tertiary subnetworks 424A, 424B, 424C, a box regression subnetwork 426 including a convolutional layer 428A, 428B, a classification subnetwork 432, and/or an output layer 450).
  • the subnetworks and/or layers in the ROI Al model 400 arc described further in connection with FIG. 4.
  • the system 100 can apply each image 404 of the one or more images to the ROI Al model 400 to obtain an identified region of interest in the respective image 404.
  • the ROI Al model 400 can produce more than one identified region of interest or may not produce an identified region of interest for an image.
  • an identified region of interest can include an area 408A, 408B and a relevancy score 412A, 412B.
  • the area can be a rectangular bounding box to encompass a potential candidate for a tumor or lesion).
  • the area 408A, 408B can have any other suitable shapes (e.g., circle, irregular shape, etc.).
  • the relevancy score 412A, 412B can indicate how relevant the subarray of pixels in the area 408A, 408B is to determine a malignancy score to be described below in connection with the malignancy Al model.
  • the relevancy score can be selected from a range of values such as between 0 and 1.
  • the system 100 can filter out any identified region of interest that has a score below a relevancy threshold (e.g., 0.5 or any suitable threshold value). Thus, the system 100 can reduce the computation cost by preventing the system 100 from processing the identified region of interest in the malignancy Al model or the second Al model.
  • the system 100 can further apply the first image having the region of interest (e.g., identified by the ROI Al model 400) to the malignancy Al model to obtain the malignancy score for the region of interest on the first image.
  • the system 100 can apply the first image including tire region of interest to the malignancy Al model to obtain the malignancy score in response to the relevancy score being higher than a relevancy threshold.
  • the malignancy Al model is the malignancy Al model 500 illustrated in FIG. 5, also referenced as the fourth Al model 500.
  • the malignancy Al model 500 can include a neural network such as a convolutional neural network to determine tire malignancy score for the region of interest.
  • the malignancy Al model 500 can estimate malignancy or abnormality of tire tissue using the first image generated by the ROI Al model 400.
  • the malignancy Al model 500 can include multiple subnetworks and/or layers (e.g., a primary subnetwork 516, the secondary' subnetwork 520, one or more tertiary subnetworks 524A, 524B, 524C, one or more tertiary subnetworks 422A, 422B, 422C, a box regression subnetwork 526 including a convolutional layer 528A, 528B, a classification subnetwork 532, and/or an output layer 550).
  • the subnetworks and/or layers in the malignancy Al model 500 are described in connection with FIG. 5.
  • the system 100 can obtain the first image including the region of interest based on the ROI Al model (e.g., ROI Al model 400) and can provide the first image to tire malignancy Al model (e.g., malignancy Al model 500) to obtain the malignancy score for the region of interest on the first image.
  • the system 100 can provide the first image without the region of interest to the malignancy Al model.
  • tire system 100 can provide the first image including other rcgion(s) of interest having a higher relevancy score than the relevancy threshold to the malignancy Al model.
  • the malignancy likelihood score can range from zero to one, inclusive, with one indicating a high risk of malignancy and zero indicating minimal or no risk of malignancy.
  • the malignancy can be a "yes" (i.e. 1) or "no" (i.e. 0), indicating if the tumor or lesion is predicted to be malignant or not malignant, respectively.
  • the malignancy likelihood score can indicate a category of risk, for example, a low risk, medium risk, or high risk category.
  • the malignancy likelihood score can be selected from a range of values, such as the integers 1-5. with 1 indicating a lowest risk level and 5 indicating a highest risk level.
  • the malignancy score can further include a classification score for classifying the type of the abnormality in the region of interest.
  • the classification score can identify that the type of the abnormality is a cancer, benign or malignant tumor, benign or malignant lesion, a brain hemorrhage, or any other suitable abnormality.
  • the malignancy score can indicate not only the severity of malignancy of the region of interest but also any other types of abnormality in tire region of interest.
  • the first image output by the Al model of block 804 (with ROI and malignancy score) is part of a plurality of images or video clip output by the Al model.
  • the output by the Al model may include two or more images or a video clip with two or more image frames, each with an ROI, where the ROI may correspond to the same abnormality across multiple images or frames and have a shared malignancy score, or the two or more images/image frames may have multiple ROIs associated with distinct abnormalities of the patient tissue.
  • the ROI(s) of the tw o or more images may be defined in temis of coordinates on the images (e.g., x-y coordinates) or the images may be modified with annotations showing the boundary(ies) of the ROIs on the images themselves.
  • the system 100 applies the first image and the malignancy score for the region of interest on the first image to a second Al model to obtain a probability output.
  • the system 100 applies the first image and the malignancy score to the second Al model.
  • the system 100 can apply the first image and the malignancy score to the second Al model in response to the malignancy score being above a malignancy threshold.
  • the region of interest location specified to the second Al model in tenns of coordinates with respect to the first image or the first image provided to the second Al model has the region of interested outlined (e.g., via a bounding box or other annotation provided by the first Al model) on the first image itself.
  • the first image is part of a plurality of images or video clip output by the Al model, where the ROI and malignancy score corresponding to a particular abnormality is indicated across multiple of the images or image frames of the video clip.
  • applying the first image and the malignancy score to the second Al model in block 806 may include applying the plurality of images or video clip with the malignancy score to the second Al model in block 806.
  • the region of interest and malignancy score corresponding to a particular abnormality may indicated to the second Al model for multiple of the images or image frames of the video clip (including the first image).
  • the second Al model (e.g., the malignancy explanation model 710) includes a large language model.
  • the second Al model can include a transformer.
  • the system 100 can provide the region of interest (e.g., an area of the first image, a relevancy score, etc.) and the malignancy score for the region of interest to the second Al model.
  • the system 100 may, in some examples, provide the information along with a prompt to the second Al model.
  • the prompt may indicate a request or instructions to the second Al model (e.g., “Provide an explanation for a patient that explains the first image and malignancy score for the region of interest on the first image.”).
  • Tire prompt may also include or specify parameters for the second Al model to use when generating an output (e.g., that is ultimately translated to an explanation).
  • the prompt may specify one or more of a length of an explanation to be generated, a tone for the explanation, sophistication or education level of an author being mimicked by the LLM when preparing the explanation, an explanation of characteristics of the data (e.g., image, ROI, score, etc.) that is being provided to the LLM, among other parameters.
  • Tire area may include a location of the region of interest in the first image, the size of tire first image, the tissue information included in tire first image, and other suitable information.
  • the system 100 can provide the region of interest and the malignancy score for the region of interest in sequence to the second Al model.
  • Tire second Al model can process the received input, including the first image and related information (e.g., region of interest, malignancy score, other images or image frames of which the first image is a part and was also provided to the second Al model in block 804) and the corresponding prompt. Then, the second Al model can produce a probability output based on the processed input, which can ultimately be converted to an explanation text (described below). In some examples, the probability output can include multiple probability values which correspond to tokens, one or more words, or part of words. In further examples, the system 100 can provide the first image including the region of interest as an image and the malignancy score to the second Al model.
  • the first image and related information e.g., region of interest, malignancy score, other images or image frames of which the first image is a part and was also provided to the second Al model in block 804
  • the second Al model can produce a probability output based on the processed input, which can ultimately be converted to an explanation text (described below).
  • the probability output can include multiple probability values which correspond
  • the second Al model can include multiple Al models (e.g., a transformer, convolutional neural networks (CNNs), recurrent neural network (RNN), long short-term memory (LSTM), and/or any other suitable Al model) to process the first image and to produce the probability output corresponding to the explanation text.
  • the system 100 can provide at least one of: one or more images of the tissue, the first region of interest, the second region of interest, the first image, or the malignancy score to the second Al model to generate an explanation of at least one of the first region of interest, the second region of interest, the first image, the malignancy score, or analysis performed by the first Al model or the second Al model.
  • the second Al model may include multiple internal Al models or Al sub-models that may be trained and output respective probability values that are ultimately used to generate the final explanation.
  • one Al sub-model may be trained to process images (e.g.. the first image) and indicate whether there are any quality' problems with underlying medium having the image (e.g., a film having the first image);
  • one Al sub-model may be trained to process images (e.g., the first image) and describe whether there are any benign findings; and
  • one Al sub-model may be trained to process images (e.g., the first image) and indicate whether there are findings suspicious for cancer.
  • the outputs of the second Al model, or sub-models thereof, may ultimately provide an explanation that provides specific details regarding the first image, for example, this region of interest is a mass, this region of interest is a calcification, or this ty pe of calcification is associated with malignancy in 25% cases.
  • this region of interest is a mass
  • this region of interest is a calcification
  • this ty pe of calcification is associated with malignancy in 25% cases.
  • the outputs of two or more Al sub-model of the second Al model may be concatenated or otherwise combined (e.g., strung together) to serve as the probability' output of the second Al model.
  • the system 100 converts the probability output to an explanation text (e.g., related to the region of interest of the first image).
  • the explanation text can include at least one of: a level of severity of an abnormality 1 in the region of interest (e.g., of a tumor, lesion, hemorrhage, etc.), a location of the region of interest in the first image, a practical meaning of the level of severity' of an abnormality' in the region of interest, a survival rate based on the level of severity of an abnormality in the region of interest, or a recommendation (e.g., related to an abnormality in the region of interest).
  • the practical meaning of the level of severity may include a non-technical explanation of the level of severity that does not require specialist knowledge related to the abnormality to understand.
  • the practical meaning may include a survival rate over a certain number of years, a discussion of potential treatments that may be suggested or employed to treat the abnormality, a recovery time for such treatments, an explanation of a likely outcome without treatment, an explanation of the likely impact that the abnormality is currently having on the patient, or the like.
  • the recommendation may include, for example, a suggested next step for the patient to address an abnormality in the region of interest (e.g., contact a particular type of physician or medical professional, schedule a particular type of follow up medical test or examination, obtain a particular type of treatment, a time period by which the patient should take a particular next step, or the like).
  • the explanation text can further include that there is no abnormal area indicating that no region of interest is suspicious.
  • tire explanation text can also include a location of the region of interest in the first image (e.g., where the ROI is located geographically in the image), for example, right upper outer quadrant. Tire explanation text can also include explanation about multiple regions of interest present in the first image.
  • the explanation text can include an explanation about an artifact, or a benign finding, not only a malignancy.
  • the system 100 may have a lookup table or other mapping that maps the probability output (e.g., numerical tokens) to one or more words or parts of words, which may be strung together to form the explanation text.
  • an array of numerical tokens that form the probability output may be converted in block 810 to generate the explanation text comprising a string of words with punctuation (e.g., forming one or more sentences or paragraphs) in a spoken language (e.g., English. French. German, Japanese, etc.).
  • the system 100 can perform image analysis using the first Al model to produce the malignancy score for the region of interest, which can be an accurate result (see block 804), and perform the explanation task using the second Al model, which is a separate model from the first Al model and a large language model (see block 806 and 808).
  • the second Al model can produce the explanation text, which is limited to the malignancy score or other concrete outputs (of the first Al model) that have been medically or scientifically validated, such as via FDA clearance of software that generates the malignancy score.
  • the explanation text produced from the second Al model is more accurate and fact-based than an existing large language model.
  • the user may manually select a region of interest (e.g., via a graphical user interface on one of the computing devices 150). Then, the system 100 can provide the user-selected region of interest to the first Al model or the malignancy Al model to obtain the malignancy score for the region of interest. The system 100 can provide the image and the region of interest and the malignancy score to second Al model to describe whether there is a suspicious finding within the region of interest.
  • the system 100 provides a report including the explanation text for the region of interest on the first image corresponding to an abnormality in the tissue for the patient.
  • the report can further include the malignancy score or indication of the region of interest in the first image, information of a malignant tumor, lesion, or abnormality in the region of interest, and/or any other suitable information.
  • the system 100 can display the report using the display 204 of the computing device 150 and/or the display 214 of the 152.
  • tire system 100 can transmit the report to the computing device 150, the server 152, or any other suitable device or system.
  • the report further includes tire first image (e.g., annotated with an outline of the region of interest).
  • the report 1 100 includes identifying information 1 102 (e.g., patient identifier, image identifier, etc.), a representation 1104 of an example of the first image provided to the second Al model (e.g., in block 806), an indication of a region of interest (ROI) 1106, and explanation text 1108.
  • identifying information 1 102 e.g., patient identifier, image identifier, etc.
  • representation 1104 of an example of the first image provided to the second Al model e.g., in block 806
  • Tire report 1100 is merely an example, and other reports generated by techniques described herein may take other fomis and include additional or less information, including different explanation text, different representative image(s), additional or fewer ROIs indicated, etc.
  • FIG. 9 shows an example of a flowchart describing an example method for abnormality analysis and explanation Al model training, according to aspects of the present disclosure.
  • the process 900 of FIG. 9 may be executed by the system 100 of FIG. 1, or more particularly by one or more processors thereof (e.g., tire processor 202 of tire computing device 150 and/or the processor 212 of the server 152).
  • the process 900 can be implemented as instructions on the memory 210 in the computing device 150.
  • the computing device 150 can further include the processor 202 in communication with the memory and configured to execute the instructions.
  • the process 900 can be implemented as instructions on the memory 220 in the server 152.
  • the server 152 can further include the processor 212 in communication with the memory 220 and configured to execute the instructions.
  • the process of FIG. 9 may be executed by the system 100 implementing the malignancy detection evaluation and explanation system as illustrated in FIG. 7.
  • the system 100 receives one or more training images of tissue.
  • block 902 is substantially similar to block 802 in FIG. 8.
  • the one or more training images of tissue can include three-dimensional tomosynthesis data of the tissue for the patient.
  • the system 100 receives multiple sets of training images of the tissue. Tirus, the system 100 can train Al model(s) using the multiple sets of training images of the tissue.
  • the system 100 receives a ground truth malignancy score for the region of interest in a first image.
  • the first image can be a synthetic image or an image of the one or more training images.
  • the system 100 can receive the ground truth malignancy score from the data source 102, the memory 210 of the computing device, the memory 220 of the server, or any other suitable source.
  • an expert can assess whether a malignant tumor or lesion in the region of interest in the first image exists. If a malignant tumor or lesion in the region of interest in the first image exists, the expert can label the ground truth malignancy score (e.g.. I or any other suitable indication) for the region of interest on the first image.
  • the expert can label the ground truth malignancy score (e.g., 0 or any other suitable indication) for the region of interest on the first image to indicate that the region of interest does not include a malignant tumor or lesion.
  • the ground truth malignancy score can include a category of risk (i.e., a low risk, medium risk, or high risk category).
  • the ground truth malignancy score can be selected from a range of values, such as the integers 1-5, with 1 indicating a lowest risk level and 5 indicating a highest risk level.
  • the ground truth malignancy score can further include a classification score for classify the type of the abnormality in the region of interest.
  • the ground truth classification score can identify that the type of the abnormality is a cancer, a brain hemorrhage, or any other suitable abnormality.
  • the system 100 trains a first Al model based on the first image and the ground truth malignancy score by adjusting first hyperparameters of the first Al model to produce a first output close or closer to tire ground truth malignancy score for the region of interest in the first image.
  • the system 100 can train the first Al model by means of a loss function (e.g., a regression loss function, a mean absolute error loss function, a mean bias error loss function, a hinge loss function, etc.).
  • a loss function e.g., a regression loss function, a mean absolute error loss function, a mean bias error loss function, a hinge loss function, etc.
  • the system 100 can determine whether the predicted output from the first Al model deviates more than a predetermined threshold from the ground truth malignancy score. Then, the system 100 can adjust hyperparameters of the first Al model to reduce the error in prediction.
  • the system 100 receives a ground truth explanation for a malignant tumor or lesion in the region of interest.
  • the system 100 can receive the ground truth explanation from the data source 102, tire memory' 210 of the computing device, the memory 220 of the server, or any other suitable source.
  • an expert can provide an explanation of a malignant tumor or lesion in the region of interest in the first image.
  • the explanation can be provided based on the region of interest and the ground truth malignancy score.
  • the expert can provide the explanation based on the location of the tumor or lesion, the shape of the tumor or lesion, the severity of the tumor or lesion, etc.
  • the explanation can be the ground truth explanation.
  • the system 100 can convert the explanation from the expert to the ground truth explanation, which include probability values corresponding to tokens, words, or part of words.
  • the system 100 trains a second Al model based on the first image and the ground truth explanation by adjusting second hyperparameters of the second Al model to produce a second output close or closer to the ground truth explanation.
  • the system 100 can train the second Al model by means of a loss function (e.g., a regression loss function, a mean absolute error loss function, a mean bias error loss function, a hinge loss function, etc.).
  • a loss function e.g., a regression loss function, a mean absolute error loss function, a mean bias error loss function, a hinge loss function, etc.
  • the system 100 can detennine whether the predicted output from the second Al model deviates more than a predetermined threshold from the ground truth explanation.
  • the system 100 can adjust hyperparameters of the second Al model to reduce the error in prediction.
  • the second Al model is a trained large language model based on various medical records or experts’ proven explanations. Then, the second Al model can be fine-tuned based on the ground truth malignant score for the
  • FIG. 10 shows an example of a flowchart describing an example method for determining a malignancy likelihood score for breast tissue of a patient, according to aspects of the present disclosure.
  • the process 1000 of FIG. 10 may be executed by the system 100 of FIG. 1, or more particularly by one or more processors thereof (e.g., the processor 202 of the computing device 150 and/or the processor 212 of tire server 152).
  • the process 1000 can be implemented as instructions on the memory 210 in the computing device 150.
  • the computing device 150 can further include the processor 202 in communication with tire memory and configured to execute the instructions.
  • the process 1000 can be implemented as instructions on the memory 220 in the server 152.
  • Tire server 152 can further include tire processor 212 in communication with the memory 220 and configured to execute the instructions.
  • the process 1000 of FIG. 10 may be executed by the system 100 implementing the malignancy detection evaluation and explanation system as illustrated in FIG. 7.
  • the system 100 provides each image of multiple images of the breast tissue to a first artificial intelligence (Al) model (c.g., ROI Al model 400) comprising a first trained neural network that is trained to identify regions of interest.
  • Al artificial intelligence
  • the system 100 receives a first identified region of interest from the first Al model corresponding to a first image of the plurality of images of the breast tissue.
  • the system 100 receives a second identified region of interest from the first Al model corresponding to a second image of the plurality of images of the breast tissue.
  • the system 100 generates a synthetic image by populating a pixel array of the synthetic image based on at least the first identified region of interest and the second identified region of interest.
  • the system 100 provides the synthetic image to a second Al model (e g., malignancy Al model 500) comprising a second trained neural network.
  • a second Al model e g., malignancy Al model 500
  • the system 100 detennines a malignancy likelihood score using the second Al model.
  • blocks 1002-1012 are substantially similar to block 804 in FIG. 8.
  • the system 100 provides at least one of the plurality of images of the breast tissue, the first region of interest, the second region of interest, the synthetic image, or tire malignancy score to a large language model (e.g., malignancy explanation model 710) to generate an explanation of at least one of the first region of interest, the second region of interest, the synthetic image, the malignancy score, or the analysis performed by the first trained neural network or the second trained neural network.
  • a large language model e.g., malignancy explanation model 710
  • block 1014 is substantially similar to blocks 806 and 808 in FIG. 8.
  • the system 100 displays a report including the malignancy likelihood score and the explanation.
  • block 1016 is substantially similar to block 810 in FIG. 8. Accordingly, in some examples, the report 1100 of FIG. 11 is displayed in block 1016. However, in other examples, a report having a different form than the report 1100 is displayed.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • General Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • Epidemiology (AREA)
  • Data Mining & Analysis (AREA)
  • Biomedical Technology (AREA)
  • Pathology (AREA)
  • Databases & Information Systems (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Medical Treatment And Welfare Office Work (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

La présente divulgation concerne des procédés et des systèmes d'analyse et d'explication d'anomalie. Les procédés et les systèmes consistent à : recevoir une pluralité d'images de tissu pour un patient, appliquer la pluralité d'images à un premier modèle d'intelligence artificielle (IA) afin d'obtenir un score pour une région d'intérêt sur une première image, appliquer la première image et le score pour la région d'intérêt sur la première image à un second modèle d'lA afin d'obtenir une sortie de probabilité, convertir la sortie de probabilité en un texte d'explication, et fournir un rapport comprenant le texte d'explication pour la région d'intérêt sur la première image pour le patient.
PCT/US2024/033449 2023-06-16 2024-06-11 Analyse entraînée par ia et explication d'images médicales Pending WO2024258873A2 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202363521620P 2023-06-16 2023-06-16
US63/521,620 2023-06-16

Publications (2)

Publication Number Publication Date
WO2024258873A2 true WO2024258873A2 (fr) 2024-12-19
WO2024258873A3 WO2024258873A3 (fr) 2025-04-24

Family

ID=93852852

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2024/033449 Pending WO2024258873A2 (fr) 2023-06-16 2024-06-11 Analyse entraînée par ia et explication d'images médicales

Country Status (1)

Country Link
WO (1) WO2024258873A2 (fr)

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200372636A1 (en) * 2017-11-22 2020-11-26 The Trustees Of Columbia University In The City Of New York System method and computer-accessible medium for determining breast cancer response using a convolutional neural network
EP4081952A4 (fr) * 2019-12-23 2024-01-24 Deephealth, Inc. Systèmes et procédés pour analyser des données d'images bidimensionnelles et tridimensionnelles
CA3225227A1 (fr) * 2021-07-09 2023-01-12 Jason Su Procedes et systemes de depistage radiologique rapide

Also Published As

Publication number Publication date
WO2024258873A3 (fr) 2025-04-24

Similar Documents

Publication Publication Date Title
US20260004918A1 (en) System and Method for Interpretation of Multiple Medial Images using Deep Learning
CN112529834B (zh) 病理图像模式在3d图像数据中的空间分布
AU2006254689B2 (en) System and method of computer-aided detection
US12367967B2 (en) System and method for automated annotation of radiology findings
US8687860B2 (en) Mammography statistical diagnostic profiler and prediction system
US11783476B2 (en) System and method for analyzing three-dimensional image data
US20110137132A1 (en) Mammography Information System
US20070280525A1 (en) Methods and Apparatus for Computer Automated Diagnosis of Mammogram Images
US20210217166A1 (en) Automated screening of medical data
US12308108B2 (en) Automatically detecting characteristics of a medical image series
US11282193B2 (en) Systems and methods for tumor characterization
CN111247592A (zh) 用于随时间量化组织的系统和方法
CN114820483A (zh) 图像检测方法、装置及计算机设备
Lu et al. Texture analysis based on Gabor filters improves the estimate of bone fracture risk from DXA images
US12555231B2 (en) Detecting ischemic stroke mimic using deep learning-based analysis of medical images
JP7222882B2 (ja) 医用画像評価のためのディープラーニングの応用
US20240170151A1 (en) Interface and deep learning model for lesion annotation, measurement, and phenotype-driven early diagnosis (ampd)
WO2024258873A2 (fr) Analyse entraînée par ia et explication d'images médicales
EP4705996A1 (fr) Examen supplémentaire entraîné par ia d'images médicales
Salh et al. Review of Techniques Used in Computer-Aided Detection and Diagnosis for Breast Cancer Using Mammograms
US20240105315A1 (en) Medical image diagnostic system, medical image diagnostic system evaluation method, and program
US20250252758A1 (en) Determining predicted outcomes of subjects with cancer based on segmentations of biomedical images
Lee et al. Learning radiologist’s step-by-step skill for cervical spinal injury examination: Line drawing, prevertebral soft tissue thickness measurement, and swelling detection
Talluri Interpretive Applications of Artificial Intelligence in Breast Radiology
WO2025175223A1 (fr) Systèmes et procédés basés sur l'apprentissage automatique pour classifier des masses dans des images médicales

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 2024824019

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2024824019

Country of ref document: EP

Effective date: 20260116

ENP Entry into the national phase

Ref document number: 2024824019

Country of ref document: EP

Effective date: 20260116

ENP Entry into the national phase

Ref document number: 2024824019

Country of ref document: EP

Effective date: 20260116

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24824019

Country of ref document: EP

Kind code of ref document: A2