WO2024141971A1 - Système informatique chirurgical avec support pour modèles d'apprentissage automatique interdépendants - Google Patents

Système informatique chirurgical avec support pour modèles d'apprentissage automatique interdépendants Download PDF

Info

Publication number
WO2024141971A1
WO2024141971A1 PCT/IB2023/063318 IB2023063318W WO2024141971A1 WO 2024141971 A1 WO2024141971 A1 WO 2024141971A1 IB 2023063318 W IB2023063318 W IB 2023063318W WO 2024141971 A1 WO2024141971 A1 WO 2024141971A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
surgical
surgical data
processing
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/IB2023/063318
Other languages
English (en)
Inventor
Iv Frederick E. Shelton
Aaron CHOW
Kevin M. Fiebig
Shane R. Adams
Jason L. Harris
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Cilag GmbH International
Original Assignee
Cilag GmbH International
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US18/092,019 external-priority patent/US20240221923A1/en
Priority claimed from US18/092,023 external-priority patent/US12573495B2/en
Priority claimed from US18/092,015 external-priority patent/US20240221892A1/en
Application filed by Cilag GmbH International filed Critical Cilag GmbH International
Priority to EP23841347.0A priority Critical patent/EP4490747A1/fr
Publication of WO2024141971A1 publication Critical patent/WO2024141971A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H70/00—ICT specially adapted for the handling or processing of medical references
    • G16H70/20—ICT specially adapted for the handling or processing of medical references relating to practices or guidelines
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
    • G16H10/60—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for patient-specific data, e.g. for electronic patient records
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/40—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/67—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for remote operation
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/25—User interfaces for surgical systems
    • A61B2034/256—User interfaces for surgical systems having a database of accessory information, e.g. including context sensitive help or scientific articles
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges

Definitions

  • a computer-implemented method for processing surgical data to determine one or more of surgical data classifications, surgical data trends, or surgical recommendations comprising: obtaining surgical data; determining a first set of surgical data from the obtained surgical data and a second set of surgical data from the obtained surgical data, wherein the first set of surgical data is determined based on a first processing task, wherein the second set of surgical data is determined based on a second processing task, and wherein the first processing task is different from the second processing task; generating, using a first machine learning model, a first output based on the first set of surgical data and the first processing task; generating, using a second machine learning model, a second output based on the second set of surgical data and the second processing task; and generating, using a third machine learning model, a third output based on at least one of the first output and the second output and a third processing task, wherein the first processing task, second processing task and third processing task contribute to determining one or more of surgical data classification
  • machine learning is able to be used to determine suigical data classifications, surgical data trends, or surgical recommendations from complex data, with reduced processing time and increased accuracy.
  • each machine learning model is able to be specialized in the particular processing task, which increases accuracy overall.
  • each machine learning model does not use all of the surgical data obtained for the processing task, but rather a set of the surgical data (e.g. the first set, the second set), which reduces the processing time when using the machine learning model.
  • the first set of surgical data is a first subset of the obtained surgical data and the second set of surgical data is a second subset of the obtained surgical data.
  • each of the subsets includes the minimum amount of data necessary for its respective machine learning model to perform its respective processing task.
  • the steps of generating the first output and generating the second output are performed in parallel (e.g. by different processors).
  • the surgical data may comprise surgical data from a facility storage, surgical data from an edge network storage, and surgical data from a cloud network storage.
  • the surgical data may be complex data, from multiple different sources.
  • the previously described problems of increased processing time required to train and use a machine learning model are not experienced as the determination of surgical data classifications, surgical data trends, or surgical recommendations is split into a plurality of distinct processing tasks and machine learning models.
  • the facility storage, edge network storage, and cloud network storage may have different privacy levels.
  • the facility storage may have a first privacy level
  • the edge network stage may have a second privacy level
  • the cloud network storage may have a third privacy level.
  • the method may further comprise: generating a data visualization based on the third output; and sending the data visualization to a display. poll] In this way, the determined surgical data classifications, surgical data trends, or surgical recommendations can be output to an operator for the operator to act upon.
  • the third machine learning model may be a data visualization machine learning model
  • the third processing task may be a data visualization task.
  • Data visualization performed by machine learning models may enable trend identification that may not be captured by human analysis, for example, based on the multidimensional optimization performed by the first and/ or second machine learning models.
  • known automatic data visualization techniques such as AutoViz, may be performed on the third output from the third machine learning model (i.e. not specifically a data visualization machine learning model, but a model used for another purpose).
  • the method may further comprise: generating a control algorithm based on the third output; and sending the control algorithm to a surgical instrument.
  • the third machine learning model may be a surgical data classifications, surgical data trends, or surgical recommendations machine learning model
  • the third processing task may be the determination of one or more surgical data classifications, suigical data trends, or surgical recommendations.
  • a control algorithm may be automatically generated and sent to the relevant control instrument.
  • generating a control algorithm based on the third output comprises adapting a control parameter of an existing control algorithm. This potentially allows for better surgical outcomes to be obtained.
  • the method may further comprise: determining a first processing capability associated with the first processing task, wherein the first set of surgical data is further determined based on the first processing capability; determining a second processing capability associated with the second processing task, wherein the second set of surgical data is further determined based on the second processing capability; and determining a third processing capability associated with the third processing task, wherein the first output is further generated based on the third processing capability, and wherein the second output is further generated based on the third processing capability.
  • a processing capability also known as a processing capacity, is the capacity of a particular processor of a particular device or system (e.g. the device or system coupled to each of the facility storage, edge network storage, and cloud network storage). Processing capability is particularly important when processing surgical data using machine learning models since this can be a very intensive processing task, requiring a high amount of processing capability.
  • the first processing task is performed by a first processor
  • the second processing task is performed by a second processor
  • the third processing task is performed by one of the first, second or a third processor.
  • the first processor and/or second processor may determine a processing capability associated with the third processor, and used this to generate the first output and/ or the second output. For instance, the higher the processing capability of the third processing, then the more data is included in the first output and/ or second output. This may also impact the amount of data included in the first set of data and/or the second set of data. poi9] At least one of the first processing task or the second processing task may be data preparation.
  • the first machine learning model or second machine learning model is able to put data in a suitable form for the third machine learning model to process (e.g. without taking up too much processing capability at the processor responsible for performing the third processing task).
  • both the first processing task and the second processing task may be data preparation.
  • the first processing task and second processing task may have to be processed at different processors due to, for example, processing capacity of the processors or privacy level of the data that the processor is permitted to process.
  • the data preparation performed by the first processing task is different to the second processing task in that it is applied to a different set of data.
  • the third processing task may be determining one or more of surgical data classifications, surgical data trends, or surgical recommendations.
  • the first processing task may be associated with a first privacy level, wherein the second processing task is associated with a second privacy level, and wherein the third processing task is associated with a third privacy level.
  • Different privacy levels may occur because of the nature of the surgical data that is required in the processing task. For example, anonymized surgical data may not require as high a privacy level compared to non-anonymized surgical data. Privacy level can impact where the processing task is processed. In some embodiments, different processing tasks may have to be processed at different processors due to the privacy level of the surgical data that the processor is permitted to process (e.g. due to the existence of a boundary for HIPAA (Health Insurance Portability and Accountability Act) or GDPR (General Data Protection Regulation) purposes in the data processing system).
  • HIPAA Health Insurance Portability and Accountability Act
  • GDPR General Data Protection Regulation
  • Personal data is any information which is related to an identified or identifiable natural person. A patient is identifiable if they can be directly or indirectly identified.
  • Health data is a special category of personal data which is subject to a higher level of protection (see Art. 9 GDPR or the HIPPA Privacy Rule), requiring heightened security considerations due to its cognitive content. Breaches of sensitive personal data can result in the accidental or unlawful destruction, loss, alternation, unauthorized disclosure of, or access to, sensitive data, which can have significant human consequences. For example, the permanent deletion of medical records of a person potentially has significant and long-lasting consequences for the health of said person.
  • the first privacy level and/ or the second privacy level may be higher than the third privacy level.
  • the first processing task and the second processing task may be allowed to process non-anonymized data, whilst the third processing task may only be permitted to process anonymized data.
  • the first machine learning model and/or second machine learning model may ingest non-anonymized data and output anonymized data for the third machine learning model to process.
  • the first set of surgical data may be further determined based on the first privacy level, wherein the second set of surgical is further determined based on the second privacy level, wherein the first output is further generated based on the third privacy level, and wherein the second output is further generated based on the third privacy level.
  • the first privacy level and/ or the second privacy level may be higher than the third privacy level.
  • the first processing task and the second processing task may be allowed to process non-anonymized data, whilst the third processing task may only be permitted to process anonymized data.
  • the first set of surgical data and the second set of surgical data are permitted to include non-anonymized data.
  • the first output and the second output provide anonymized data for the third machine learning model.
  • a ML model (e.g., trained by the machine learning algorithm of embodiment 1) may produce more accurate and/or reliable outputs for surgical data classifications, surgical data trends, or surgical recommendations because the revised first set of surgical data, i.e. the data set without anomalies (e.g., data set with consistent data, data set without missing data, etc.), is more accurate and/or reliable than the original first set of surgical data.
  • data used to train the ML model has a direct effect on the ML model itself, with more data generally leading to a more accurate ML model.
  • the first set of surgical data comprises data associated with a first type of surgical procedure
  • the master set of surgical data comprises data associated with either the first type of surgical procedure, or a plurality of different surgical procedures.
  • the data associated with the first type of surgical procedure may relate to one or more instances of the first type of surgical procedure.
  • the step of determining that at least a first portion of the first set of surgical data is anomalous may be performed without using machine learning.
  • the surgical computing system may determine that the first set of surgical data is anomalous. The surgical computing system may determine that there is anomalous data by comparing the data within the first set of surgical data (e.g.
  • the surgical computing system may compare the data types in the first set of surgical data with the data types in the master set of data, or another set of data, to identify data types that are missing.
  • the surgical computing system may compare any numerical values of the first set of data with the ranges of numerical values in the master set of surgical data, or another set of data, to identify numerical values that are outside of existing ranges. Additionally or alternatively, the surgical computing system may determine that the first set of surgical data is anomalous based on a system issue affecting the first set of surgical data.
  • the system may have flagged an issue with surgical instrument calibration, sensor failure, data recording, sampling rate, communication of the data between devices, wireless buffer sizes in communication (e.g., if someone has a high frequency sensor polling faster than Bluetooth low energy buffer can dump data to a processor, then bits may be lost, overwritten, and/or messed up), etc.
  • the step of determining that at least a first portion of the first set of surgical data is anomalous may be performed using machine learning.
  • the first set of surgical data is processed by a ML model.
  • the ML model may be trained to identify differences within the first set of surgical data, between the first set of surgical data and the historical set of surgical data, and/ or between the first set of surgical data and the master set of surgical data.
  • the ML model may be trained to identify system issues affecting the first set of surgical data. Additionally or alternatively, the ML model may be trained to identify issues affecting other ML models (e.g. a data reduction ML model or a preprocessing ML model).
  • the step of generating substitute surgical data may be performed using machine learning.
  • the substitute surgical data may be synthetic surgical data generated by a ML model.
  • the ML model may be based on a deep neural network, for example a variational autoencoder, generative adversarial network, or neural radiance field.
  • the step of generating a revised first set of surgical data comprising at least a second portion of the first set of surgical data and the substitute surgical data is performed by swapping the first portion of the first set of surgical data with the substitute set of surgical data.
  • the second portion of the first set of surgical data is the remainder of the first set of surgical data once the first portion of surgical data is removed.
  • At least a first portion of the first set of surgical data may be anomalous comprises determining that the at least a first portion of the first set of surgical data is incomplete or irregular.
  • the method may further comprise: inserting the revised first set of surgical data into the master set of surgical data.
  • further surgical data may be inserted into the master set of surgical data as further sets of surgical data (e.g., from further surgical procedures) is received by the surgical computing system.
  • this step is only performed once the revised first set of surgical data is verified.
  • the revised first set of surgical data may be used to further train a machine learning model already trained using the master set of surgical data, wherein the machine learning model is configured to make surgical data classifications, surgical data trends, or surgical recommendations .
  • the ML model can be updated based on the data in the revised first set of surgical data to improve the accuracy in its predictions.
  • the ML model may learn and constantly improve with each set of surgical data used to further train the ML model.
  • the method may further comprise: determining that data in the master set of surgical data needs to be revised based on the revised first set of surgical data; and generating a revised master set of surgical data comprising at least a portion of the revised first set of surgical data and at least a portion of the master set of surgical data.
  • the master set of surgical data may be revised as more input data (e.g., from further surgical procedures) is received by the surgical computing system.
  • this step is only performed once the revised first set of surgical data is verified.
  • the revised master set of surgical data may be used to train a machine learning model to make surgical data classifications, surgical data trends, or surgical recommendations.
  • the updating of the master data set may enable the ML model to constantly improve its accuracy in its predictions.
  • the ML model may learn and constantly improve with each surgical procedure dataset input to the ML model.
  • the ML model may insert the revised dataset into the master dataset (e.g., to be used for future processing). With each iteration of processing data, the master data set may be updated and/ or improved.
  • the method may further comprise: obtaining a verification set of data associated with the master set of surgical data; and verifying the revised first set of surgical data based on the verification set of data. [0049] In embodiments, the verification may be performed by an operator by comparing the revised first set of surgical data with the verification set of data.
  • the verification set of data is the verified surgical data associated with historic surgical procedures of the master set of surgical data.
  • the verification set of data may be all of the data of the master set of surgical data.
  • the verification set of data may be a portion of the master set of surgical data. For example, a portion of the master set of surgical data which has been verified to a greater extent than the rest of the data (e.g. verified by an operator).
  • the substitute surgical data may be further generated based on the determined first data type.
  • Determining that the at least a first portion of the first set of surgical data is anomalous may comprise determining that the at least a first portion of the first set of surgical data is incomplete or irregular, and wherein determining that the at least a first portion of the second set of surgical data is anomalous comprises determining that the at least a first portion of the second set of surgical data is incomplete or irregular.
  • the method may further comprise: inserting the third set of surgical data into the master set of surgical data.
  • the third set of surgical data may be used to further train a machine learning model already trained using the master set of surgical data, wherein the machine learning model is configured to make surgical data classifications, surgical data trends, or surgical recommendations.
  • the revised master list of surgical data may be used to train a machine learning model to make surgical data classifications, surgical data trends, or surgical recommendations.
  • a computer- implemented method for preparing data to be sent to a first location having a first privacy classification and a second location having a second privacy classification comprising: obtaining surgical data comprising a plurality of subsets of surgical data; determining a respective privacy classification for each subset of the subsets of surgical data; determining a first processing goal and a second processing goal, wherein the first processing goal is associated with a first processing task and a first data needs, and wherein the second processing goal is associated with a second processing task and a second data needs; determining a first privacy classification threshold associated with the first processing task and a second privacy classification threshold associated with the second processing task; determining a first data package based on the first processing goal, the first data needs, and the first privacy classification threshold, wherein the first data package comprises at least a first portion of the surgical data; determining a second data package based on the second processing goal, the second data needs, and the second privacy classification threshold, wherein the
  • the surgical data may be obtained from medical records. In other embodiments, the surgical data may be obtained from one or more surgical devices.
  • the obtained surgical data may include data associated with a surgical procedure, data associated with a specific patient, data associated with similar patients, and/or the like.
  • the first processing task may relate to data reduction.
  • the first processing task may relate to training a first machine learning model to perform data reduction.
  • the first processing goal may relate to the extent of data reduction or the accuracy of data reduction.
  • the second processing task may relate to determining one or more of surgical data classifications, surgical data trends, or surgical recommendations.
  • the second processing task may relate to training a second machine learning model to determining one or more of surgical data classifications, surgical data trends, or surgical recommendations.
  • Determining the first data package may comprise (or further comprises) determining whether a respective subset of surgical data, either alone or in combination with other subsets of surgical data, meets the first data needs; and wherein determining the second data package may comprise determining whether a respective subset of surgical data, either alone or in combination with other subsets of surgical data, meets the second data needs.
  • the first data package may comprise subsets of the plurality of subsets of surgical data determined to meet the first data needs, and wherein the second data package comprises subsets of the plurality of subsets of surgical data determined to meet the second data needs.
  • the first data package may comprise subsets of the plurality of subsets of surgical data determined to meet the first data needs and to be below the first privacy classification threshold
  • the second data package comprises subsets of the plurality of subsets of surgical data determined to meet the second data needs and to be below the second privacy classification threshold.
  • Determining the first data package may comprise (or further comprise) determining whether a respective subset of surgical data contributes towards achieving the first processing goal; and wherein determining the second data package comprises determining whether a respective subset of surgical data contributes towards achieving the second processing goal.
  • the first data package may comprise one or more of the plurality of subsets of surgical data determined to contribute towards achieving the first processing goal, and wherein the second data package comprises one or more of the plurality of subsets of surgical data determined to contribute towards achieving the second processing goal.
  • the first data package comprises subsets of the plurality of subsets of surgical data determined to contribute towards achieving the first processing goal and to be below the first privacy classification threshold
  • the second data package comprises subsets of the plurality of subsets of surgical data determined to contribute towards achieving the second processing goal and to be below the second privacy classification threshold.
  • Determining the first privacy classification threshold may be based on the first processing goal; and wherein determining the second privacy classification threshold is based on the second processing goal.
  • the method may further comprise: determining that the first processing goal has updated; determining an updated first privacy classification threshold based on the updated first processing goal; determining a third data package based on the updated first processing goal and the updated first privacy classification threshold; determining an updated second processing goal based on the determination that the first processing goal has updated; determining an updated second privacy classification threshold based on the updated second processing goal; and determining a fourth data package based on the updated second processing goal and the updated second privacy classification threshold.
  • the computing system may determine that the first set of surgical data is problematic (e.g., incomplete, erroneous, irregular, etc.)
  • the computing system may determine the first set of surgical data is problematic, for example, based on comparison to the master set of data.
  • the computing system may generate substitute data.
  • the substitute data may be generated based on the master set of data and the first set of data.
  • the substitute data may be generated based on a data type that is problematic in the first set of data.
  • the computing system may generate a second dataset (e.g., revised first set of surgical data), for example, that includes the substitute data and a portion of the first set of data (e.g., the non-problematic portion of the first set of data).
  • a data package may refrain from including data that is below (e.g., or above) the classification threshold.
  • the classification threshold may be associated with a privacy level. For example, privacy may be balanced with processing task importance to determine data exchange and data packages. For example, private data may be refrained from being sent to a processing system associated with a minimally important processing task. However, private data may be sent to a processing system associated with an important processing task.
  • FIG. 1 is a block diagram of a computer-implemented surgical system.
  • FIG. 1 is a block diagram of a computer-implemented surgical system 100.
  • An example surgical system such as the surgical system 100, may include one or more surgical systems (e.g., surgical sub-systems) 102, 103, 104.
  • surgical system 102 may include a computer-implemented interactive surgical system.
  • surgical system 102, 103, 104 may include a surgical computing system, such as surgical hub 106 and/or computing device 116, in communication with a cloud computing system 108.
  • the cloud computing system 108 may include a cloud server 109 and a cloud storage unit 110.
  • Surgical systems 102, 103, 104 may each computer-enabled surgical equipment and devices.
  • the robotic system 113 may enable robotic surgical procedures.
  • the robotic system 113 may receive information, settings, programming, controls and the like from the surgical hub 106 for example, the robotic system 113 may send data, such as sensor data, feedback information, video information, operational logs, and the like to the surgical hub 106.
  • the environmental sensing system 115 may include one or more devices, for example, used for measuring one or more environmental attributes, for example, as further described in FIG. 2.
  • the robotic system 113 may include a plurality of devices used for performing a surgical procedure, for example, as further described in FIG. 2.
  • the surgical system 102 may be in communication with a remote server 109 that may be part of a cloud computing system 108.
  • the one or more sensing systems 111, 115 may measure the biomarkers using one or more sensors, for example, photosensors (e.g., photodiodes, photoresistors), mechanical sensors (e.g., motion sensors), acoustic sensors, electrical sensors, electrochemical sensors, thermoelectric sensors, infrared sensors, etc.
  • the one or more sensors may measure the biomarkers as described herein using one of more of the following sensing technologies: photoplethysmography, electrocardiography, electroencephalography, colorimetry, impedimentary, potentiometry, amperometry, etc.
  • the biomarkers measured by the one or more sensing systems 111, 115 may include, but are not limited to, sleep, core body temperature, maximal oxygen consumption, physical activity, alcohol consumption, respiration rate, oxygen saturation, blood pressure, blood sugar, heart rate variability, blood potential of hydrogen, hydration state, heart rate, skin conductance, peripheral temperature, tissue perfusion pressure, coughing and sneezing, gastrointestinal motility, gastrointestinal tract imaging, respiratory tract bacteria, edema, mental aspects, sweat, circulating tumor cells, autonomic tone, circadian rhythm, and/ or menstrual cycle.
  • the biomarkers may relate to physiologic systems, which may include, but are not limited to, behavior and psychology, cardiovascular system, renal system, skin system, nervous system, gastrointestinal system, respiratory system, endocrine system, immune system, tumor, musculoskeletal system, and/ or reproductive system.
  • Information from the biomarkers may be determined and/or used by the computer-implemented patient and the surgical system 100, for example.
  • the information from the biomarkers may be determined and/or used by the computer-implemented patient and the surgical system 100 to improve said systems and/or to improve patient outcomes, for example.
  • the one or more sensing systems 111, 115, biomarkers, and physiological systems are described in more detail in U.S. App No.
  • FIG. 2 shows an example of a surgical system 202 in a surgical operating room. As illustrated in FIG. 2, a patient is being operated on by one or more health care professionals (HCPs). The HCPs are being monitored by one or more HCP sensing systems 220 worn by the HCPs.
  • HCPs health care professionals
  • the HCPs and the environment surrounding the HCPs may also be monitored by one or more environmental sensing systems including, for example, a set of cameras 221, a set of microphones 222, and other sensors that may be deployed in the operating room.
  • the HCP sensing systems 220 and the environmental sensing systems may be in communication with a surgical hub 206, which in turn may be in communication with one or more cloud servers 209 of the cloud computing system 208, as shown in FIG. 1.
  • the environmental sensing systems may be used for measuring one or more environmental attributes, for example, HCP position in the surgical theater, HCP movements, ambient noise in the surgical theater, temperature/humidity in the surgical theater, etc. As illustrated in FIG.
  • a primary display 223 and one or more audio output devices are positioned in the sterile field to be visible to an operator at the operating table 224.
  • a visualization/notification tower 226 is positioned outside the sterile field.
  • the visualization/notification tower 226 may include a first non-sterile human interactive device (HID) 227 and a second non-sterile HID 229, which may face away from each other.
  • the HID may be a display or a display with a touchscreen allowing a human to interface directly with the HID.
  • a human interface system guided by the surgical hub 206, may be configured to utilize the HIDs 227, 229, and 223 to coordinate information flow to operators inside and outside the sterile field.
  • the surgical hub 206 may cause an HID (e.g., the primary HID 223) to display a notification and/ or information about the patient and/ or a surgical procedure step.
  • the surgical hub 206 may prompt for and/ or receive input from personnel in the sterile field or in the non-sterile area.
  • the surgical hub 206 may cause an HID to display a snapshot of a surgical site, as recorded by an imaging device 230, on a non-sterile HID 227 or 229, while maintaining a live feed of the surgical site on the primary HID 223.
  • the snapshot on the non-sterile display 227 or 229 can permit a non- sterile operator to perform a diagnostic step relevant to the surgical procedure, for example.
  • the surgical hub 206 may be configured to route a diagnostic input or feedback entered by a non-sterile operator at the visualization tower 226 to the primary display 223 within the sterile field, where it can be viewed by a sterile operator at the operating table.
  • the input can be in the form of a modification to the snapshot displayed on the non-sterile display 227 or 229, which can be routed to the primary display 223 by the surgical hub 206.
  • a surgical instrument 231 is being used in the surgical procedure as part of the surgical system 202.
  • the hub 206 may be configured to coordinate information flow to a display of the surgical instrument 231.
  • U.S. Patent Application Publication No. US 2019-0200844 Al U.S.
  • Patent Application No. 16/209,385) titled METHOD OF HUB COMMUNICATION, PROCESSING, STORAGE AND DISPEAY, filed December 4, 2018, the disclosure of which is herein incorporated by reference in its entirety.
  • a diagnostic input or feedback entered by a non-sterile operator at the visualization tower 226 can be routed by the hub 206 to the surgical instrument display within the sterile field, where it can be viewed by the operator of the surgical instrument 231.
  • Example surgical instruments that are suitable for use with the surgical system 202 are described under the heading “Surgical Instrument Hardware” and in U.S. Patent Application Publication No. US 2019-0200844 Al (U.S. Patent Application No.
  • FIG. 2 illustrates an example of a surgical system 202 being used to perform a surgical procedure on a patient who is lying down on an operating table 224 in a surgical operating room 235.
  • a robotic system 234 may be used in the surgical procedure as a part of the surgical system 202.
  • the robotic system 234 may include a surgeon’s console 236, a patient side cart
  • the patient side cart 232 can manipulate at least one removably coupled surgical tool 237 through a minimally invasive incision in the body of the patient while the surgeon views the surgical site through the surgeon’s console 236.
  • An image of the surgical site can be obtained by a medical imaging device 230, which can be manipulated by the patient side cart 232 to orient the imaging device 230.
  • the robotic hub 232 can manipulate at least one removably coupled surgical tool 237 through a minimally invasive incision in the body of the patient while the surgeon views the surgical site through the surgeon’s console 236.
  • An image of the surgical site can be obtained by a medical imaging device 230, which can be manipulated by the patient side cart 232 to orient the imaging device 230.
  • the imaging device 230 may include at least one image sensor and one or more optical components. Suitable image sensors may include, but are not limited to, Charge- Coupled Device (CCD) sensors and Complementary MetaLOxide Semiconductor (CMOS) sensors.
  • CCD Charge- Coupled Device
  • CMOS Complementary MetaLOxide Semiconductor
  • imaging devices suitable for use with the present disclosure include, but are not limited to, an arthroscope, angioscope, bronchoscope, choledochoscope, colonoscope, cytoscope, duodenoscope, enteroscope, esophagogastro-duodenoscope (gastroscope), endoscope, laryngoscope, nasopharyngo-neproscope, sigmoidoscope, thoracoscope, and ureteroscope.
  • an arthroscope angioscope
  • bronchoscope choledochoscope
  • colonoscope colonoscope
  • cytoscope duodenoscope
  • enteroscope esophagogastro-duodenoscope (gastroscope)
  • endoscope laryngoscope
  • nasopharyngo-neproscope sigmoidoscope
  • thoracoscope ureteroscope
  • the imaging device may employ multi-spectrum monitoring to discriminate topography and underlying structures.
  • a multi-spectral image is one that captures image data within specific wavelength ranges across the electromagnetic spectrum. The wavelengths may be separated by filters or by the use of instruments that are sensitive to particular wavelengths, including light from frequencies beyond the visible light range, e.g., IR and ultraviolet. Spectral imaging can allow extraction of additional information that the human eye fails to capture with its receptors for red, green, and blue.
  • the use of multi-spectral imaging is described in greater detail under the heading “Advanced Imaging Acquisition Module” in U.S. Patent Application Publication No. US 2019-0200844 Al (U.S. Patent Application No.
  • Multispectrum monitoring can be a useful tool in relocating a surgical field after a surgical task is completed to perform one or more of the previously described tests on the treated tissue. It is axiomatic that strict sterilization of the operating room and surgical equipment is required during any surgery. The strict hygiene and sterilization conditions required in a “surgical theater,” i.e., an operating or treatment room, necessitate the highest possible sterility of all medical devices and equipment.
  • the sterile field may be considered a specified area, such as within a tray or on a sterile towel, that is considered free of microorganisms, or the sterile field may be considered an area, immediately around a patient, who has been prepared for a surgical procedure.
  • the sterile field may include the scrubbed team members, who are properly attired, and all furniture and fixtures in the area.
  • Wearable sensing system 211 illustrated in FIG. 1 may include one or more sensing systems, for example, HCP sensing systems 220 as shown in FIG. 2.
  • the HCP sensing systems 220 may include sensing systems to monitor and detect a set of physical states and/or a set of physiological states of a healthcare personnel (HCP).
  • HCP may be a surgeon or one or more healthcare personnel assisting the surgeon or other healthcare service providers in general.
  • a sensing system 220 may measure a set of biomarkers to monitor the heart rate of an HCP.
  • a sensing system 220 worn on a surgeon’s wrist may use an accelerometer to detect hand motion and/ or shakes and determine the magnitude and frequency of tremors.
  • the sensing system 220 may send the measurement data associated with the set of biomarkers and the data associated with a physical state of the surgeon to the surgical hub 206 for further processing.
  • One or more environmental sensing devices may send environmental information to the surgical hub 206.
  • the environmental sensing devices may include a camera 221 for detecting hand/body position of an HCP.
  • the environmental sensing devices may include microphones 222 for measuring the ambient noise in the surgical theater.
  • Other environmental sensing devices may include devices, for example, a thermometer to measure temperature and a hygrometer to measure humidity of the surroundings in the surgical theater, etc.
  • the surgical hub 206 alone or in communication with the cloud computing system, may use the surgeon biomarker measurement data and/ or environmental sensing information to modify the control algorithms of hand-held instruments or the averaging delay of a robotic interface, for example, to minimize tremors.
  • the HCP sensing systems 220 may measure one or more surgeon biomarkers associated with an HCP and send the measurement data associated with the surgeon biomarkers to the surgical hub 206.
  • the HCP sensing systems 220 may use one or more of the following RF protocols for communicating with the surgical hub 20006: Bluetooth, Bluetooth Tow-Energy (BEE), Bluetooth Smart, Zigbee, Z-wave, IPv6 Low-power wireless Personal Area Network (6L0WP N), Wi-Fi.
  • the surgeon biomarkers may include one or more of the following: stress, heart rate, etc.
  • the environmental measurements from the surgical theater may include ambient noise level associated with the surgeon or the patient, surgeon and/ or staff movements, surgeon and/ or staff attention level, etc.
  • the surgical hub 206 may use the surgeon biomarker measurement data associated with an HCP to adaptively control one or more surgical instruments 231.
  • the surgical hub 206 may send a control program to a surgical instrument 231 to control its actuators to limit or compensate for fatigue and use of fine motor skills.
  • the surgical hub 206 may send the control program based on situational awareness and/or the context on importance or criticality of a task.
  • the control program may instruct the instrument to alter operation to provide more control when control is needed.
  • FIG. 3 shows an example surgical system 302 with a surgical hub 306.
  • the surgical hub 306 may be paired with, via a modular control, a wearable sensing system 311, an environmental sensing system 315, a human interface system 312, a robotic system 313, and an intelligent instrument 314.
  • the hub 306 includes a display 348, an imaging module 349, a generator module 350, a communication module 356, a processor module 357, a storage array 358, and an operating-room mapping module 359.
  • the hub 306 further includes a smoke evacuation module 354 and/or a suction/irrigation module 355.
  • the various modules and systems may be connected to the modular control either directly via a router or via the communication module 356.
  • the operating theater devices may be coupled to cloud computing resources and data storage via the modular control.
  • the human interface system 312 may include a display sub-system and a notification sub-system.
  • the modular control may be coupled to non-contact sensor module.
  • the non-contact sensor module may measure the dimensions of the operating theater and generate a map of the surgical theater using, ultrasonic, laser-type, and/or the like, non-contact measurement devices. Other distance sensors can be employed to determine the bounds of an operating room.
  • An ultrasound-based non-contact sensor module may scan the operating theater by transmitting a burst of ultrasound and receiving the echo when it bounces off the perimeter walls of an operating theater as described under the heading “Surgical Hub Spatial Awareness Within an Operating Room” in U.S. Provisional Patent Application Serial No. 62/611,341, titled INTERACTIVE SURGICAE PLATFORM, filed December 28, 2017, which is herein incorporated by reference in its entirety.
  • the sensor module may be configured to determine the size of the operating theater and to adjust Bluetooth-pairing distance limits.
  • a laser-based non-contact sensor module may scan the operating theater by transmitting laser light pulses, receiving laser light pulses that bounce off the perimeter walls of the operating theater, and comparing the phase of the transmitted pulse to the received pulse to determine the size of the operating theater and to adjust Bluetooth pairing distance limits, for example.
  • energy application to tissue, for sealing and/ or cutting is generally associated with smoke evacuation, suction of excess fluid, and/ or irrigation of the tissue. Fluid, power, and/or data lines from different sources are often entangled during the surgical procedure. Valuable time can be lost addressing this issue during a surgical procedure. Detangling the lines may necessitate disconnecting the lines from their respective modules, which may require resetting the modules.
  • the combo generator module also includes a smoke evacuation component, at least one energy delivery cable for connecting the combo generator module to a surgical instrument, at least one smoke evacuation component configured to evacuate smoke, fluid, and/or particulates generated by the application of therapeutic energy to the tissue, and a fluid line extending from the remote surgical site to the smoke evacuation component.
  • the fluid line may be a first fluid line, and a second fluid line may extend from the remote surgical site to a suction and irrigation module 355 slidably received in the hub enclosure 360.
  • the hub enclosure 360 may include a fluid interface. Certain surgical procedures may require the application of more than one energy type to the tissue. One energy type may be more beneficial for cutting the tissue, while another different energy type may be more beneficial for sealing the tissue.
  • a bipolar generator can be used to seal the tissue while an ultrasonic generator can be used to cut the sealed tissue.
  • a hub modular enclosure 360 is configured to accommodate different generators and facilitate an interactive communication therebetween.
  • the hub modular enclosure 360 may enable the quick removal and/or replacement of various modules.
  • aspects of the present disclosure present a modular surgical enclosure for use in a surgical procedure that involves energy application to tissue.
  • the modular surgical enclosure includes a first energy-generator module, configured to generate a first energy for application to the tissue, and a first docking station comprising a first docking port that includes first data and power contacts, wherein the first energy-generator module is slidably movable into an electrical engagement with the power and data contacts and wherein the first energy-generator module is slidably movable out of the electrical engagement with the first power and data contacts.
  • the modular surgical enclosure also includes a second energy -generator module configured to generate a second energy, different than the first energy, for application to the tissue, and a second docking station comprising a second docking port that includes second data and power contacts, wherein the second energy generator module is slidably movable into an electrical engagement with the power and data contacts, and wherein the second energy-generator module is slidably movable out of the electrical engagement with the second power and data contacts.
  • the modular surgical enclosure also includes a communication bus between the first docking port and the second docking port, configured to facilitate communication between the first energygenerator module and the second energy-generator module. Referring to FIG.
  • a hub modular enclosure 360 that allows the modular integration of a generator module 350, a smoke evacuation module 354, and a suction/irrigation module 355.
  • the hub modular enclosure 360 further facilitates interactive communication between the modules 359, 354, and 355.
  • the generator module 350 can be with integrated monopolar, bipolar, and ultrasonic components supported in a single housing unit slidably insertable into the hub modular enclosure 360.
  • the generator module 350 can be configured to connect to a monopolar device 351, a bipolar device 352, and an ultrasonic device 353.
  • the generator module 350 may comprise a series of monopolar, bipolar, and/or ultrasonic generator modules that interact through the hub modular enclosure 360.
  • the hub modular enclosure 360 can be configured to facilitate the insertion of multiple generators and interactive communication between the generators docked into the hub modular enclosure 360 so that the generators would act as a single generator.
  • the modular communication hub 465 and the devices may be connected in a room in a healthcare facility specially equipped for surgical operations.
  • the modular communication hub 465 may include a network hub 461 and/or a network switch 462 in communication with a network router 466.
  • the modular communication hub 465 may be coupled to a local computer system 463 to provide local computer processing and data manipulation.
  • the computer system 463 may comprise a processor and a network interface.
  • the processor may be coupled to a communication module, storage, memory, non-volatile memory, and input/ output (I/O) interface via a system bus.
  • the system bus can be any of several types of bus structure(s) including the memory bus or memory controller, a peripheral bus or external bus, and/or a local bus using any variety of available bus architectures including, but not limited to, 9-bit bus, Industrial Standard Architecture (ISA), Micro-Charmel Architecture (MSA), Extended ISA (EISA), Intelligent Drive Electronics (IDE), VESA Eocal Bus (VLB), Peripheral Component Interconnect (PCI), USB, Advanced Graphics Port (AGP), Personal Computer Memory Card International Association bus (PCMCIA), Small Computer Systems Interface (SCSI), or any other proprietary bus.
  • ISA Industrial Standard Architecture
  • MSA Micro-Charmel Architecture
  • EISA Extended ISA
  • IDE Intelligent Drive Electronics
  • VLB VESA Eocal Bus
  • PCI Peripheral
  • the processor may be any single-core or multicore processor such as those known under the trade name ARM Cortex by Texas Instruments.
  • the processor may be an LM4F230H5QR ARM Cortex-M4F Processor Core, available from Texas Instruments, for example, comprising an on-chip memory of 256 KB single -cycle flash memory, or other nonvolatile memory, up to 40 MHz, a prefetch buffer to improve performance above 40 MHz, a 32 KB single-cycle serial random access memory (SRAM), an internal read-only memory (ROM) loaded with StellarisWare® software, a 2 KB electrically erasable programmable readonly memory (EEPROM), and/or one or more pulse width modulation (PWM) modules, one or more quadrature encoder inputs (QEI) analogs, one or more 12-bit analog-to-digital converters (ADCs) with 12 analog input channels, details of which are available for the product datasheet.
  • QEI quadrature encoder inputs
  • the processor may comprise a safety controller comprising two controller-based families such as TMS570 and RM4x, known under the trade name Hercules ARM Cortex R4, also by Texas Instruments.
  • the safety controller may be configured specifically for IEC 61508 and ISO 26262 safety critical applications, among others, to provide advanced integrated safety features while delivering scalable performance, connectivity, and memory options.
  • the computer system 463 may include software that acts as an intermediary between users and the basic computer resources described in a suitable operating environment. Such software may include an operating system.
  • the operating system which can be stored on the disk storage, may act to control and allocate resources of the computer system.
  • Modular devices la-ln located in the operating theater may be coupled to the modular communication hub 465.
  • the network hub 461 and/or the network switch 462 may be coupled to a network router 466 to connect the devices la-ln to the cloud computing system 464 or the local computer system 463.
  • Data associated with the devices la-ln may be transferred to cloud-based computers via the router for remote data processing and manipulation.
  • Data associated with the devices la-ln may also be transferred to the local computer system 463 for local data processing and manipulation.
  • Modular devices 2a-2m located in the same operating theater also may be coupled to a network switch 462.
  • the network switch 462 may be coupled to the network hub 461 and/ or the network router 466 to connect the devices 2a-2m to the cloud 464.
  • a tracking system 528 may be configured to determine the position of the longitudinally movable displacement member.
  • the position information may be provided to the processor 522, which can be programmed or configured to determine the position of the longitudinally movable drive member as well as the position of a firing member, firing bar, and I-beam knife element. Additional motors may be provided at the tool driver interface to control I-beam firing, closure tube travel, shaft rotation, and articulation.
  • a display 524 may display a variety of operating conditions of the instruments and may include touch screen functionality for data input. Information displayed on the display 524 may be overlaid with images acquired via endoscopic imaging modules.
  • the microcontroller 521 may be any single-core or multicore processor such as those known under the trade name ARM Cortex by Texas Instruments.
  • the motor driver 529 may comprise an H-bridge driver comprising field-effect transistors (FETs), for example.
  • the motor 530 can be powered by a power assembly releasably mounted to the handle assembly or tool housing for supplying control power to the surgical instrument or tool.
  • the power assembly may comprise a battery which may include a number of battery cells connected in series that can be used as the power source to power the surgical instrument or tool.
  • the battery cells of the power assembly may be replaceable and/or rechargeable.
  • the battery cells can be lithium-ion batteries which can be couplable to and separable from the power assembly. poi78]
  • the motor driver 529 may be an A3941 available from Allegro Microsystems, Inc.
  • Procedure data and/or patient record data may be associated with a related healthcare data system 716 in communication with the surgical computing device 704.
  • the procedure data may include information related to the instruments and/or replaceable instrument components to be employed in a given procedure, such as a master list for example.
  • the surgical computing device 704 may record (e.g., capture barcode scans) of the instruments and/or replaceable instrument components being put to use in the procedure. Such surgical information may be used to algorithmically confirm that appropriate configurations of surgical instruments and/ or replaceable components are being used. See U.S. Patent Application Publication No. US 2020-0405296 Al (U.S. Patent Application No.
  • Intelligent surgical instruments may sense and measure certain operational parameters in the course of their operation.
  • intelligent surgical instruments such as surgical robots, digital laparoscopic devices, and the like, may use such measurements to improve operation, for example to limit over compression, to reduce collateral damage, to minimize tissue tension, to optimize usage location, and the like.
  • See U.S. Patent Application Publication No. US 2018- 0049822 Al U.S. Patent Application No. 15/237,753
  • CONTROE OF ADVANCEMENT RATE AND APPLICATION FORCE BASED ON MEASURED FORCES filed August 16, 2016, the contents of which is hereby incorporated by reference herein in its entirety.
  • Such surgical information may be communicated to the surgical computing device 704.
  • the surgical computing device 704 can be configured to derive the contextual information pertaining to the surgical procedure from the data based upon, for example, the particular combination(s) of received data or the particular order in which the data is received from the data sources 726.
  • the contextual information inferred from the received data can include, for example, the type of surgical procedure being performed, the particular step of the surgical procedure that the surgeon is performing, the type of tissue being operated on, or the body cavity that is the subject of the procedure.
  • Surgical information may flow between the surgical computing device 704 and one or more edge computing devices 714. Aspects of the information flows, including, for example, information flow endpoints, information storage, data interpretation, and the like, may be managed relative to the surgical system 700 (e.g., relative to the healthcare facility) See U.S. Patent Application Publication No. US 2019-0206564 Al (U.S. Patent Application No. 16/209,490), titled METHOD FOR FACILITY DATA COLLECTION AND INTERPRETATION, filed December 4, 2018, the contents of which is hereby incorporated by reference herein in its entirety. Surgical information, as presented in its one or more information flows, may be used in connection with one or more artificial intelligence (Al) systems to further enhance the operation of the surgical system 700.
  • Al artificial intelligence
  • FIG. 7B shows an example computer-implement surgical system 730 with a plurality of information flows 732.
  • a surgical computing device 704 may communication with and/or incorporate one or more surgical data sources.
  • an imaging module 733 (and endoscope) may exchange surgical information with the surgical computing device 704.
  • Such information may include information from the imaging module 733 (and endoscope), such as video information, current settings, system status information, and the like.
  • the imaging module 733 may receive information from the surgical computing device 704, such as control information, configuration information, operational updates (such as software /firmware), and the like.
  • a generator module 734 may exchange surgical information with the surgical computing device 704.
  • information may include information from the generator module 734 (and corresponding energy device), such as electrical information (e.g., current, voltage, impedance, frequency, wattage), activity state information, sensor information such as temperature, current settings, system events, active time duration, and activation timestamp, and the like.
  • the generator module 734 may receive information from the surgical computing device 704, such as control information, configuration information, changes to the nature of the visible and audible notifications to the healthcare professional (e.g., changing the pitch, duration, and melody of audible tones), electrical application profiles and/or application logic that may instruct the generator module to provide energy with a defined characteristic curve over the application time, operational updates (such as software /firmware), and the like.
  • a smoke evacuator 735 may exchange surgical information with the surgical computing device 704.
  • Such information may include information from the smoke evacuator 735, such as operational information (e.g., revolutions per minute), activity state information, sensor information such as air temperature, current settings, system events, active time duration, and activation timestamp, and the like.
  • a communication module 739, a processor module 737, and/or a storage array 738 may exchange surgical information with the surgical computing device 704.
  • the communication module 739, the processor module 737, and/or the storage array 738 may constitute all or part of the computing platform upon which the surgical computing device 704 runs.
  • the communication module 739, the processor module 737, and/or the storage array 738 may provide local computing resources to other devices in the surgical system 730.
  • Information from the communication module 739, the processor module 737, and/or the storage array 738 to the surgical computing device 704 may include logical computing-related reports, such as processing load, processing capacity, process identification, CPU %, CPU time, threads, GPU%, GPU time, memory utilization, memory thread, memory ports, energy usage, bandwidth related information, packets in, packets out, data rate, channel utilization, buffer status, packet loss information, system events, other state information, and the like.
  • the communication module 739, the processor module 737, and/or the storage array 738 may receive information from the surgical computing device 704, such as control information, configuration information, operational updates (such as software/ firmware), and the like.
  • the communication module 739, the processor module 737, and/or the storage array 738 may also receive information from the surgical computing device 704 generated by another element or device of the surgical system 730.
  • data source information may be sent to and stored in the storage array.
  • data source information may be processed by the processor module 737.
  • an intelligent instrument 740 (with or without a corresponding display) may exchange surgical information with the surgical computing device 704.
  • Such information may include information from the intelligent instrument 740 relative to the instrument’s operation, such as device electrical and/or mechanical information (e.g., current, voltage, impedance, frequency, wattage, torque, force, pressure, etc.), load state information (e.g., information regarding the identity, type, and/ or status of reusables, such as staple cartridges), internal sensor information such as clamping force, tissue compression pressure and/or time, system events, active time duration, and activation timestamp, and the like.
  • device electrical and/or mechanical information e.g., current, voltage, impedance, frequency, wattage, torque, force, pressure, etc.
  • load state information e.g., information regarding the identity, type, and/ or status of reusables, such as staple cartridges
  • internal sensor information such as clamping force, tissue compression pressure and/or time, system events, active time duration, and activation timestamp, and the like.
  • the intelligent instrument 740 may receive information from the surgical computing device 704, such as control information, configuration information, changes to the nature of the visible and audible notifications to the healthcare professional (e.g., changing the pitch, duration, and melody of audible tones), mechanical application profiles and/or application logic that may instruct a mechanical component of the instrument to operate with a defined characteristic (e.g., blade/anvil advance speed, mechanical advantage, firing time, etc.), operational updates (such as software /firmware), and the like.
  • control and configuration information may be used to modify operational parameters, such as motor velocity for example.
  • the sensor module 741 may receive information from the surgical computing device 704, such as control information, configuration information, changes to the nature of observation (e.g., frequency, resolution, observational type etc.), triggers that define specific events for observation, on control, off control, data buffering, data preprocessing algorithms, operational updates (such as software /firmware), and the like.
  • a visualization system 742 may exchange surgical information with the surgical computing device 704.
  • Such information may include information from the visualization system 742, such visualization data itself (e.g., still image, video, advanced spectrum visualization, etc.), visualization metadata (e.g., visualization type, resolution, frame rate, encoding, bandwidth, etc.).
  • Information from the surgical robot 743 may include any aforementioned information as applied to robotic instruments, sensors, and devices. Information from the surgical robot 743 may also include information related to the robotic operation or control of such instruments, such as electrical/mechanical feedback of robot articulators, system events, system settings, mechanical resolution, control operation log, articulator path information, and the like. The surgical robot 743 may receive information from the surgical computing device 704, such as control information, configuration information, operational updates (such as software /firmware), and the like.
  • a computer-implement surgical system e.g., computer-implement surgical system 750
  • further surgical information may be generated to reflect the changes.
  • a second surgical computing system 704b e.g., surgical hub
  • the messaging flow described here represents further surgical information flows 755 to be employed as disclosed herein (e.g., further consolidated, analyzed, and/or processed according to an algorithm, such as a machine learning algorithm).
  • the two surgical computing systems 704a, 704b request permission from a surgical operator for the second surgical computing system 704b (with the corresponding surgical robot 756) to take control of the operating room from the existing surgical computing system 704a.
  • the second surgical computing system 704b presents in the operating theater with control of the corresponding surgical robot 756, a robot visualization tower 758, a mono hat tool 759, and a robot stapler 749.
  • the permission can be requested through a surgeon interface or console 751.
  • the second surgical computing system 704b messages the existing surgical computing system 704a a request a transfer of control of the operating room.
  • the surgical computing systems 704a, 704b can negotiate the nature of their interaction without external input based on previously gathered data.
  • the surgical computing systems 704a, 704b may collectively determine that the next surgical task requires use of a robotic system. Such determination may cause the existing surgical computing system 704a to autonomously surrender control of the operating room to the second surgical computing system 704b. Upon completion of the surgical task, the second surgical computing system 704b may then autonomously return the control of the operating room to the existing surgical computing system 704a. As illustrated in FIG. 7C, the existing surgical computing system 704a has transferred control to the second surgical computing system 704b, which has also taken control of the surgeon interface 751 and the secondary display 752. The second surgical computing system 704b assigns new identification numbers to the newly transferred devices.
  • the existing surgical computing system 704a retains control the handheld stapler 753, the handheld powered dissector 754, and visualization tower 757.
  • the existing surgical computing system 704a may perform a supporting role, wherein the processing and storage capabilities of the existing surgical computing system 704a are now available to the second surgical computing system 704b.
  • FIG. 7D illustrates an example surgical information flow in the context of a surgical procedure and a corresponding example use of the surgical information for predictive modeling.
  • the surgical information disclosed herein may provide data regarding one or more surgical procedures, including the surgical tasks, instruments, instrument settings, operational information, procedural variations, and corresponding desirable metrics, such as improved patient outcomes, lower cost (e.g., fewer resources utilized, less surgical time, etc.).
  • Surgical information 762 from a plurality of surgical procedures 764 may be collected.
  • the surgical information 762 may be collected from the plurality of surgical procedures 764 by collecting data represented by the one or more information flows disclosed herein, for example.
  • example instance of surgical information 766 may be generated from the example procedure 768 (e.g, a lung segmentectomy procedure as shown on a timeline 769).
  • Surgical information 766 may be generated during the preoperative planning and may include patient record information.
  • Surgical information 766 may be generated from the data sources (e.g., data sources 726) during the course of the surgical procedure, including data generated each time medical personnel utilize a modular device that is paired with the surgical computing system (e.g., surgical computing system 704).
  • the surgical computing system may receive this data from the paired modular devices and other data sources
  • the surgical computing system itself may generate surgical information as part of its operation during the procedure.
  • the surgical computing system may record information relating to configuration and control operations.
  • the surgical computing system may record information related to situational awareness activities.
  • the surgical computing system may record the recommendations, prompts, and/or other information provided to the heathcare team (e.g., provided via a display screen) that may be pertinent for the next procedural step.
  • the surgical computing system may record configuration and control changes (e.g., the adjusting of modular devices based on the context). Such configuration and control changes may include activating monitors, adjusting the field of view (FOV) of a medical imaging device, changing the energy level of an ultrasonic surgical instrument or RF electrosurgical instrument, or the like.
  • the hospital staff members retrieve the patient's EMR from the hospital's EMR database. Based on select patient data in the EMR, the surgical computing system determines that the procedure to be performed is a thoracic procedure.
  • the staff members scan the incoming medical supplies for the procedure. The surgical computing system may cross-reference the scanned supplies with a list of supplies that are utilized in various types of procedures.
  • the surgical computing system may confirm that the mix of supplies corresponds to a thoracic procedure. Further, the surgical computing system may determine that the procedure is not a wedge procedure (because the incoming supplies either lack certain supplies that are necessary for a thoracic wedge procedure or do not otherwise correspond to a thoracic wedge procedure).
  • the medical personnel may also scan the patient band via a scanner that is communicably connected to the surgical computing system. The surgical computing system may confirm the patient's identity based on the scanned data.
  • the medical staff turns on the auxiliary equipment.
  • the auxiliary equipment being utilized can vary according to the type of surgical procedure and the techniques to be used by the surgeon. In this example, the auxiliary equipment may include a smoke evacuator, an insufflator, and medical imaging device.
  • the auxiliary equipment may pair with the surgical computing system.
  • the surgical computing system may derive contextual information about the surgical procedure based on the types of paired. In this example, the surgical computing system determines that the surgical procedure is a VATS procedure based on this particular combination of paired devices.
  • the contextual information about the surgical procedure may be confirmed by the surgical computing system via information from the patient's EMR.
  • the surgical computing system may retrieve the steps of the procedure to be performed. For example, the steps may be associated with a procedural plan (e.g., a procedural plan specific to this patient’s surgery, a procedural plan associated with a particular suigeon, a procedural plan template for the procedure generally, or the like).
  • the staff members attach the EKG electrodes and other patient monitoring devices to the patient.
  • the EKG electrodes and other patient monitoring devices pair with the surgical computing system.
  • the surgical computing system may receive data from the patient monitoring devices.
  • the medical personnel induce anesthesia in the patient.
  • the surgical computing system may record information related to this procedural step such as data from the modular devices and/or patient monitoring devices, including EKG data, blood pressure data, ventilator data, or combinations thereof, for example.
  • the patient's lung subject to operation is collapsed (ventilation may be switched to the contralateral lung).
  • the surgical computing system may determine that this procedural step has commenced and may collect surgical information accordingly, including for example, ventilator data, one or more timestamps, and the like
  • the medical imaging device e.g., a scope
  • the surgical computing system may receive the medical imaging device data (i.e., video or image data) through its connection to the medical imaging device.
  • the data from the medical imaging device may include imaging data and/or imaging metadata, such as the angle at which the medical imaging device is oriented with respect to the visualization of the patient's anatomy, the number or medical imaging devices presently active, and the like.
  • the surgical computing system may record positioning information of the medical imaging device.
  • one technique for performing a VATS lobectomy places the camera in the lower anterior corner of the patient's chest cavity above the diaphragm.
  • Another technique for performing a VATS segmentectomy places the camera in an anterior intercostal position relative to the segmental fissure.
  • the surgical computing system may be trained to recognize the positioning of the medical imaging device according to the visualization of the patient's anatomy.
  • one technique for performing a VATS lobectomy utilizes a single medical imaging device.
  • Another technique for performing a VATS segmentectomy uses multiple cameras.
  • Yet another technique for performing a VATS segmentectomy uses an infrared light source (which may be communicably coupled to the surgical computing system as part of the visualization system).
  • the surgical team begins the dissection step of the procedure.
  • the surgical computing system may collect data from the RF or ultrasonic generator indicating that an energy instrument is being fired.
  • the surgical computing system may cross-reference the received data with the retrieved steps of the surgical procedure to determine that an energy instrument being fired at this point in the process (i.e., after the completion of the previously discussed steps of the procedure) corresponds to the dissection step.
  • the energy instrument may be an energy tool mounted to a robotic arm of a robotic surgical system.
  • the surgical team proceeds to the ligation step of the procedure.
  • the surgical computing system may collect data received from the generator indicating that an RF or ultrasonic instrument is being fired and including the electrical and status information associated with the firing. Surgeons regularly switch back and forth between surgical stapling/ cutting instruments and surgical energy (i.e., RF or ultrasonic) instruments depending upon the particular step in the procedure.
  • the surgical computing system may collect surgical information 766 in view of the particular sequence in which the stapling/ cutting instruments and surgical energy instruments are used.
  • robotic tools may be used for one or more steps in a surgical procedure. The surgeon may alternate between robotic tools and handheld surgical instruments and/ or can use the devices concurrently, for example. Next, the incisions are closed up and the post-operative portion of the procedure begins.
  • the surgical computing system may collect surgical information regarding the patient emerging from the anesthesia based on ventilator data (e.g., the patient's breathing rate begins increasing), for example.
  • the medical personnel remove the various patient monitoring devices from the patient.
  • the surgical computing system may collect information regarding the conclusion of the procedure. For example, the surgical computing system may collect information related to the loss of EKG, BP, and other data from the patient monitoring devices.
  • the surgical information 762 (including the surgical information 766) may be structured and/or labeled. The surgical computing system may provide such structure and/or labeling inherently in the data collection.
  • Al may be used to perform complex tasks based on observations of data.
  • Al may be used to enable computing systems to perform cognitive tasks and solve complex tasks.
  • Al may include using machine learning and machine learning techniques.
  • ME techniques may include performing complex tasks, for example, without being programmed (e.g., explicitly programmed).
  • a ML technique may improve over time based on completing tasks with different inputs.
  • a ML process may train itself, for example using input data and/ or a learning dataset.
  • Machine learning (ML) techniques may be employed, for example, in the medical field.
  • ML may be used on a set of data (e.g., a set of surgical data) to produce an output (e.g., reduced surgical data, processed surgical data).
  • the output of a ML process may include identified trends or relationships of the data that were input for processing.
  • the outputs may include verifying results and/or conclusions associated with the input data.
  • an input to a ML process may include medical data, such as surgical images and patient scans.
  • the ML process may output a determined medical condition based on the input surgical images and patient scans.
  • the ML process may be used to diagnose medical conditions, for example, based on the surgical scans.
  • ML processes may improve themselves, for example, using the historic data that trained the ML processes and/or the input data. Therefore, ML processes may be constantly improving with added inputs and processing.
  • the ML processes may update based on input data.
  • ML techniques for data reductions may include using one or more of the following: CUR matrix decomposition; a decision tree; expectation-maximization (EM) processes (e.g., algorithms); explicit semantic analysis (ESA); exponential smoothing forecast; generalized linear model; k- means clustering (e.g., nearest neighbor); Naive Bayes; neural network processes; a multivariate analysis; an o-cluster; a singular value decomposition; Q-learning; a temporal difference (TD); deep adversarial networks; support vector machines (SVM); linear regression; reducing dimensionality; linear discriminant analysis (LDA); adaptive boosting (e.g., AdaBoost); gradient descent (e.g., Stochastic gradient descent (SGD)); outlier detection; and/or the like.
  • CUR matrix decomposition e.g., a decision tree
  • EM expectation-maximization
  • ESA explicit semantic analysis
  • exponential smoothing forecast generalized linear model
  • k- means clustering e.g., nearest neighbor
  • a CUR matrix decomposition may include using a matrix decomposition model (e.g., process, algorithm), such as a low-rank matrix decomposition model.
  • CUR matrix decomposition may include a low-rank matrix decomposition process that is expressed (e.g., explicitly expressed) in a number (e.g., small number) of columns and/or rows of a data matrix (e.g., the CUR matrix decomposition may be interpretable).
  • CUR matrix decomposition may include selecting columns and/or rows associated with statistical leverage and/or a large influence in the data matrix. Using CUR matrix decomposition may enable identification of attributes and/ or rows in the data matrix.
  • a random decision forest may add randomness (e.g., additional randomness) to a model, for example, while growing the trees.
  • a random forest may be used to search for a best feature among a random subset of features, for example, rather than searching for the most important feature (e.g., while splitting a node). Searching for the best feature among a random subset of features may result in a wide diversity that may result in a better (e.g., more efficient and/or accurate) model.
  • a random forest may include using parallel ensembling. Parallel ensembling may include fitting (e.g., several) decision tree classifiers in parallel, for example, on different data set sub-samples.
  • ESA may be used at a level of semantics (e.g., meaning) rather than on vocabulary (e.g., surface form vocabulary) of words or a document.
  • ESA may focus on the meaning of a set of text, for example, as a combination of the concepts found in the text.
  • ESA may be used in document classification.
  • ESA may be used for a semantic relatedness calculation (e.g., how similar in meaning words or pieces of text are to each other).
  • ESA may be used for information retrieval.
  • ESA may be used in document classification, for example.
  • Document classification may include tagging documents for managing and sorting. Tagging a document (e.g., with a keyword) may allow for easier searching.
  • ML techniques may be used to perform data reduction, for example, using linear regression.
  • Linear regression may be used to predict continuous outcomes.
  • linear regression may be used to predict the value of a variable (e.g., dependent variable) based on the value of a different variable (e.g., independent variable).
  • Linear regression may apply a linear approach for modeling a relationship between a scalar response and one or more explanatory variables (e.g., dependent and/or independent variables).
  • Simple linear regression may refer to linear regression use cases associated with one explanatory variable.
  • Multiple linear regression may refer to linear regression use cases associated with more than one explanatory variables.
  • Linear regression may model relationships, for example, using linear predictor functions.
  • the linear predictor functions may estimate unknown model parameters from a data set.
  • linear regression may be used to identify patterns within a training dataset.
  • the identified patterns may relate to values and/or label groupings.
  • the model may learn a relationship between the (e.g., each) label and the expected outcomes.
  • the model may be used on raw data outside the training data set (e.g., data without a mapped and/ or known output).
  • the trained model using linear regression may determine calculated predictions associated with the raw data, for example, such as identifying seasonal changes in sales data.
  • MT techniques may be used to perform data reduction, for example, a generalized linear model (GLM).
  • a GTM may be used as a flexible generalization of linear regression.
  • GTM may generalize linear regression, for example, by enabling a linear model to be related to a response variable.
  • a Naive Bayes model may be used to assign class labels to problem instances (e.g., represented as vectors of feature values).
  • the class labels may be drawn from a set (e.g., finite set).
  • Different processes e.g., algorithms
  • a family of processes e.g., family of algorithms
  • the family of processes may be based on a principle where the Naive Bayes classifiers (e.g., all the Naive Bayes) classifiers assume that the value of a feature is independent of the value of a different feature (e.g., given the class variable).
  • ML techniques may be used to perform data reduction, for example, using a neural network.
  • SVMs may behave differently, for example, based on different mathematical functions (e.g., the kernel, kernel functions).
  • kernel functions may include one or more of the following: linear, polynomial, radial basis function (RBF), sigmoid, etc.
  • the kernel functions may be used as a SVM classifier. SVM may be limited in use cases, for example, where a data set contains high amounts of noise (e.g., overlapping target classes).
  • SGD may include an iterative process used to optimize a function (e.g., objective function).
  • SGD may be used to optimize an objective function, for example, with certain smoothness properties.
  • Stochastic may refer to random probability.
  • SGD may be used to reduce computational burden, for example, in high-dimensional optimization problems.
  • SGD may be used to enable faster iterations, for example, while exchanging for a lower convergence rate.
  • a gradient may refer to the slop of a function, for example, that calculates a variable’s degree of change in response to another variable’s changes.
  • Gradient descent may refer to a convex function that outputs a partial derivative of a set of its input parameters.
  • a may be a learning rate and Ji may be a training example cost of the ith iteration.
  • the equation may represent the stochastic gradient descent weight update method at the jth iteration.
  • SGD may be applied to problems in text classification and/ or natural language processing (NLP).
  • NLP natural language processing
  • SGD may be sensitive to feature scaling (e.g., may need to use a range of hyperparameters, for example, such as a regularization parameter and a number of iterations).
  • ML techniques may be used to perform data reduction, for example, such as using outlier detection.
  • An outlier may be a data point that contains information (e.g., useful information) on an abnormal behavior of a system described by the data.
  • Outlier detection processes may include univariate processes and multivariate processes.
  • ML processes may be trained, for example, using one or more training methods.
  • ML processes may be trained using one or more of the following training techniques: supervised learning; unsupervised learning; semi-supervised learning; reinforcement learning; and/or the like.
  • Machine learning may be supervised (e.g., supervised learning).
  • a supervised learning algorithm may create a mathematical model from training a dataset (e.g., training data).
  • FIG. 8A illustrates an example supervised learning framework 800.
  • the training data e.g., training examples 802, for example, as shown in FIG. 8A
  • a training example 802 may include one or more inputs and one or more labeled outputs.
  • the labeled output(s) may serve as supervisory feedback.
  • a training example 802 may be represented by an array or vector, sometimes called a feature vector.
  • the training data may be represented by row(s) of feature vectors, constituting a matrix.
  • an objective function e.g., cost function
  • a supervised learning algorithm may leam a function (e.g., a prediction function) that may be used to predict the output associated with one or more new inputs.
  • a suitably trained prediction function may determine the output 804 (e.g., labeled outputs) for one or more inputs 806 that may not have been a part of the training data (e.g., input data without mapped labeled outputs, for example, as shown in FIG. 8A).
  • Example algorithms may include linear regression, logistic regression, neutral network, nearest neighbor, Naive Bayes, decision trees, SVM, and/or the like.
  • Example problems solvable by supervised learning algorithms may include classification, regression problems, and the like.
  • Machine learning may be unsupervised (e.g., unsupervised learning).
  • FIG. 8B illustrates an example unsupervised learning framework 810.
  • An unsupervised learning algorithm 814 may train on a dataset that may contain inputs 811 and may find a structure 812 (e.g., pattern detection and/or descriptive modeling) in the data.
  • the structure 812 in the data may be similar to a grouping or clustering of data points.
  • the algorithm 814 may learn from training data that may not have been labeled. Instead of responding to supervisory feedback, an unsupervised learning algorithm may identify commonalities in training data and may react based on the presence or absence of such commonalities in each training datum.
  • the training may include operating on a training input data to generate an model and/ or output with particular energy (e.g., such as a cost function), where such energy may be used to further refine the model (e.g., to define model that minimizes the cost function in view of the training input data).
  • energy e.g., such as a cost function
  • Example algorithms may include Apriori algorithm, K-Means, K-Nearest Neighbors (KNN), K-Medians, and the like.
  • Example problems solvable by unsupervised learning algorithms may include clustering problems, anomaly/ outlier detection problems, and the like
  • Machine learning may be semi-supervised (e.g., semi-supervised learning).
  • An operation step in reinforcement learning may include the agent observing an input state.
  • An operation step in reinforcement learning may include using a decision making function to make the agent perform an action.
  • An operation step may include (e.g., after an action is performed) the agent receiving a reward and/or reinforcement from the environment.
  • An operation step in reinforcement learning may include storing the state-action pair information about the reward.
  • Machine learning may be a part of a technology platform called cognitive computing (CC), which may constitute various disciplines such as computer science and cognitive science.
  • CC systems may be capable of learning at scale, reasoning with purpose, and interacting with humans naturally.
  • self-teaching algorithms that may use data mining, visual recognition, and/or natural language processing, a CC system may be capable of solving problems and optimizing human processes.
  • the hypothesis function may be a non-linear function (e.g., a highly non-linear function) that may include linear functions and logistic functions nested together with the outermost layer consisting of one or more logistic functions.
  • the NN algorithm may include a cost function to minimize classification errors by adjusting the biases and weights through a process of feedforward propagation and backward propagation. When a global minimum may be reached, the optimized hypothesis function with its layers of adjusted biases and weights may be deemed trained and constitute the model the training process has produced.
  • Data collection may be performed for machine learning as a first stage of the machine learning lifecycle. Data collection may include steps such as identifying various data sources, collecting data from the data sources, integrating the data, and the like.
  • the outputs may pass conclusions, results, and/or supporting metadata to the other ML models.
  • the outputs may be a portion of the complete dataset used in previous ML model processing.
  • multiple ML models may be processing data in different hub networks.
  • the different hub ML models may feed their results to ML models in the edge-network and/ or cloud network.
  • the information feeding from one system to a subsequent system may be variable (e.g., dependent on the capacities of the receiving system).
  • the information feeding from one system to a subsequent system may be variable, for example, based on the privacy level of the data and the receiving system’s status within a protected HIPAA network.
  • HCPs may look at the entirety of a dataset and determine that a problematic datapoint does not fit or does not have a rational explanation.
  • the problematic datapoint may be overridden but still allow for the collection of the semi-erroneous data.
  • HCPs may determine that datapoints are irregular but there are enough regular datapoints to continue.
  • an anesthesiologist may determine that a surgical procedure is in a critical step and the data is needed to perform the step.
  • the anesthesiologist may determine that there are sufficient accurate datapoints to make logical conclusions (e.g., based on knowledge, intuition, other data) in order to continue the procedure in a safe manner.
  • the interrelated ML models may include an ML model associated with performing data reduction.
  • an ML model may determine if a baseline (e.g., standard) control algorithm (e.g., parameter) should be substituted with a different (e.g., irregular) control algorithm (e.g., parameter).
  • the ML model may determine that the different (e.g., irregular) control algorithm may enable a surgical instrument to operate in a manner adapting to the surgical procedure.
  • the ML model may determine to use a different control algorithm, for example, based on a different biomarker of a functional instrument measurement.
  • the ML model may determine errant data sets relative to the ML boundary (e.g., as a separate process /computation), for example, to enable the ML model to determine if a baseline control algorithm should be substituted for a different control algorithm.
  • Serial use of models may include feeding results from a first ML model into a second ML model, for example, to compartmentalize the stages of an analysis.
  • Serial use of models may be useful, for example, if the stages produce meaningful trends the user may use as insight.
  • Serial use of models may be useful, for example, if there are checks along the stages to ensure that errors are more propagated within the several layers into the algorithm (e.g., in case the data is unbalanced or flawed).
  • Serial use of models may allow separation of overall processing resources, for example, such that multiple systems, locations, and/or separate networks may be used (e.g., to determine the overall trending/pattern identification), as shown in FIG. 10.
  • serial utilization of stacked ML algorithms may include training the ML models on the same set of training data.
  • the ML algorithms may include using a layer (e.g., additional layer) of a meta-classifier that takes in predicted values of the model and processes the predicted results, for example, to reduce error and strengthen the best outcomes from the different modeling techniques.
  • the data may be fed through the same level ML models separately with the outputs compared and adjusted by a meta-classifier.
  • Record sampling may be performed to form datasets that may be reduced (e.g., to identify key variables of data that need to be present to make a set more representative). For example, a system may determine to refrain from discarding data that has omitted data in categories and/ or portions of the data that are not influential in the determination of trends and/ or results (e.g., missing data would not affect overall processing task).
  • Algorithmic templates may be created using base datasets, for example, to evaluate a final value. Adding amounts of data that are properly mapped (e.g., accurate) may allow for evaluation of a trained ML model to see if the prediction (e.g., output) is correct.
  • Leveling data quality may include aggregating datasets.
  • mortality statistics may be used as a means to link outcomes with procedure steps (e.g., order, difficult, etc.) to complete missing nominal monitored statistics.
  • an APHAR risk score may be used by HCPs to estimate post-operative outcomes as a means for using the combined output of the lower fidelity clinical model as a means to determine a missing piece of data the higher fidelity ML model uses to make a prediction.
  • bariatric suitability pre-operational scoring may be used to complete data sets.
  • using one combined measure in combination to another combined measure to fill in missing aspects of either or another combined biometric aspect may be performed.
  • a patient’s APGAR and prolonged air leak risk scoring may be used to determine secondary uncollected data that in turn could be used by the machine learning to identify potential post-operative infection risk.
  • Clinical scoring systems may be limited by subjective limitations. For example, clinical scoring systems may employ subjective rating scales (e.g., patient’s pain level may differ between patients). Subjective rating scales may be difficult to evaluate.
  • Leveling data quality may include fixing imbalanced data, for example, by rescaling the data.
  • Data rescaling may include data normalization. Data rescaling may improve the quality of a dataset by reducing dimensions and/ or avoiding situations where some values overweight other values. Min-max normalization may be used.
  • Min-max normalization may include transforming numerical values to ranges (e.g., from 0.0 to 1.0 where 0.0 represents a minimal value and 1.0 represents a maximum value), for example, that may even out the weight of an attribute compared to other attributes in the dataset.
  • Decimal scaling may be used to perform data rescaling.
  • Decimal scaling may include moving decimal points in a direction to rescale the data.
  • ML processes may be used to ensure that the data is within a threshold amount of rescaling, for example, before further analysis. Data may be entered incorrectly (e.g., decimal point may be omitted). An ML process may detect that the incorrectly entered data is beyond a reasonable range and should be flagged for further analysis and/or review.
  • Leveling data quality may include fixing inadequate data, for example, using synthetic data.
  • Synthetic data may include artificially generated samples that mimic real-world data. Synthetic data may induce bias in data. The impact of synthetic data may be limited and/ or determined, for example, to minimize inadvertent data shifting due top the use and/ or inclusion of the synthetic data.
  • ML models may experience drift in predicting outputs for base datasets with the inclusion of synthetic data. The output of the ML model synthesizing data may be input to another ML model, for example, to ensure the synthetic data is not producing inappropriate results.
  • Leveling data quality may include fixing inconsistent medical term interchangeability. For example, a natural language filter may be used. A natural language filter may be used on medical implication terms within a dataset.
  • Sentiment analysis may be paired with geotracking to model how happy a population is.
  • Leveling data quality may include ensuring data consistency.
  • data formatting may be used to ensure data consistency.
  • Data formatting may include date formats, money denominations and symbols, numeric range settings, and/ or the like.
  • Discretizing data may be used to ensure data consistence. Predictions may be more effective, for example, based on turning numerical values into categorical values. Turning numerical values into categorical values may be performed by dividing the range of values into a number of groups.
  • Data structure may be used to compensate for data incompleteness. For example, consistency of a classification of learned instances may be improved and/ or ensured, for example, to ensure conclusions are trustworthy and/or reliable.
  • Context of the surgical procedure, patient, and/or surgical step may be used to assist an ML model in determining a floating boundary for groupings.
  • different (e.g., ten different) liver resection procedures may be recorded using a monitored scope.
  • the system may be aware that the data is associated with liver resection jobs.
  • the system may determine that the instruments are being used at the liver at predefined steps of the procedure.
  • the steps may be used to identify the liver (e.g., color, shape, location, etc.), for example, which may enable ML processes to define an accurate range of acceptable elements and/or aspects.
  • ML models may encounter scenarios where the models do not perform as expected (e.g., edge cases).
  • An edge case may be a problem and/or situation that occurs (e.g., only) at a certain operating parameter (e.g., minimum or maximum operating parameter).
  • An edge case may involve input values that may use special handling in an ML model.
  • Unit tests may be created, for example, to validate the behavior of ML models in edge cases. The unit tests may test the boundary conditions of an algorithm, function, and/ or method. A series of edge cases around a boundary may be used to give reasonable coverage and confidence (e.g., using an assumption that if it behaves correctly at the edges, it should behave correctly everywhere else).
  • Edge cases may occur, for example, based on a bias, variance, unpredictability, and/or the like.
  • a bias may be associated with the ML model being simple (e.g., too simple).
  • Bias may occur, for example, if an ML model cannot achieve good performance on a training data set. Bias may indicate that the architecture of an ML model does not have a structure that can represent nuances in training data. Variance may occur, for example, if the ML model is inexperienced (e.g., too inexperienced). If an ML model achieves good performance on its training data but performs poorly in testing, the training data set may be too small to adequately reflect the range of variability in a ML model’s operational environment.
  • Unpredictability may occur, for example, if the ML model operates in an environment experiencing variability and/or surprises.
  • ML may rely on finding regular patterns in input data.
  • a statistical variation may exist in data, but a ML model with an appropriate architecture and trained using enough training data may be able to find enough data regularity (e.g., achieve small enough bias and variance), for example, to make reliable decisions and minimizer edge cases.
  • a system may run multiple models (e.g., ML models) on differing portions of an incomplete dataset, for example, to determine which parameters have and do not have impacts (e.g., significant impacts) on outcomes.
  • the ML models may run metadata related to the portions of the data that are impactful but missing portions of the data, for example, to determine if there is metadata around the data collection that may help fill in the data (e.g., intelligent substitution or averaging) or determine trends that may be used in substitution to the primary missing data. For example, bleeding events may have a direct relationship to blood pressure of a patient. Blood pressure may not be tracked in real-time within the operating room during a surgical procedure. An electrocardiogram (EKG) version of heart rate monitoring may be used, for example, as a proxy for portions of the dataset that is missing blood pressure measurements with a nominal heart rate being set to a nominal blood pressure.
  • EKG electrocardiogram
  • the evaluation of an advanced energy device may be compared with bleeding results and using the blood pressure of the patient event, for example, if some of the patients did not have active blood pressure monitoring at the time of the surgery and imaging of the surgical site with the laparoscope.
  • the computing system may create a separate (e.g., independent) more complete dataset, for example, generated from and/or synthetically created and compared to the incomplete data set.
  • the separately generated dataset may be used to ensure regularity and can be using in ML models for processing. For example, similar datasets with similar outcomes and backgrounds may be combined into a more complete dataset for later analysis.
  • Utilization of outcomes resulting from similar procedures, patient biomarkers, and/or predictive trend measures may be used to create directional synthetic data and/or substitution of data (e.g., to complete an incomplete dataset). This may differ from random data generation because it is based on a known and/ or measured aspect of the patient, HCP, procedure, and/ or outcome.
  • the generated data may be supported by pre-established relationships of measured factors. For example, a first patient with irregular blood sugar may be tagged with a related stress level, which may be associated with high heart rate, which may result in difficult to manage bleeding issues.
  • a second patient may have similar difficult to managed bleeding, for example, as an event resulting from the same manger of advanced energy device usage. The second patient may not have data associated with blood sugar and/ or diabetes co-morbidities.
  • the heart rate variability may be a related measure of stress and/or pain, for example, which may be used to indicate both incomplete sets of data are resulting from stress or paint (e.g., not the blood sugar level, which may be a result, not a cause, of the stress). Both datasets may be made more complete with the measure of stress as the additional tag and/or category, for example, allowing both to be more complete and included with the analysis.
  • Synthetic data may be determined, for example, based on a probabilistic map of expected values from training data.
  • a probabilistic map may be generated, for example, by running known numbers through a trained MT model and recording the data outputs as a result. The generated map may be used as a search reference, for example, to predict missing portions of data.
  • the MT model may compartmentalize relationships of limited datasets collected to the interrelated but isolated outcomes of sub-functions, for example, which may enable the use of the more limited dataset directly.
  • the MT model may related the results higher into more advanced combined relationships.
  • the MT model may insert the generated substitute data into the dataset (e.g., to complete the dataset).
  • the MT model may determine that the initially incomplete and/or erroneous dataset is ready for subsequent processing (e.g., complete and/or regular).
  • the MT model may output the updated data set.
  • the master data set may be updated, for example, as more input data (e.g., from future surgical procedures) are fed into the MT model for processing.
  • the MT model may learn and constantly improve with each surgical procedure dataset input to the MT model.
  • the MT model may insert the revised dataset into the master dataset (e.g., to be used for future processing). With each iteration of processing data, the master data set may be updated and/ or improved.
  • a validation set may be used, for example, to verify outputs (e.g., from MT models). For example, a portion of a dataset may be set aside as validation data.
  • the validation dataset may be used on control algorithms, for example, that are generated from a cloud network and/or hospital network level cloud.
  • Validation datasets may be datasets that record data with a higher quality data than a standard procedure is expected to collect.
  • Certain data may be generated, such as, for example, jamming a device, and/ or operating outside of standard bounds, to put into the validation data set.
  • Devices and systems may be loaded and/ or overloaded to account for possible outcomes (e.g., including failure outcomes).
  • the validation dataset may be used to train MT models for multiple possible outcomes.
  • Validation datasets may be used to probe control algorithms (e.g., from other sources). For example, if a validation dataset is returned with predicted results that are different than what occurred in the procedure, the indication may be used to correct an error before deployment of the algorithm and/or a modification of the algorithm. If the validation dataset is returned with the correct predicted results from different control algorithms, an indication may be used to indicate there is a different insight due to some factor recorded in the dataset.
  • a master output may be used to check against an ML model output to confirm validation.
  • the master output may take timing to process the (e.g., all) applicable data sets to confirm validation.
  • portions of the algorithm and/or datasets may be validated (e.g., as opposed to the entire composition of the algorithm), for example, based on a risk-based approach.
  • the risk-based approach may expedite the results (e.g., while limited confidence in the output). The faster the output is produced may be associated with the higher the risk associated with the output.
  • a full master set of datasets may be created, for example, using highly instrumented procedures with exhaustive data collection and/or annotation practices (e.g., to ensure quality of data).
  • the master dataset may be used to train the first iteration(s) of an MT model, for example, before the MT model is deployed for use in operating theaters. Additional data may be collected for the master dataset, for example, after the deployment of the product. Additional data may be collected from controlled and/or singled out procedures that may be tooled for comprehensive data acquisition and/or labeling. System directed investigation of possible but inconclusive relationships from the original data may be performed. The additional data may be directed by a first MT model related to relationships that it identified that could have an interrelationship but the dataset was inconclusive. Targeted data collection and/ or analysis may be used to seek information and/ or interrelationships of a sub-portion of a primary set of information.
  • Preliminary relationship adjustment of some of the instruments within its reach may be used to result in minor changes in operation, for example, to monitor resulting behavior within the normal operation parameters of the device and/ or subsystem to extract relationship data.
  • an RF Bipolar device may use tissue impendence and terminations of a weld.
  • the triggering points may have a target impendence with a standard deviation that is acceptable for the triggering event to change the behavior. If the system identifies a potential relationship between the impedance value, the tissue type, the tissue thickness, and/or the resulting weld integrity, the system may direct generators that identify this set of parameters to adjust the impendence level trigger within its predefined acceptable range to one side or another side of the range (e.g., to validate or refute the potential relationship).
  • the master data set may be updated (e.g., a revised master data set may be generated) based on the updated Data Set A (e.g., as shown at 50704).
  • the ML model may determine a data type associated with portions of data in the data set.
  • a data type may be one or more of the following: surgical instrument parameters, surgical equipment parameters, patient information, patient biomarkers, HCP information, and/ or the like.
  • a data type may indicate that a piece of data is a patient biomarker, such as heart rate, for example.
  • the ML model may determine that there is a missing portion of heart rate data during a surgical procedure.
  • the ML model may determine to generate substitute data of the same type (e.g., substitute heart rate data).
  • ML models may be used to take multiple sets of problematic (e.g., incomplete, irregular, and/or erroneous) data and generate an independent complete dataset.
  • an ML model may receive a first dataset and a second dataset.
  • the ML model may be used to determine that the first and second datasets are problematic.
  • the ML model may determine that the first and second datasets are problematic (e.g., incomplete, irregular, and/or erroneous), for example, based on comparison to a verified data set (e.g., master data set).
  • a trigger for data exchange may be limited based on privacy concerns.
  • a trigger for data exchange may be expanded based on a processing system’s data needs (e.g., integral analysis needs).
  • the data exchange may consider both the privacy concerns and the processing system’s data needs.
  • a balancing test may be performed (e.g., considering the privacy concerns and the processing system’s data needs) to determine the data exchange behavior between systems.
  • Different systems performing different processing tasks may interact, for example, to determine data exchange behavior.
  • Data exchange between systems may be determined, for example, to meet processing goals of different processing systems.
  • processing systems e.g., ML models
  • the data received at the first data storage and/or first processing system may include non-private data and/or redacted data (e.g., data with private and/or confidential data removed).
  • a second data storage and/or second processing system (e.g., a level below the first data storage and/ or first processing system, for example, in the hierarchy) may receive a second data package.
  • the second data package may include the data in the first data package.
  • the second data package may include data associated with a privacy level higher than the privacy level in the first data package (e.g., the data in the second data package may have a low private information level).
  • the second data storage and/or second processing system may be enabled to store and/or process data associated with a higher privacy level than the first data storage and/ or first processing system.
  • the privacy level classifications for portions of surgical data may be compared to thresholds (e.g., privacy level thresholds) associated with data storages and/or processing systems, for example, to determine whether the portion of surgical data can be stored and/ or processed at the respective data storages and/or processing systems.
  • the thresholds may be predefined (e.g., based on HIPAA boundaries).
  • the thresholds may be used to balance privacy concerns with processing data needs, for example.
  • a data storage and/ or processing system within a controlled data network may have a privacy threshold that enables receiving more private and/ or confidential data.
  • a data storage and/or processing system outside a controlled data network may have a privacy threshold that restricts receiving private and/or confidential data (e.g., receives only non-private data).
  • portions of surgical data that are associated with data that identifies a patient may be classified with a high privacy or critical privacy level.
  • Surgical data associated with a high privacy or critical privacy level may be refrained from being transmitted (e.g., transmitted without redaction) to a location outside the facility network and/or cloud network.
  • Surgical data associated with a low privacy level or not private level may be enabled to be transmitted to (e.g., any) data storage and/or processing system (e.g., outside the facility and/ or edge network).
  • Private information in surgical data may be redacted, for example, to lower the privacy concerns associated with data.
  • surgical data associated with a high privacy level may be redacted (e.g., the identifying information may be redacted), for example, so the surgical data can be classified as a lower privacy level.
  • the redacted surgical data may conform to privacy limitations associated with a data storage and/or processing system (e.g., outside the facility and/ or edge network), for example, because it does not contain the identifying information (e.g., anymore).
  • the processing system may determine subsets of surgical data from the obtained surgical data.
  • subsets of surgical data may be discretized portions of the obtained surgical data.
  • the subsets of surgical data may determined, for example, based on the type of data, data format, data contents, and/or the like.
  • a subset of data may include data (e.g., only data) associated with a specific surgical instrument.
  • a subset of data may be a table of records (e.g., with fields as columns) associated with a specific patient.
  • a subset of data may include a particular column of data within a table of records, for example.
  • a subset of data may include any portion of the obtained surgical data (e.g., a specific data entry in a table of data, a row of data in a table of data, a column of data in a table of data).
  • a subset of data may include data associated with a specific surgical procedure.
  • a subset of data may include data associated with a specific surgical procedure for a specific patient, for example.
  • the second subset of data may be subjected to restrictions on transmittal (e.g., HIPAA restrictions).
  • the second subset of data may be refrained from being sent to a data storage and/or processing system outside the facility network and/or edge network (e.g., refrained from being sent to a cloud network).
  • the processing system 50750 may determine processing goal(s).
  • the processing goal(s) may include an overarching processing goal (e.g., associated with a MF model hierarchy 50756).
  • the processing goal(s) may include separate processing goals for each ML model in the ML model hierarchy.
  • a first ML model may be associated with data reduction and/ or data preparation and a second ML model may be associated with trend analysis and/ or the like.
  • the ML models may perform processing tasks as described herein with respect to FIGs. 9-17.
  • a data needs (e.g., for each ML model) may be determined based on a determined processing goal (e.g., for each of the ML models).
  • the data needs may include the data used (e.g., needed) to perform and/or complete the processing goal.
  • a processing goal may be data reduction to perform trend analysis.
  • the data needs associated with the processing goal may include data used to perform the trend analysis.
  • the data needs may consider subsequent ML models (e.g., the subsequent ML model’s processing goals). For example, a first ML model may perform preprocessing and data reduction on data and a second ML model may perform trend analysis for a specific biomarker. The data needs for the first ML model may consider the data used in the second ML model.
  • data packages may be determined, for example, for the ML models in the ML model hierarchy 50756. Different data packages may be determined and sent to the ML models. For example, a first data package may be determined for ML Model 1 50758 and an Nth data package may be determined for ML Model N 50760. ML Model N 50760 may be the lowest level in the ML Model Hierarchy 50756. The lowest level in the ML Model Hierarchy 50756 (e.g., ML Model N 50760) may receive the most complete data package (e.g., as compared to the other data packages determined for the other ML models in the ML Model Hierarchy).
  • the lower the level in the ML Model Hierarchy the more complete the data package may be.
  • the more complete data packages may include more surgical data (e.g., private and/or confidential data) as compared with data packages determined for higher level ML models.
  • the level-based system may be designed, for example, to limit private information from being sent to specific levels in the ML Model Hierarchy.
  • the lowest level ML model e.g., only the lowest level ML model
  • the level-based system may provide added security precautions, for example, in the event of a data breach.
  • the processing system may determine a first data package for a first ML model and a second data package for a second ML model.
  • the second ML model may be a lower level ML model in the ML Model Hierarchy as compared to the first ML model.
  • the first data package may be determined based on the data needs and/ or processing goals associated with the first ML model.
  • the second data package may be determined based on the data needs and/ or processing goals associated with the second ML model.
  • the output of the first ML model may be sent to the second ML model, for example.
  • the first data package may be determined based on considering that the output of the first ML model will be sent to the second ML model.
  • the second data package may include the data included in the first data package.
  • the second data package may include at least a portion of the data included in the first data package.
  • the ML Model Hierarchy may include ML models outside of the processing system 50750.
  • FIG. 19 illustrates example ML models in located in the facility network, edge network, and cloud network.
  • the ML models may process data at a different location and/or in a different processing system.
  • the processing system 50750 may be located in a medical facility (e.g., within a facility network 50800 as shown in FIG. 19 and/ or within an edge network 50802 as shown in FIG. 19).
  • the facility network may be contained within the edge network (e.g., as shown in FIG. 19).
  • the ML Model Hierarchy may include ML models within the facility network, edge network, cloud network, and/or the like.
  • a first ML model (e.g., highest level ML model) in a ML model hierarchy may have a classification threshold associated with a zero-privacy level.
  • the first ML model may be enabled to receive subsets of data associated with zero privacy implications (e.g., no private and/ or confidential information).
  • the first ML model may be refrained from being sent and/or receiving subsets of data with any privacy implications.
  • Subsequent ML models (e.g., lower level models) in the ML model hierarchy may have privacy classification thresholds that are associated with receiving more private data.
  • the second data package may be determined to include a second portion of data that is below (e.g., or alternatively above) the first privacy classification threshold.
  • Data exchange behavior may be dynamic. For example, processing goals associated with ML models may change. The changed processing goals may affect how data is exchanged between systems (e.g., ML models). For example, a change in processing goals (e.g., in a ML model hierarchy) may be determined. Based on the change in processing goals, an updated processing goal may be determined (e.g., for a ML model). The change in processing goal in a first ML model may affect the processing goals and/ or data exchange of other ML models in the ML model hierarchy.
  • the output from such a machine learning model may not be accurate due to the complexity of the relationships in the training data that it has to generalize.
  • the performance of a machine learning model is limited by the quality of the data used in the machine learning algorithm to train the model. While researchers and practitioners have focused on improving the quality of machine learning algorithms and models themselves (for example, by adapting the architecture of a neural network), there are limited efforts towards improving the data quality. Failure to do so can result in inaccurate outcomes, which are particularly undesirable when dealing with surgical data that is used for determining surgical classifications, surgical data trends, or surgical recommendations. Poor surgical classifications, surgical data trends and surgical recommendations may have significant effect on a patient’s health.
  • method further comprises: determining a first processing capability associated with the first processing task, wherein the first data set is further determined based on the first processing capability; determining a second processing capability associated with the second processing task, wherein the second data set is further determined based on the second processing capability; and determining a third processing capability associated with the third processing task, wherein the first output is further generated based on the third processing capability, and wherein the second output is further generated based on the third processing capability.
  • at least one of the first processing task or the second processing task is associated with data preparation.
  • the third processing task is associated with determining one or more of surgical data classifications, surgical data trends, or surgical recommendations .
  • the processor is further configured to: insert the revised first set of surgical data into the master set of surgical data.
  • the processor is further configured to: determine that data in the master set of surgical data needs to be revised based on the revised first set of surgical data; and generate a revised master set of surgical data comprising at least a third portion of the revised first set of surgical data and a portion of the master set of surgical data.
  • the processor is further configured to: obtain a verification set of data associated with the master set of surgical data; and determine that the revised first set of surgical data is valid based on the verification set of data. 22.
  • determining that the at least a first portion of the first set of surgical data is problematic comprises determining that the at least a first portion of the first set of surgical data is incomplete or irregular. 26.
  • the method further comprises: determining that the first processing goal has changed; based on the determination that the first processing goal has changed, determining an updated first processing goal; determining an updated first classification threshold based on the updated first processing goal; determining a third data package based on the updated first processing goal and the updated first classification threshold; determining an updated second processing goal based on the determination that the first processing goal has changed; determining an updated second classification threshold based on the updated second processing goal; and determining a fourth data package based on the updated second processing goal and the updated second classification threshold. 55.
  • a surgical computing system comprising: a processor configured to: obtain a surgical data comprising a plurality of subsets of surgical data; determine a respective privacy level for each subset of the subsets of surgical data; determine a first processing goal and a second processing goal, wherein the first processing goal is associated with a first processing task and a first data needs, and wherein the second processing goal is associated with a second processing task and a second data needs; determine a first privacy threshold associated with the first processing task and a second privacy threshold associated with the second processing task; determine a first data package based on the first processing goal, the first data needs, and the first privacy threshold, wherein the first data package comprises at least a first portion of the surgical data; determine a second data package based on the second processing goal, the second data needs, and the second privacy threshold, wherein the second data package comprises at least a second portion of the surgical data; and send the first data package and the second data package.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • Epidemiology (AREA)
  • General Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • Biomedical Technology (AREA)
  • Data Mining & Analysis (AREA)
  • Pathology (AREA)
  • Databases & Information Systems (AREA)
  • General Business, Economics & Management (AREA)
  • Business, Economics & Management (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Surgery (AREA)
  • Urology & Nephrology (AREA)
  • Bioethics (AREA)
  • Medical Treatment And Welfare Office Work (AREA)

Abstract

L'invention concerne des systèmes, des procédés et des instrumentalités pour utiliser des modèles d'apprentissage automatique (AA) interdépendants (par exemple, des algorithmes). Les modèles AA interdépendants peuvent agir collectivement pour exécuter des parties complémentaires d'une analyse chirurgicale. Les modèles AA peuvent être utilisés à divers emplacements. Par exemple, des modèles AA peuvent être mis en œuvre dans un réseau d'installation, un réseau en nuage, un réseau périphérique et/ou similaire. L'emplacement des modèles AA peut influencer le type de données du processus de modèles AA. Par exemple, des modèles AA utilisés à l'extérieur d'une limite HIPAA (par exemple, un réseau en nuage) peuvent traiter des informations non privées et/ou non confidentielles. Les modèles AA peuvent être utilisés pour permettre d'obtenir leurs résultats respectifs dans d'autres modèles AA pour permettre d'obtenir un résultat plus complet. L'invention concerne également des systèmes, des procédés et des instrumentalités pour agréger et/ou répartir des données chirurgicales disponibles dans un ensemble de données plus utilisable pour une interaction de modèle d'apprentissage automatique (AA) (par exemple, un algorithme). Un modèle AA peut être plus précis et/ou fiable si on utilise des données complètes et/ou régulières. L'agrégation et/ou la répartition de données chirurgicales disponibles peuvent permettre un jeu de données plus complet et/ou régulier pour une analyse de modèle AA. L'invention concerne en outre des systèmes, des procédés et des instrumentalités pour un système informatique chirurgical présentant un support pour une interaction de modèle d'apprentissage automatique. Un comportement d'échange de données entre des modèles d'apprentissage automatique (AA) et des mémoires de données peut être déterminé et mis en œuvre. Par exemple, un échange de données peut être déterminé sur la base d'implications de confidentialité associées à un modèle AA et/ou à un stockage de données. Un échange de données peut être déterminé sur la base d'objectifs de traitement associés à des modèles AA.
PCT/IB2023/063318 2022-12-30 2023-12-28 Système informatique chirurgical avec support pour modèles d'apprentissage automatique interdépendants Ceased WO2024141971A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP23841347.0A EP4490747A1 (fr) 2022-12-30 2023-12-28 Système informatique chirurgical avec support pour modèles d'apprentissage automatique interdépendants

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US18/092,015 2022-12-30
US18/092,019 US20240221923A1 (en) 2022-12-30 2022-12-30 Surgical computing system with support for machine learning model interaction
US18/092,023 US12573495B2 (en) 2022-12-30 2022-12-30 Surgical computing system with support for interrelated machine learning models
US18/092,019 2022-12-30
US18/092,023 2022-12-30
US18/092,015 US20240221892A1 (en) 2022-12-30 2022-12-30 Surgical computing system with support for interrelated machine learning models

Publications (1)

Publication Number Publication Date
WO2024141971A1 true WO2024141971A1 (fr) 2024-07-04

Family

ID=89619692

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2023/063318 Ceased WO2024141971A1 (fr) 2022-12-30 2023-12-28 Système informatique chirurgical avec support pour modèles d'apprentissage automatique interdépendants

Country Status (2)

Country Link
EP (1) EP4490747A1 (fr)
WO (1) WO2024141971A1 (fr)

Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140263552A1 (en) 2013-03-13 2014-09-18 Ethicon Endo-Surgery, Inc. Staple cartridge tissue thickness sensor system
US20170202605A1 (en) 2016-01-15 2017-07-20 Ethicon Endo-Surgery, Llc Modular battery powered handheld surgical instrument and methods therefor
US20170296213A1 (en) 2016-04-15 2017-10-19 Ethicon Endo-Surgery, Llc Systems and methods for controlling a surgical stapling and cutting instrument
US20180049822A1 (en) 2016-08-16 2018-02-22 Ethicon Endo-Surgery, Llc Control of advancement rate and application force based on measured forces
US20190059929A1 (en) 2017-08-29 2019-02-28 Ethicon Llc Methods, systems, and devices for controlling electrosurgical tools
US20190104919A1 (en) 2012-05-20 2019-04-11 Ethicon Llc Method for situational awareness for surgical network or surgical network connected device capable of adjusting function based on a sensed situation or usage
US20190200844A1 (en) 2017-12-28 2019-07-04 Ethicon Llc Method of hub communication, processing, storage and display
US20190200981A1 (en) 2017-12-28 2019-07-04 Ethicon Llc Method of compressing tissue within a stapling device and simultaneously displaying the location of the tissue within the jaws
US20190201047A1 (en) 2017-12-28 2019-07-04 Ethicon Llc Method for smart energy device infrastructure
US20190201137A1 (en) 2017-12-28 2019-07-04 Ethicon Llc Method of robotic hub communication, detection, and control
US20190206564A1 (en) 2017-12-28 2019-07-04 Ethicon Llc Method for facility data collection and interpretation
US20190206569A1 (en) 2017-12-28 2019-07-04 Ethicon Llc Method of cloud based data analytics for use with the hub
US20190206562A1 (en) 2017-12-28 2019-07-04 Ethicon Llc Method of hub communication, processing, display, and cloud analytics
US20200405296A1 (en) 2019-06-28 2020-12-31 Ethicon Llc Packaging for a replaceable component of a surgical stapling system
US10881399B2 (en) 2017-06-20 2021-01-05 Ethicon Llc Techniques for adaptive control of motor velocity of a surgical stapling and cutting instrument
WO2021113168A1 (fr) * 2019-12-03 2021-06-10 Tornier, Inc. Modèles appris par machine dans la prise en charge d'interventions chirurgicales
US20210244487A1 (en) * 2020-02-12 2021-08-12 DePuy Synthes Products, Inc. Technologies for monitoring and predicting impaction state of an orthopaedic surgical implement during an orthopaedic surgical procedure

Patent Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190104919A1 (en) 2012-05-20 2019-04-11 Ethicon Llc Method for situational awareness for surgical network or surgical network connected device capable of adjusting function based on a sensed situation or usage
US20140263552A1 (en) 2013-03-13 2014-09-18 Ethicon Endo-Surgery, Inc. Staple cartridge tissue thickness sensor system
US9345481B2 (en) 2013-03-13 2016-05-24 Ethicon Endo-Surgery, Llc Staple cartridge tissue thickness sensor system
US20170202605A1 (en) 2016-01-15 2017-07-20 Ethicon Endo-Surgery, Llc Modular battery powered handheld surgical instrument and methods therefor
US20170296213A1 (en) 2016-04-15 2017-10-19 Ethicon Endo-Surgery, Llc Systems and methods for controlling a surgical stapling and cutting instrument
US20180049822A1 (en) 2016-08-16 2018-02-22 Ethicon Endo-Surgery, Llc Control of advancement rate and application force based on measured forces
US10881399B2 (en) 2017-06-20 2021-01-05 Ethicon Llc Techniques for adaptive control of motor velocity of a surgical stapling and cutting instrument
US20190059929A1 (en) 2017-08-29 2019-02-28 Ethicon Llc Methods, systems, and devices for controlling electrosurgical tools
US20190200981A1 (en) 2017-12-28 2019-07-04 Ethicon Llc Method of compressing tissue within a stapling device and simultaneously displaying the location of the tissue within the jaws
US20190201047A1 (en) 2017-12-28 2019-07-04 Ethicon Llc Method for smart energy device infrastructure
US20190201137A1 (en) 2017-12-28 2019-07-04 Ethicon Llc Method of robotic hub communication, detection, and control
US20190206564A1 (en) 2017-12-28 2019-07-04 Ethicon Llc Method for facility data collection and interpretation
US20190206569A1 (en) 2017-12-28 2019-07-04 Ethicon Llc Method of cloud based data analytics for use with the hub
US20190206562A1 (en) 2017-12-28 2019-07-04 Ethicon Llc Method of hub communication, processing, display, and cloud analytics
US20190200844A1 (en) 2017-12-28 2019-07-04 Ethicon Llc Method of hub communication, processing, storage and display
US20200405296A1 (en) 2019-06-28 2020-12-31 Ethicon Llc Packaging for a replaceable component of a surgical stapling system
WO2021113168A1 (fr) * 2019-12-03 2021-06-10 Tornier, Inc. Modèles appris par machine dans la prise en charge d'interventions chirurgicales
US20210244487A1 (en) * 2020-02-12 2021-08-12 DePuy Synthes Products, Inc. Technologies for monitoring and predicting impaction state of an orthopaedic surgical implement during an orthopaedic surgical procedure

Also Published As

Publication number Publication date
EP4490747A1 (fr) 2025-01-15

Similar Documents

Publication Publication Date Title
US20240221892A1 (en) Surgical computing system with support for interrelated machine learning models
US20240221923A1 (en) Surgical computing system with support for machine learning model interaction
US12531156B2 (en) Method for advanced algorithm support
US12573495B2 (en) Surgical computing system with support for interrelated machine learning models
US12315617B2 (en) Surgical data processing associated with multiple system hierarchy levels
US12475978B2 (en) Adaptable operation range for a surgical device
US20240216065A1 (en) Surgical computing system with intermediate model support
US20240221896A1 (en) Surgical data processing associated with multiple system hierarchy levels
US20240220763A1 (en) Data volume determination for surgical machine learning applications
US12562254B2 (en) Surgical data specialty harmonization for training machine learning models
US20240221931A1 (en) Adaptive surgical data throttle
US20240221960A1 (en) Modifying globally or regionally supplied surgical information related to a surgical procedure
US12254977B2 (en) Detection of knock-off or counterfeit surgical devices
US20240216081A1 (en) Peer-to-peer surgical instrument monitoring
US12573496B2 (en) Advanced data timing in a surgical computing system
WO2024141967A1 (fr) Traitement de données chirurgicales associé à de multiples niveaux de hiérarchie de système
WO2024141971A1 (fr) Système informatique chirurgical avec support pour modèles d'apprentissage automatique interdépendants
WO2024141974A1 (fr) Données chirurgicales pour apprentissage automatique

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23841347

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023841347

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2023841347

Country of ref document: EP

Effective date: 20241008

NENP Non-entry into the national phase

Ref country code: DE