WO2020087960A1 - 一种影像识别的方法、装置、终端设备和医疗系统 - Google Patents

一种影像识别的方法、装置、终端设备和医疗系统 Download PDF

Info

Publication number
WO2020087960A1
WO2020087960A1 PCT/CN2019/093602 CN2019093602W WO2020087960A1 WO 2020087960 A1 WO2020087960 A1 WO 2020087960A1 CN 2019093602 W CN2019093602 W CN 2019093602W WO 2020087960 A1 WO2020087960 A1 WO 2020087960A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
lesion
medical image
recognition
degree
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/093602
Other languages
English (en)
French (fr)
Inventor
付星辉
孙钟前
尚鸿
杨巍
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to EP19879486.9A priority Critical patent/EP3876192B1/en
Priority to JP2020561046A priority patent/JP7152513B2/ja
Publication of WO2020087960A1 publication Critical patent/WO2020087960A1/zh
Priority to US17/078,878 priority patent/US11410306B2/en
Anticipated expiration legal-status Critical
Priority to US17/856,043 priority patent/US11610310B2/en
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30092—Stomach; Gastric
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images

Definitions

  • the present invention relates to the field of image processing technology, and in particular, to a method, device, terminal device, and medical system for image recognition.
  • the terminal device can determine the probability of a patient's cancer based on the patient's esophagus image through computer technology. In this way, the doctor can perform further diagnostic analysis based on the output result, which improves the accuracy and efficiency of medical diagnosis.
  • a conventional machine learning method eg, wavelet operator
  • a machine learning method is generally used to classify the image based on pre-extracted feature information to obtain a disease classification result.
  • Embodiments of the present invention provide an image recognition method, device, terminal device, and medical system, which are used to improve the efficiency and accuracy of recognition when performing disease recognition based on medical images.
  • the medical image is recognized through a second recognition model to generate a lesion degree recognition result indicating the degree of the lesion.
  • Acquisition unit used to acquire medical images to be identified
  • the discriminating unit is used to discriminate the medical image through the first recognition model and generate a disease recognition result indicating whether the medical image includes a disease;
  • the recognition unit is configured to recognize the medical image through the second recognition model when the lesion recognition result indicates that the medical image includes a lesion, and generate a lesion degree recognition result indicating the degree of the lesion.
  • Obtain a recognition model to identify the degree of lesion output after identifying the medical image to be identified, and the degree of lesion recognition result includes: the multiple image blocks included in the medical image, the degree of recognition of the lesion degree of each image block and its medical image Region information of the region, and / or the disease degree indicator image after corresponding indication information is set in the corresponding region according to the recognition result of the disease degree of each image block;
  • the recognition model to recognize medical images specifically includes: dividing the medical image into multiple image blocks, and for each image block, extracting the feature information of the image block and determining the degree of lesion recognition of the image block according to the extracted feature information result.
  • the input unit is used to obtain the medical image to be recognized, and to recognize the medical image to be recognized through the recognition model;
  • the obtaining unit is used to obtain a lesion degree recognition result output after the recognition model recognizes the medical image to be recognized, and the lesion degree recognition result includes: among multiple image blocks included in the medical image, the lesion degree recognition result of each image block and The area information in the medical image, and / or the pathological degree indication image after corresponding indication information is set in the corresponding area according to the identification result of the pathological degree of each image block;
  • the recognition model to recognize medical images specifically includes: dividing the medical image into multiple image blocks, and for each image block, extracting the feature information of the image block and determining the degree of lesion recognition of the image block according to the extracted feature information result.
  • the terminal device of each embodiment may include at least one processing unit and at least one storage unit, where the storage unit stores a computer program, and when the program is executed by the processing unit, causes the processing unit to perform any of the steps of the image recognition method described above .
  • the medical system of each embodiment may include an image acquisition device and an image recognition device, wherein,
  • Image collection device used to collect medical images of patients
  • the image recognition device is used to obtain the medical image collected by the image collection device, and to discriminate the medical image through the first recognition model, to generate a disease recognition result indicating whether the medical image includes a disease, and when the disease recognition result When indicating that the medical image includes a lesion, identify the medical image through a second recognition model to generate a lesion degree recognition result indicating the degree of the lesion;
  • the display device is used for presenting the recognition result of the lesion degree.
  • Another medical system of each embodiment may include an image acquisition device and an image recognition device, where,
  • Image collection device used to collect medical images of patients
  • the image recognition device is used to obtain the medical images collected by the image collection device, and to recognize the medical images including the lesion through the second recognition model to generate the recognition result of the lesion degree; wherein, the second recognition model is used to recognize the medical image The extent of the lesion;
  • the display device is used for presenting the recognition result of the lesion degree.
  • the computer-readable storage medium of each embodiment stores a computer program, and when the program is executed by one or more processors, the one or more processors may be caused to perform any of the above steps of the image recognition method.
  • whether the medical image is a medical image with a lesion is determined by the first recognition model, and then, the medical image with the lesion is detected by the second recognition model Further identification is performed to obtain a lesion degree recognition result to indicate the extent to which the medical image includes the lesion. This does not require manual analysis and custom feature extraction schemes, which improves the efficiency and accuracy of medical image recognition.
  • FIG. 1 is a schematic structural diagram of a terminal device according to an embodiment of the present invention.
  • 3a is a schematic diagram of the principle of a first recognition model in an embodiment of the present invention.
  • FIG. 3b is an example diagram 1 of an esophagus image in an embodiment of the present invention.
  • 3c is a schematic diagram of the principle of a second recognition model in an embodiment of the present invention.
  • 3d is a schematic diagram of an image block in an embodiment of the present invention.
  • 3e is an example diagram 2 of an esophageal image in an embodiment of the present invention.
  • FIG. 3f is an example of an image indicating the degree of lesion of an esophageal image in an embodiment of the present invention
  • 3g is a schematic structural diagram of a medical system according to an embodiment of the present invention.
  • 4a is a schematic structural diagram 1 of an image recognition device according to an embodiment of the present invention.
  • 4b is a second schematic structural diagram of an image recognition device according to an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of a terminal device in an embodiment of the present invention.
  • embodiments of the present invention provide an image identification method, device, terminal device, and medical system.
  • Terminal device A device that can install various types of applications and can display the entities provided in the installed applications.
  • the electronic device can be mobile or fixed. For example, mobile phones, tablet computers, in-vehicle devices, personal digital assistants (PDAs) or other electronic devices that can achieve the above functions.
  • PDAs personal digital assistants
  • Medical imaging It is the material reproduction of human visual perception, which can be obtained by optical equipment, such as cameras, mirrors, telescopes, and microscopes; it can also be created manually, such as hand-painted images. Pathology can be recorded and stored on paper media, film, and other media sensitive to light signals. With the development of digital acquisition technology and signal processing theory, more and more medical images are stored in digital form.
  • CNN Convolutional Neural Network
  • Dense convolutional network Each layer is connected to any other layer in a feed-forward form, that is, any layer is not only connected to the adjacent layer, but also to all subsequent layers All have direct connections.
  • the first recognition model a model obtained by training DenseNet training on normal image samples and lesion image samples, which is used to determine whether the medical image has a lesion.
  • whether or not a medical image has a lesion also referred to as whether the medical image includes a lesion, refers to whether the medical image shows that a disease has occurred in the organ, that is, whether the medical image includes the image content corresponding to the organ lesion.
  • Second recognition model a model obtained by training a medical image sample with various degrees of lesions using a convolutional neural network, which is used to identify the degree of lesions in medical images.
  • Lesion recognition result It is used to indicate whether the input medical image is a medical image with a lesion.
  • Pathological degree recognition result used to indicate the degree of pathological changes in medical images.
  • Esophageal cancer is a malignant tumor in the upper digestive tract.
  • Patients with esophageal cancer at different times have very different treatment procedures. Early patients are mainly treated by endoscopic minimally invasive treatment. They can be discharged within 3 to 5 days after surgery. The cost of treatment is low and there are few complications. 90% can be cured.
  • patients with advanced esophageal cancer are mainly treated by thoracotomy / abdominal / neck "three incision" surgery. This treatment is invasive, expensive, and unsatisfactory. The cure rate is less than 40%.
  • the normal esophageal image is generally smooth on the surface mucosa, and the diseased esophageal image has obvious features, such as bumps and erosion.
  • the difference between normal and diseased esophageal images may be a small area in the image ( For example, the color depth and the roughness of the skin) may also be due to the change in the overall smoothness of the image.
  • This application uses artificial intelligence technology to identify lesions from medical images and identify the extent of the lesions.
  • AI systems refer to computer systems that exhibit intelligent behavior.
  • the functions of the AI system include learning, maintaining a large number of knowledge bases, performing reasoning, applying analytical capabilities, discerning the relationship between facts, exchanging ideas with others, understanding the communication of others, and perceiving and understanding the situation, etc.
  • AI systems can make machines progress through their own learning and judgment.
  • AI systems create new knowledge by looking for previously unknown patterns in data, and drive solutions by learning data patterns.
  • the recognition rate of the AI system can be improved, and the user's taste can be more accurately understood. Therefore, the existing rule-based intelligent systems are gradually replaced by AI systems.
  • neural networks are usually used. Neural networks are computer systems designed, constructed, and configured to simulate the human nervous system.
  • the neural network architecture consists of an input layer, an output layer, and one or more hidden layers.
  • the input layer inputs data into the neural network.
  • the output layer produces guess results.
  • the hidden layer assists in information dissemination.
  • a neural network or artificial neural network is based on a collection of connected units called neurons or artificial neurons. Each connection (synapse) between neurons can transmit signals to another neuron.
  • the receiving (post-synaptic) neuron can process the signal and then signal the downstream neuron connected to the neuron.
  • Neurons can have states, usually represented by real numbers, usually between 0 and 1.
  • Neurons and synapses can also have weights that change as the learning progresses, and the weights are used to increase or decrease the strength of the signal they send downstream.
  • the neuron may have a threshold so that the downstream signal is only sent when the aggregated signal is below (or above) this level.
  • neurons are organized in layers. Different layers can perform different types of conversions on their inputs. The signal may move from the first (input) layer to the last (output) layer after traversing the layers multiple times.
  • the initial layer can detect primitives (eg, pupils, irises, eyelashes, etc. in the eye), and its output is fed forward to deeper layers that perform more abstract generalizations (eg, eyes, mouth). And so on, until the last layer performs complex object recognition (for example, face).
  • Neural networks are trained using data such as a series of data points. The neural network guesses which response should be given, and compares the guess with the correct "best" guess for each data point. If an error occurs, the neuron is adjusted and the process is repeated.
  • Some embodiments of the present application may use convolutional neural networks in neural networks to process and recognize medical images.
  • Convolutional neural networks also known as ConvNets or CNN
  • CNN convolutional neural networks
  • CNNs convolutional neural networks
  • CNNs convolutional neural networks
  • DenseNet-121 is used to train normal image samples and lesion image samples to obtain a first recognition model, so as to determine whether a medical image has a lesion.
  • a convolutional neural network Train the medical image with the lesion to obtain the second recognition model, and determine the canceration recognition result of the medical image based on the lesion degree of the image block with the most severe lesion in the medical image.
  • the above description takes esophageal cancer as an example, but the diagnosis of other cancers also has similar problems.
  • the method provided in the embodiments of the present application to perform preliminary lesion discrimination on medical images through the first recognition model, and to further identify the degree of lesions on the medical images where the lesions occur through the second recognition model to determine whether the medical images include
  • the degree of the lesion greatly improves the efficiency and accuracy of cancer identification.
  • the second recognition model can also be used to directly recognize the medical image to determine the degree of canceration of the medical image.
  • An image recognition method provided by an embodiment of the present invention may be applied to a terminal device, and the terminal device may be a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), and so on.
  • the terminal device may be a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), and so on.
  • FIG. 1 shows a schematic structural diagram of a terminal device.
  • the terminal device 100 includes a processor 110, a memory 120, a power supply 130, a display unit 140, and an input unit 150.
  • the processor 110 is the control center of the terminal device 100, connects various components using various interfaces and lines, and executes various functions of the terminal device 100 by running or executing software programs and / or data stored in the memory 120, thereby The equipment is monitored overall.
  • the processor 110 may include one or more processing units.
  • the memory 120 may mainly include a storage program area and a storage data area.
  • the storage program area may store an operating system, various application programs, etc .
  • the storage data area may store data created according to the use of the terminal device 100 and the like.
  • the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
  • FIG. 1 is only an example of a terminal device, and does not constitute a limitation on the terminal device, and may include more or fewer components than those illustrated, or combine some components, or different components.
  • FIG. 2 it is an implementation flowchart of an image recognition method provided by the present invention.
  • the specific implementation process of the method is as follows:
  • Step 200 The terminal device obtains the medical image to be recognized.
  • Medical imaging is the physical reproduction of human visual perception. Medical images can be acquired by optical devices, such as cameras, mirrors, telescopes, and microscopes; they can also be created manually, such as painting images by hand. Medical images can be recorded and stored on paper media, film, and other media sensitive to optical signals. With the development of digital acquisition technology and signal processing theory, more and more medical images are stored in digital form.
  • the medical image may be a taken medical picture, for example, the internal body acquired through an endoscope (neuroscopy, urethral cystoscopy, resectoscope, laparoscope, arthroscopy, sinusoscope, and laryngoscope, etc.) image.
  • endoscope neuroscopy, urethral cystoscopy, resectoscope, laparoscope, arthroscopy, sinusoscope, and laryngoscope, etc.
  • the technical solution provided by the present invention can be applied to the recognition of various images.
  • the recognition of the esophageal image is only used as an example for description, and the recognition of other images will not be repeated here.
  • Step 201 The terminal device inputs the medical image into the first recognition model to obtain a lesion recognition result.
  • the terminal device normalizes the medical image to be recognized to a specified size and inputs a pre-trained first recognition model to obtain a lesion recognition result indicating whether the input medical image is a medical image where a lesion has occurred.
  • the designated size of the medical image can be normalized to 224 * 224 pixels.
  • the lesion identification result may include a lesion label, and / or a lesion probability.
  • the lesion label may include a normal image label and a medical image label.
  • the first recognition model when the medical image is discriminated by the first recognition model to generate a lesion recognition result indicating whether the medical image includes a lesion, the first recognition model can use a trained deep learning network to Searching for a lesion feature in the medical image, and generating the lesion recognition result according to the search result.
  • the deep learning network may be a neural network with multiple layers of perceptrons, and may include, but is not limited to, convolutional neural networks, recurrent neural networks, deep belief networks, and so on.
  • the lesion feature is that the deep learning network learns from the first medical image set of the labeled normal organ and the second medical image set of the organ with the lesion during training, and exists in the second medical image set and Image features that do not exist in the first set of medical images.
  • a constraint condition can be set in the objective function when training the deep learning network. For example, the feature response of the learned lesion features in the medical image of normal organs is less than the first The threshold (the first threshold may be a value close to 0), and the characteristic response in the medical image of the diseased organ is distributed in the dimension of the diseased area.
  • the first recognition model is obtained by using DenseNet to train normal image samples and lesion image samples in advance.
  • the terminal device obtains the first recognition error according to the average value of the cross entropy of the lesion recognition result of each medical image and the sum of the corresponding constraint expression values, and the regular value, and optimizes the first recognition model according to the first recognition error.
  • the regular value is obtained through the L2 regular function.
  • the first recognition error is negatively correlated with the above average value and positively correlated with the above regular value.
  • the constrained expression value of the medical image is effective when the result of manual judgment of the medical image is normal.
  • the cross entropy is obtained through the cross entropy expression.
  • Cross entropy is positively correlated with the actual analysis results of medical images and the logarithm of the lesion recognition results.
  • the constraint expression value is obtained through the constraint expression.
  • the constraint expression value is positively correlated with the square of the norm of the feature vector extracted from the medical image, and negatively correlated with the actual analysis result of the medical image.
  • the constraint expression value is non-zero to take effect.
  • the constraint expression value of 0 does not take effect.
  • the regular value is obtained through the L2 regular function.
  • the regular value is positively related to the square of the norm of the model parameter of the first recognition model.
  • the following objective function when the terminal device determines the first recognition error, the following objective function may be used:
  • i is the serial number of the medical image (or medical image sample)
  • n is the total number of medical images (or medical image samples)
  • y is the actual analysis result of the medical image
  • x is the medical image
  • w is the number of the first recognition model Model parameters
  • f represents the first recognition model
  • p is the feature vector extracted from the medical image using the first recognition model
  • r and ⁇ are the set weight values.
  • y i log f (x i ; w) is the cross-entropy expression, Is a constraint expression, It is a L2 regular function.
  • the constraint expression value when the actual analysis result is a lesion, the constraint expression value is 0, that is, it does not take effect.
  • the constraint expression value is The constrained expression is used to make the first medical recognition model's lesion recognition result for normal medical images tend to 0, and conversely, the lesion recognition result for the medical image of the lesion is distributed in the dimension of each lesion area.
  • the first recognition model when training the first recognition model, can be adjusted by the first recognition error obtained by the target function; when the first recognition model is used to recognize and apply medical images, it can be obtained by the target function The first recognition error corrects the first recognition model and determines the accuracy of the recognition.
  • DenseNet-121 in DenseNet is used to train normal image samples and lesion image samples to obtain a first recognition model, and the first recognition model is further optimized through the first recognition error.
  • the normal image sample includes at least: normal image and normal image label.
  • the medical image samples include at least medical images and medical image tags.
  • DenseNet-121 uses the configuration parameters shown in Table 1 for configuration.
  • the DenseNet-121 network structure contains 4 dense blocks (dense Block), growth rate (growth-rate) is set to 12, 3 transition layers (transition layer), the characteristic compression ratio of transition layer is set to 0.5, and finally through the classification layer ( Classification (Layer) outputs the lesion recognition result.
  • FIG. 3a it is a schematic diagram of a principle of a first recognition model.
  • the terminal device inputs the medical image to the first recognition model, and passes through the convolutional layer, dense layer-1, convolutional layer, pooling layer, dense layer-N, convolutional layer, and linear layer in order to output lesion recognition result.
  • FIG. 1 is an example of an esophageal image.
  • the terminal device inputs the esophageal image (a) and the esophageal image (b) in FIG. 3b to the first recognition model, and outputs the esophageal image (a). It is a normal medical image, and the esophageal image (b) is the esophageal image of the lesion.
  • the first recognition model can be used to classify the medical images according to the extracted features of each small area of the medical image and the high-level semantic features of the overall medical image to obtain the lesion recognition result.
  • the medical images can be preliminarily screened through the first recognition model to screen out the medical images with lesions, so that the medical images with lesions can be directly recognized through the subsequent second recognition model, which improves the processing efficiency.
  • Step 202 The terminal device determines whether the lesion recognition result is a medical image where the lesion has occurred, and if so, step 203 is performed, otherwise, step 204 is performed.
  • Step 203 The terminal device inputs the medical image to the second recognition model to obtain the recognition result of the lesion degree.
  • the terminal device recognizing the medical image through the second recognition model specifically includes: dividing the medical image whose lesion recognition result is the lesion into a plurality of image blocks, and separately extracting the feature information of each image block, and determining each according to the extracted feature information The recognition result of the lesion degree of the image block.
  • the medical image when the lesion recognition result indicates that the medical image includes a lesion, the medical image can be recognized by the second recognition model to generate a lesion degree recognition result indicating the degree of the lesion.
  • the second recognition model may use the trained second deep learning network to search the medical image for the lesion degree feature corresponding to the first lesion degree, and generate the lesion degree recognition result according to the search result.
  • the second deep learning network may be a neural network with multiple layers of perceptrons, and may include, but is not limited to, a convolutional neural network, a recurrent neural network, a deep belief network, and so on.
  • the lesion degree feature is that the second deep learning network is trained from a set of labeled third medical images without organs with the first degree of disease and a fourth set of medical images with organs with the first degree of disease during training
  • the image features learned in which exist in the fourth medical image set and do not exist in the third medical image set.
  • the learned lesion features of the first lesion degree are higher than those of the first lesion
  • the characteristic response in the medical image of the diseased organ with a low degree is less than the first threshold (the first threshold may be a value close to 0), while the characteristic response in the medical image of the diseased organ with the first degree of disease is distributed in the lesion area Dimensionally.
  • the second recognition model is obtained based on convolutional neural network training, and in the process of training and recognition application, the second recognition error is used to optimize the second recognition model.
  • the lesion degree recognition result is used to indicate the degree to which the medical image includes the lesion.
  • the recognition result of the lesion degree includes: the recognition result of the lesion degree of each image block and the area information in the medical image, and / or the lesion degree indication image after the corresponding indication information is set in the corresponding area according to the recognition result of each image block .
  • the lesion degree recognition result also includes the lesion degree label of the medical image.
  • the lesion degree label is: the first recognition result of the image block with the most serious lesion among the multiple image blocks segmented from the medical image; or, based on the feature information of all image blocks
  • the second recognition result of the determined degree of lesion of the medical image or a comprehensive result determined according to the first recognition result and the second recognition result.
  • the disease severity label can be inflammation, early cancer, intermediate cancer, and advanced cancer, and can also be the probability of canceration.
  • the area information may be information such as coordinates or location names of image blocks.
  • the degree of lesion indication indicates that each area in the image can be set with different colors or patterns for indication according to different recognition results.
  • the terminal device may adopt the following methods when determining the recognition result of the lesion degree of each image block through the second recognition model:
  • the first method is: first, according to the feature information of each image block, determine the canceration probability of each image block, and obtain the association relationship between the canceration probability range and the lesion degree label; then, according to the association relationship, determine The lesion degree label corresponding to the canceration probability range to which the highest canceration probability belongs, and the obtained lesion degree label is determined as the first recognition result; then, according to the canceration probability of each image block, the corresponding indication information is set in the corresponding area to obtain the lesion The degree indication image; finally, the canceration probability and area information of each image block, the lesion degree indication image and the first recognition result are determined as the disease degree recognition result of the medical image.
  • FIG. 3c is a schematic diagram of the principle of a second recognition model.
  • Figure 3d is a schematic diagram of an image block.
  • the terminal device inputs the medical image to the second recognition model, divides the medical image into a plurality of image blocks shown in FIG. 3d through the second recognition model, and separately extracts the feature information of each image block, and According to the extracted feature information, estimate the canceration probability of each image block separately; then, the terminal device sorts the canceration probability of each image block, and tags the lesion degree label of the image block with the highest canceration probability as the lesion of the medical image Degree label, and then obtain the canceration degree recognition result.
  • the second method is: according to the feature information of each image block, determine the canceration probability of the medical image, and obtain the association between the canceration probability range and the lesion degree label; then, according to the association, determine the canceration to which the canceration probability belongs The lesion degree label corresponding to the probability range, and the obtained lesion degree label is determined as the second recognition result; then, according to the canceration probability of each image block, corresponding indication information is set in the corresponding region to obtain the lesion degree indication image; finally, the The canceration probability and area information of each image block, the lesion degree indication image and the second recognition result are determined as the recognition result of the disease degree of the medical image.
  • the disease degree label can be determined through the feature information of the overall medical image.
  • the third method is to obtain the first recognition result and the second recognition result through the first method and the second method described above, respectively, and determine the recognition result with the most serious lesion degree among the first recognition result and the second recognition result as The comprehensive result, and the canceration probability and area information of each image block, the lesion degree indication image, and the comprehensive result are determined as the lesion degree recognition result of the medical image.
  • FIG. 3e is an example image 2 of an esophageal image.
  • (c) in FIG. 3e is an example cancer image of an esophageal image
  • (d) is an example inflammation image of an esophageal image.
  • the terminal device inputs the medical images of (c) and (d) in FIG. 3e to the second recognition model, and the recognition result of the degree of lesion corresponding to (c) in FIG. 3e is a medical image with cancer, and the corresponding image in (d) in FIG. 3e
  • the result of recognition of the degree of lesion is a medical image of inflammation.
  • FIG. 3f is an example of an indication image of the degree of lesion of an esophageal image.
  • the terminal device inputs the left image of (e), the left image of (f), the left image of (g), and the left image of (h) of FIG. 3f into the second recognition model, and the output recognition results of the lesion degree are:
  • the right picture of (e) of FIG. 3f is the lesion degree indication image of the left picture of FIG. 3f (e); the right picture of (f) of FIG. 3f is the lesion degree indication image of the left picture of (f) of FIG. 3f;
  • the right picture of (g) of FIG. 3f is the lesion degree indication image of the left picture of (g) of FIG. 3f;
  • the right picture of (h) of FIG. 3f is the disease degree indication image of the left picture of (h) of FIG. 3f;
  • the user can determine the lesion degree of the medical image through the lesion degree label, and instruct the image according to the lesion degree to determine the judgment basis of the lesion degree.
  • the second recognition model is obtained based on convolutional neural network training, and the second recognition error is used to optimize the second recognition model.
  • the second recognition error is obtained by using a specified loss function, and the cross entropy in the loss function is determined based on the recognition result of the degree of lesion of the image block with the highest degree of lesion in each image block of the medical image.
  • the loss function may be a maximum pooling loss function (max pooling loss), a label allocation loss function (labed assignment loss), and a sparse loss function (sparsity loss).
  • the network structure shown in Table 2 contains two branches.
  • the branch pointed by the right arrow is the configuration parameter for medical image recognition using the first method above
  • the branch pointed by the left arrow is the configuration for medical image recognition by the second method. parameter.
  • the common layer outputs the feature information extracted for each image block
  • the branch pointed by the arrow on the right, through 1 * 1 convolution obtains the canceration probability of each image block according to the feature information of each image block output by the common layer, and then based on For each canceration probability, a lesion degree recognition result is obtained.
  • the branch pointed by the arrow on the left, through the overall image convolution obtains the canceration probability of the medical image based on the feature information of each image block output by the common layer, and then obtains the lesion degree recognition result.
  • the comprehensive result may be determined according to the first recognition result output by the branch pointed by the left arrow and the second recognition result output by the branch pointed by the right arrow.
  • the model parameters of the common layer are updated according to the second recognition error determined by the synthesis result, and the branches pointed by the left and right arrows are based on the corresponding recognition
  • the identification error determined by the results is optimized for the model parameters.
  • Step 204 The terminal device outputs a lesion recognition result indicating that the medical image is normal.
  • step 202 is used to make a preliminary judgment on the medical image to determine whether the medical image has a lesion.
  • step 201 and step 202 may not be executed, that is, the canceration recognition of the medical image is directly performed through the subsequent step 203.
  • the specific implementation process of image recognition can also be:
  • the terminal device obtains the medical image to be recognized.
  • the terminal device inputs the medical image to the second recognition model to obtain the recognition result of the lesion degree.
  • the terminal device inputs the medical image to the second recognition model to obtain the recognition result of the lesion degree.
  • the medical system 300 includes an image acquisition device 301, an image recognition device 302, and a display device 303.
  • the image acquisition device 301 is used to shoot a patient's lesion (eg, inside the body) and other locations through a built-in camera or endoscope, etc., to collect medical images of the patient.
  • the endoscope may be a neuroscope, a urethral cystoscope, a resectoscope, a laparoscope, an arthroscope, a sinusoscope, and a laryngoscope.
  • the image recognition device 302 is used to obtain the medical image collected by the image collection device 301, and determine whether the medical image is a diseased medical image through the first recognition model, generate a disease recognition result, and use the second recognition model to treat the diseased medical
  • the images are further identified to obtain a lesion degree recognition result to indicate the extent to which the medical image includes lesions. Further, the image recognition device 302 may also directly use the second recognition model to recognize the medical image where the lesion has occurred, and obtain a recognition result of the degree of the lesion.
  • the display device 303 is used to obtain the lesion recognition result or the lesion degree recognition result output by the image recognition device 302, and present the lesion recognition result or the lesion degree recognition result to the user.
  • the medical system 300 can collect the medical images of the patient through the image collection device 301, and recognize the collected medical images through the image recognition device 302 to obtain the lesion recognition result or the lesion degree recognition result, and present the lesion to the user through the display device 303 Identification result or lesion degree identification result.
  • an embodiment of the present invention also provides a device for image recognition. Since the principle of the above device and equipment to solve the problem is similar to a method for image recognition, the implementation of the above device can be referred to the method implementation. The repetition is not repeated here.
  • FIG. 4a it is a schematic structural diagram 1 of an image recognition device according to an embodiment of the present invention, including:
  • the obtaining unit 410 is used to obtain medical images to be recognized
  • the discriminating unit 411 is used to discriminate the medical image through the first recognition model to generate a lesion recognition result, and the first recognition model is used to discriminate whether the medical image has a lesion;
  • the recognition unit 412 is used for recognizing the medical image of the lesion through the second recognition model to generate a recognition result of the degree of lesion.
  • the second recognition model is used for recognizing the extent to which the medical image includes the lesion.
  • the discriminating unit 411 may use the trained deep learning network in the first recognition model to search for lesion features in the medical image, and generate the lesion recognition result according to the search result.
  • the lesion feature is that the deep learning network learns from the first medical image set of the labeled normal organ and the second medical image set of the organ with the lesion during training and exists in the second medical Image features in the image collection and not in the first medical image collection.
  • the recognition unit 412 may use the second deep learning network trained in the second recognition model to search for the lesion degree feature corresponding to the first lesion degree in the medical image, and generate the lesion degree according to the search result Recognize the results.
  • the lesion degree feature is the third medical image set of the second deep learning network from the labeled organs without the first lesion degree and the first Image features learned from the four medical image collections that exist in the fourth medical image collection and do not exist in the third medical image collection.
  • the recognition unit 412 is specifically configured to: divide the medical image whose lesion recognition result is a lesion into multiple image blocks;
  • the recognition result of the lesion degree includes: the recognition result of the lesion degree of each image block and the area information in the medical image, and / or, after the corresponding indication information is set in the corresponding area according to the recognition result of the lesion degree of each image block
  • the degree of lesion indicates the image.
  • the recognition result of the lesion degree further includes a lesion degree label of the medical image, and the lesion degree label of the medical image is:
  • the second recognition result of the degree of lesion of the medical image determined according to the feature information of all image blocks; or,
  • the integrated result determined according to the first recognition result and the second recognition result The integrated result determined according to the first recognition result and the second recognition result.
  • the second recognition model is obtained based on convolutional neural network training, and the second recognition error is used to optimize the second recognition model
  • the second recognition error is obtained by using a specified loss function, and the cross entropy in the loss function is determined based on the recognition result of the lesion degree of the image block with the highest degree of lesion among each image block of the medical image.
  • the first recognition model is obtained based on DenseNet training, and the first recognition error is used to optimize the first recognition model;
  • the first recognition error is obtained based on the average value of the cross entropy of each medical image's lesion recognition result and the sum of the corresponding constraint expression values, and the regular value, where the constraint expression value of the medical image is preset Constrained expressions are obtained.
  • the constrained expressions are used to obtain valid constrained expression values when the artificial discrimination results of medical images are normal.
  • FIG. 4b it is a schematic structural diagram 2 of an image recognition device according to an embodiment of the present invention, including:
  • the input unit 420 is used to obtain the medical image to be recognized, and to recognize the medical image to be recognized through the recognition model;
  • the obtaining unit 421 is used to obtain a lesion degree recognition result output after the recognition model recognizes the medical image to be recognized, and the lesion degree recognition result includes: among multiple image blocks included in the medical image, the lesion degree recognition result of each image block And the area information in the medical image, and / or the pathological degree indication image after corresponding indication information is set in the corresponding area according to the identification result of the pathological degree of each image block;
  • the recognition model to recognize medical images specifically includes: dividing the medical image into multiple image blocks, and for each image block, extracting the feature information of the image block and determining the degree of lesion recognition of the image block according to the extracted feature information result.
  • the first recognition model is used to determine whether the medical image is a diseased medical image, and then the second recognition model is used The medical image is further identified to obtain a lesion degree recognition result to indicate the degree of the lesion included in the medical image. This does not require manual analysis and custom feature extraction schemes, which improves the efficiency and accuracy of medical image recognition.
  • an embodiment of the present invention also provides a terminal device 500.
  • the terminal device 500 is used to implement the methods described in the foregoing method embodiments.
  • the terminal shown in FIG. 2 is implemented.
  • the device 500 may include a memory 501, a processor 502, an input unit 503, and a display panel 504.
  • the memory 501 is used to store a computer program executed by the processor 502.
  • the memory 501 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, application programs required for at least one function, etc .; the storage data area may store data created according to the use of the terminal device 500 and the like.
  • the processor 502 may be a central processing unit (CPU) or a digital processing unit.
  • the input unit 503 may be used to obtain user instructions input by the user.
  • the display panel 504 is used to display information input by the user or provided to the user. In the embodiment of the present invention, the display panel 504 is mainly used to display the display interface of each application program in the terminal device and the control entity displayed in each display interface .
  • the display panel 504 may be configured in the form of a liquid crystal display (LCD) or an OLED (organic light-emitting diode).
  • LCD liquid crystal display
  • OLED organic light-emitting diode
  • a specific connection medium between the above-mentioned memory 501, processor 502, input unit 503, and display panel 504 is not limited.
  • the memory 501, the processor 502, the input unit 503, and the display panel 504 are connected by a bus 505.
  • the bus 505 is indicated by a thick line in FIG. 5, and the connection between other components is only It is for illustrative purposes, not for limitation.
  • the bus 505 can be divided into an address bus, a data bus, and a control bus. For ease of representation, only a thick line is used in FIG. 5, but it does not mean that there is only one bus or one type of bus.
  • the memory 501 may be volatile memory (volatile memory), such as random-access memory (RAM); the memory 501 may also be non-volatile memory (non-volatile memory), such as read-only memory, flash memory
  • volatile memory volatile memory
  • non-volatile memory non-volatile memory
  • read-only memory flash memory
  • flash memory flash memory
  • HDD hard disk
  • SSD solid-state drive
  • the memory 501 can be used to carry or store the desired program code in the form of instructions or data structures and can be used by Any other media accessed by the computer, but not limited to this.
  • the memory 501 may be a combination of the aforementioned memories.
  • the processor 502 configured to implement the embodiment shown in FIG. 2, includes:
  • the processor 502 is configured to call a computer program stored in the memory 501 to execute the embodiment shown in FIG. 2.
  • An embodiment of the present invention also provides a computer-readable storage medium that stores computer-executable instructions required to execute the above-mentioned processor, which includes programs required to execute the above-mentioned processor.
  • various aspects of an image recognition method provided by the present invention may also be implemented in the form of a program product, which includes program code.
  • the program product runs on a terminal device, the program code is used
  • the terminal device may execute the embodiment shown in FIG. 2.
  • the program product may use any combination of one or more readable media.
  • the readable medium may be a readable signal medium or a readable storage medium.
  • the readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of readable storage media (non-exhaustive list) include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.
  • the program product for image recognition may use a portable compact disk read-only memory (CD-ROM) and include program code, and may run on a computing device.
  • CD-ROM portable compact disk read-only memory
  • the program product of the present invention is not limited to this.
  • the readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.
  • the readable signal medium may include a data signal that is propagated in baseband or as part of a carrier wave, in which readable program code is carried. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above.
  • the readable signal medium may also be any readable medium other than a readable storage medium, and the readable medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.
  • the program code contained on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
  • the program code for performing the operations of the present invention can be written in any combination of one or more programming languages.
  • the programming language includes entity-oriented programming languages such as Java, C ++, etc., as well as conventional procedural programming Language-such as "C" language or similar programming language.
  • the program code may be executed entirely on the user's computing device, partly on the user's device, as an independent software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server On the implementation.
  • the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (eg, using Internet services Provide entropy to connect via the Internet).
  • LAN local area network
  • WAN wide area network
  • the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.
  • computer usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions can be provided to the processor of a general-purpose computer, special-purpose computer, embedded processing machine, or other programmable data processing device to produce a machine that enables the generation of instructions executed by the processor of the computer or other programmable data processing device
  • These computer program instructions may also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, the instructions The device implements the functions specified in one block or multiple blocks of the flowchart one flow or multiple flows and / or block diagrams.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operating steps are performed on the computer or other programmable device to produce computer-implemented processing, which is executed on the computer or other programmable device
  • the instructions provide steps for implementing the functions specified in one block or multiple blocks of the flowchart one flow or multiple flows and / or block diagrams.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)
  • Endoscopes (AREA)

Abstract

本发明属于图像处理技术领域，公开了一种影像识别的方法、装置、终端设备和医疗系统，本发明公开的一种影像识别的方法包括，通过第一识别模型判别医疗影像是否为包括病变的医疗影像，然后，通过第二识别模型对包括病变的医疗影像进行进一步识别，获得病变程度识别结果，以指示医疗影像所包括的病变的程度。这不需要人工进行分析以及定制特征抽取方案，提高了医疗影像识别的效率和精确度。

Description

一种影像识别的方法、装置、终端设备和医疗系统
本申请要求于2018年10月30日提交中国专利局、申请号为201811278418.3、发明名称为“一种影像识别的方法、装置、终端设备和医疗系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及图像处理技术领域，尤其涉及一种影像识别的方法、装置、终端设备和医疗系统。
背景
随着计算机技术的发展，计算机技术逐渐应用于医疗影像(如，内窥镜食管影像)分析，以辅助医生进行医疗诊断。例如，终端设备通过计算机技术可以根据患者的食管影像，判断患者发生癌变的概率。这样，医生可以根据输出结果，进行进一步的诊断分析，这提高了医疗诊断的精确度和效率。现有技术下，终端设备通过影像进行疾病分析时，通常采用传统机器学习方法(如，小波算子)基于预先提取的特征信息对影像进行疾病分类，获得疾病分类结果。
技术内容
本发明实施例提供一种影像识别的方法、装置、终端设备和医疗系统，用以在根据医疗影像进行疾病识别时，提高识别的效率和精确度。
各实施例的影像识别的方法可以包括：
获取待识别的医疗影像；
通过第一识别模型对医疗影像进行判别，生成用于指示所述医疗影像是否包括病变的病变识别结果；
当所述病变识别结果指示所述医疗影像包括病变时，通过第二识别模型对所述医疗影像进行识别，生成用于指示所述病变的程度的病变程度识别结果。
这样，避免了人工进行特征分析和方案定制的繁琐步骤，提高了医疗影像识别的精确度和效率。
各实施例的影像识别的装置可以包括：
获取单元，用于获取待识别的医疗影像；
判别单元，用于通过第一识别模型对医疗影像进行判别，生成用于指示所述医疗影像是否包括病变的病变识别结果；
识别单元，用于当所述病变识别结果指示所述医疗影像包括病变时，通过第二识别模型对所述医疗影像进行识别，生成用于指示所述病变的程度的病变程度识别结果。
各实施例的另一医疗影像的识别方法可以包括：
获取待识别的医疗影像，并通过识别模型对待识别的医疗影像进行识别；
获得识别模型对待识别的医疗影像进行识别后输出的病变程度识别结果，且病变程度识别结果包括：医疗影像包括的多个影像块中，每个影像块的病变程度识别结果及其在医疗影像中的区域信息，和/或根据每个影像块的病变程度识别结果在其对应区域设置了相应指示信息后的病变程度指示影像；
其中，识别模型对医疗影像进行识别具体包括：将医疗影像分割为多个影像块，并针对每个影像块，提取该影像块的特征信息并根据提取的特征信息确定该影像块的病变程度识别结果。
各实施例的另一影像识别的装置可以包括：
输入单元，用于获取待识别的医疗影像，并通过识别模型对待识别的医疗影像进行识别；
获得单元，用于获得识别模型对待识别的医疗影像进行识别后输出的病变程度识别结果，且病变程度识别结果包括：医疗影像包括的多个影像块中，每个影像块的病变程度识别结果及其在医疗影像中的区域信息，和/或根据每个影像块的病变程度识别结果在其对应区域设置了相应指示信息后的病变程度指示影像；
其中，识别模型对医疗影像进行识别具体包括：将医疗影像分割为多个影像块，并针对每个影像块，提取该影像块的特征信息并根据提取的特征信息确定该影像块的病变程度识别结果。
各实施例的终端设备可以包括至少一个处理单元、以及至少一个存储单元，其中，存储单元存储有计算机程序，当程序被处理单元执行时，使得处理单元执行上述任意一种影像识别的方法的步骤。
各实施例的医疗系统可以包括影像采集装置和影像识别装置，其中，
影像采集装置，用于采集病人的医疗影像；
影像识别装置，用于获取影像采集装置采集的医疗影像，并通过第一识别模型对医疗影像进行判别，生成用于指示所述医疗影像是否包括病变的病变识别结果，以及当所述病变识别结果指示所述医疗影像包括病变时，通过第二识别模型对所述医疗影像进行识别，生成用于指示所述病变的程度的病变程度识别结果；
显示装置，用于呈现病变程度识别结果。
各实施例的另一医疗系统可以包括影像采集装置和影像识别装置，其中，
影像采集装置，用于采集病人的医疗影像；
影像识别装置，用于获取影像采集装置采集的医疗影像，并通过第二识别模型对包括病变的医疗影像进行识别，生成病变程度识别结果；其中，第二识别模型用于识别医疗影像所包括的病变的程度；
显示装置，用于呈现病变程度识别结果。
各实施例的计算机可读存储介质存储有计算机程序，当所述程序被一个或多个处理器执行时，可以使所述一个或多个处理器执行以上任意一种影像识别的方法的 步骤。
本发明实施例的一种影像识别的方法、装置、终端设备和医疗系统中，通过第一识别模型判别医疗影像是否为发生病变的医疗影像，然后，通过第二识别模型对发生病变的医疗影像进行进一步识别，获得病变程度识别结果，以指示医疗影像包括病变的程度。这不需要人工进行分析以及定制特征抽取方案，提高了医疗影像识别的效率和精确度。
本发明的其它特征和优点将在随后的说明书中阐述，并且，部分地从说明书中变得显而易见，或者通过实施本发明而了解。本发明的目的和其他优点可通过在所写的说明书、权利要求书、以及附图中所特别指出的结构来实现和获得。
附图简要说明
此处所说明的附图用来提供对本发明的进一步理解，构成本发明的一部分，本发明的示意性实施例及其说明用于解释本发明，并不构成对本发明的不当限定。在附图中：
图1为本发明实施方式中一种终端设备的结构示意图；
图2为本发明实施方式中一种影像识别的方法的实施流程图；
图3a为本发明实施方式中一种第一识别模型的原理示意图；
图3b为本发明实施方式中一种食管影像的示例图一；
图3c为本发明实施方式中一种第二识别模型的原理示意图；
图3d为本发明实施方式中一种影像块示意图；
图3e为本发明实施方式中一种食管影像的示例图二；
图3f为本发明实施方式中一种食管影像的病变程度指示影像示例图；
图3g为本发明实施方式中一种医疗系统的结构示意图；
图4a为本发明实施方式中一种影像识别的装置的结构示意图一；
图4b为本发明实施方式中一种影像识别的装置的结构示意图二；
图5为本发明实施方式中终端设备结构示意图。
实施本发明的方式
为了在医疗影像进行疾病识别时，提高识别的效率和精确度，本发明实施例提供了一种影像识别的方法、装置、终端设备和医疗系统。
首先，对本发明实施例中涉及的部分用语进行说明，以便于本领域技术人员理解。
1、终端设备：可以安装各类应用程序，并且能够将已安装的应用程序中提供的实体进行显示的设备，该电子设备可以是移动的，也可以是固定的。例如，手机、平板电脑、车载设备、个人数字助理(personal digital assistant，PDA)或其它能够实现上述功能的电子设备等。
2、医疗影像：是人对视觉感知的物质再现，可以由光学设备获取，如照相机、镜子、望远镜及显微镜等；也可以人为创作，如，手工绘画图像等。病理可以记录、 保存在纸质媒介、胶片等等对光信号敏感的介质上。随着数字采集技术和信号处理理论的发展，越来越多的医疗影像以数字形式存储。
3、卷积神经网络(Convolutional Neural Network，CNN)：在本质上是一种输入到输出的映射，它能够学习大量的输入与输出之间的映射关系，而不需要任何输入和输出之间的精确的数学表达式，只要用已知的模式对卷积网络加以训练，网络就具有输入输出对之间的映射能力。
4、稠密卷积网络(DensNet)：每一层均以前馈的形式与其他任一层连接的卷积神经网络，即任一层不仅与相邻层有连接，而且与它的随后的所有层都有直接连接。
5、第一识别模型：采用DenseNet训练对正常影像样本和病变影像样本进行训练获得的模型，用于判别医疗影像是否发生病变。
本文中，医疗影像是否发生病变，也称为医疗影像是否包括病变，是指医疗影像是否显示其中的器官发生了病变，也即，医疗影像是否包括器官病变对应的影像内容。
6、第二识别模型：采用卷积神经网络对各种不同病变程度的医疗影像样本训练获得的模型，用于识别医疗影像发生病变的程度。
7、病变识别结果：用于指示输入的医疗影像是否为发生病变的医疗影像。
8、病变程度识别结果：用于指示医疗影像发生病变的程度。
另外，本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，在不做特别说明的情况下，一般表示前后关联对象是一种“或”的关系。
近年来，新发肿瘤病例中，食管癌排名第3名，其中，我国有47.7万，占全球百分之五十。食管癌是上消化道部位的恶性肿瘤。不同时期的食管癌患者，治疗过程有很大差异。早期患者主要通过内镜微创治疗，术后3～5天就可以出院，治疗费用低并发症少，90％可以治愈。而中晚期食管癌患者主要通过开胸/腹/颈“三切口”手术进行治疗，这种治疗方式的创伤大、治疗费用高并且疗效不理想，治愈率不足40％。
因此，若在患者患有早期癌症时，及时确诊以及治疗，会极大地减少患者的痛苦和负担。医生通常通过内窥镜获得的医疗影像进行癌变分析，以预测患者是否患有早期癌症。但是，由于需要通过专业医生对医疗影像进行辛苦地分析，不同医生的分析水平也各不相同，因此，早期癌症的发现率非常低，不足百分之五十。
目前，随着计算机技术的发展，计算机技术(如，传统机器学习方法)逐渐开始应用到癌变分析中，以辅助医生进行医疗诊断。但是，采用传统机器学习方法对医疗影像进行识别时，需要专业人员对医疗影像进行深刻地特征分析，并根据分析结果定制特征抽取方案，这会耗费专业人员的大量时间，降低了癌变识别效率。此外，抽取的特征更加偏向于某一类通用的特性，覆盖面较小，鲁棒性差，难以在实 际应用中推广使用，获得的癌变识别的精确度较低。
考虑到正常食管影像一般表面粘膜光滑，而病变食管影像具有明显的特征，如，凸起和糜烂等。但是，实际应用中，由于患者的年龄，地域或者受检查时的状态(如，食管未冲洗干净，包含异物)等因素影响，正常食管影像和病变食管影像差别可能在于图像中一个小块区域(如，颜色深浅，皮肤粗糙程度)，也可能在于图像整体光滑度变化，对食管影像进行分类时，既需要提取小区域的特征，还需要提取整体食管影像的高级语义特征。
本申请利用人工智能技术从医疗影像中识别病变，并识别病变的程度。
人工智能(AI)系统是指表现出智能行为的计算机系统。AI系统的功能包括学习、维护大量知识库、执行推理、应用分析能力、辨别事实之间的关系、与他人交流思想、理解他人的交流、以及感知和理解状况，等。
与基于规则的智能系统不同，AI系统可以通过自身学习和判断使机器不断进步。AI系统通过查找数据中先前未知的模式来创建新知识，通过学习数据模式来驱动解决方案。在不断使用中，AI系统的识别率可以提高，并且可以更准确地理解用户的品味。因此，现有的基于规则的智能系统逐渐被AI系统取代。
AI系统中，通常使用神经网络。神经网络是设计、构造和配置为模拟人类神经系统的计算机系统。神经网络体系结构由输入层、输出层和一个或多个隐藏层组成。输入层将数据输入神经网络。输出层产生猜测结果。隐藏层协助信息传播。这些系统通过研究示例来学习处理任务或做出决定。神经网络或人工神经网络基于称为神经元或人工神经元的连接单元的集合。神经元之间的每个连接(突触)可以将信号传输到另一个神经元。接收(突触后)神经元可以处理信号，然后发信号通知连接到神经元的下游神经元。神经元可具有状态，通常由实数表示，通常在0和1之间。神经元和突触也可具有随着学习进行而变化的权重，权重用于增加或减少其向下游发送的信号的强度。此外，神经元可以具有阈值，使得仅当聚合信号低于(或高于)该水平时才发送下游信号。通常，神经元按层组织。不同的层可以对其输入执行不同类型的转换。信号可能在多次遍历各层之后从第一(输入)层移动到最后(输出)层。在具有多个隐藏层的人工网络中，初始层可以检测基元(例如，眼睛中的瞳孔，虹膜，睫毛等)，其输出被向前馈到执行更抽象概括的更深层(例如，眼睛，嘴巴)。依此类推，直到最后的层执行复杂的对象识别(例如，脸部)。
神经网络使用诸如一系列数据点的数据进行训练。神经网络猜测应该给出哪种响应，并且将猜测与每个数据点的正确的“最佳”猜测进行比较。如果发生错误，则调整神经元，并且该过程重复进行。
本申请的一些实施例可以使用神经网络中的卷积神经网络对医学影像进行处理和识别。卷积神经网络(也称作ConvNets或CNN)是神经网络的一种，它在图像识别和分类等领域已被证明非常有效。随着卷积神经网络(CNNs)的迅速发展，学术界涌现出一大批非常高效的模型，如GoogleNet、VGGNet、ResNet等，在各 种计算机视觉任务上均崭露头角。但随着网络层数的加深，网络在训练过程中的前传信号和梯度信号在经过很多层之后可能会逐渐消失。先前有一些非常好的工作来解决这一问题。如在Highway和ResNet结构中均提出了一种数据旁路(skip-layer)的技术来使得信号可以在输入层和输出层之间高速流通，核心思想都是创建了一个跨层连接来连通网路中前后层。后来，出现了一种全新的连接模式，为了最大化网络中所有层之间的信息流，将网络中的所有层两两都进行了连接，使得网络中每一层都接受它前面所有层的特征作为输入。由于网络中存在着大量密集的连接，这种网络结构被称为DenseNet。
一些实施例中，采用DenseNet-121对正常影像样本和病变影像样本进行训练，获得第一识别模型，以对医疗影像是否发生病变进行判别。
进一步地，由于发生癌变的医疗影像中至少有一个影像块是有癌变特征的，非癌变的医疗影像中没有任何影像块是有癌变特征的，因此，本申请实施例中，采用卷积神经网络对发生病变的医疗影像进行训练获得第二识别模型，并以医疗影像中发生病变程度最严重的影像块的病变程度确定该医疗影像的癌变识别结果。
上述说明以食道癌为例的，但是对于其他癌症的诊断，也同样存在类似的问题，这样，针对不同的癌症，采用其医疗影像训练出相对应的第一识别模型和第二识别模型后，就可以采用本申请实施例提供的方法，通过第一识别模型对医疗影像进行初步的病变判别，并通过第二识别模型对发生病变的医疗影像进行进一步的病变程度的识别，以确定医疗影像包括病变的程度，极大地提高了癌变识别的效率和精确度。当然，还可以直接采用第二识别模型直接对医疗影像进行识别，确定医疗影像的癌变程度。
本发明实施例提供的一种影像识别的方法，可应用于终端设备中，该终端设备可以为手机、平板电脑、PDA(Personal Digital Assistant，掌上电脑)等。
图1示出了一种终端设备的结构示意图。参阅图1所示，终端设备100包括：处理器110、存储器120、电源130、显示单元140、输入单元150。
处理器110是终端设备100的控制中心，利用各种接口和线路连接各个部件，通过运行或执行存储在存储器120内的软件程序和/或数据，执行终端设备100的各种功能，从而对终端设备进行整体监控。
处理器110可包括一个或多个处理单元。
存储器120可主要包括存储程序区和存储数据区。其中，存储程序区可存储操作系统、各种应用程序等；存储数据区可存储根据终端设备100的使用所创建的数据等。此外，存储器120可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件等。
本领域技术人员可以理解，图1仅仅是终端设备的举例，并不构成对终端设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件。
参阅图2所示，为本发明提供的一种影像识别的方法的实施流程图，该方法的 具体实施流程如下：
步骤200：终端设备获取待识别的医疗影像。
医疗影像是人对视觉感知的物质再现。医疗影像可以由光学设备获取，如照相机、镜子、望远镜及显微镜等；也可以人为创作，如，手工绘画图像等。医疗影像可以记录、保存在纸质媒介、胶片等等对光信号敏感的介质上。随着数字采集技术和信号处理理论的发展，越来越多的医疗影像以数字形式存储。
各实施例中，医疗影像可以为拍摄的医疗图片，如，通过内窥镜(神经镜、尿道膀胱镜、电切镜、腹腔镜、关节镜、鼻窦镜以及喉镜等)获取的身体内部的影像。本发明提供的技术方案可以应用于对各种影像的识别，本发明实施例中仅以对食管影像进行识别为例进行说明，对于其它影像的识别，在此不再赘述。
步骤201：终端设备将医疗影像输入第一识别模型，获得病变识别结果。
终端设备将待识别的医疗影像归一化至指定大小后输入预先训练的第一识别模型，获得用于指示输入的医疗影像是否为发生病变的医疗影像的病变识别结果。
各实施例中，医疗影像的指定大小可以归一化处理为224*224像素。病变识别结果可以包含病变标签，和/或病变概率。病变标签可以包含正常影像标签和医疗影像标签。
各实施例中，通过第一识别模型对所述医疗影像进行判别，生成用于指示所述医疗影像是否包括病变的病变识别结果时，所述第一识别模型可以利用经过训练的深度学习网络在所述医疗影像中搜索病变特征，根据搜索结果生成所述病变识别结果。深度学习网络可以是具有多层感知器的神经网络，可以包括，但不限于，卷积神经网络、循环神经网络、深度置信网络等。
病变特征为所述深度学习网络在训练时从经过标记的正常器官的第一医疗影像集合和发生病变的器官的第二医疗影像集合中学习得到的、存在于所述第二医疗影像集合中且不存在于所述第一医疗影像集合中的图像特征。为了使学习到的病变特征更准确，可在对深度学习网络进行训练时，在其目标函数中设置一约束条件，例如，学习到的病变特征在正常器官的医学影像中的特征响应小于第一阈值(第一阈值可以是接近0的值)，而在病变器官的医学影像中的特征响应分布于病变区域的维度上。
其中，第一识别模型是预先采用DenseNet对正常影像样本和病变影像样本进行训练获得的。终端设备根据各个医疗影像的病变识别结果的交叉熵和相应约束表达式值的加和的平均值，以及正则值获得第一识别误差，并根据第一识别误差对第一识别模型进行优化。正则值是通过L2正则函数获得的。
其中，第一识别误差与上述平均值呈负相关，与上述正则值呈正相关。医疗影像的约束表达值在医疗影像的人工判别结果为正常时生效。交叉熵通过交叉熵表达式获得。交叉熵与医疗影像的实际分析结果，以及病变识别结果的对数均呈正相关。约束表达式值通过约束表达式获得。约束表达式值与医疗影像提取的特征向量的范 数的平方呈正相关，与医疗影像的实际分析结果呈负相关。在医疗影像的实际分析结果为正常时，约束表达式值为非零即可以生效。当医疗影像的实际分析结果为病变时，约束表达式值为0即不生效。正则值是通过L2正则函数获得的。正则值与第一识别模型的模型参数的范数的平方呈正相关。
各实施例中，终端设备确定第一识别误差时，可以采用以下目标函数：
Figure PCTCN2019093602-appb-000001
其中，i为医疗影像(或医疗影像样本)的序号，n为医疗影像(或医疗影像样本)的总数量，y为医疗影像的实际分析结果，x为医疗影像，w为第一识别模型的模型参数，f表示第一识别模型，p为采用第一识别模型对医疗影像提取的特征向量，r和λ为设置的权重值。y ilog f(x i；w)为交叉熵表达式，
Figure PCTCN2019093602-appb-000002
为约束表达式，
Figure PCTCN2019093602-appb-000003
为L2正则函数。
这样，通过上述约束表达式可知，实际分析结果为病变时，约束表达式值为0，即不生效，实际分析结果为正常时，约束表达式值为
Figure PCTCN2019093602-appb-000004
约束表达式用于使得第一识别模型对正常的医疗影像的病变识别结果趋于0，反之，对于病变的医疗影像的病变识别结果分布于各病变区域的维度上。
其中，在对第一识别模型进行训练时，可以通过目标函数获得的第一识别误差，对第一识别模型进行调整；在采用第一识别模型对医疗影像进行识别应用时，可以通过目标函数获得的第一识别误差对第一识别模型进行纠偏，以及确定识别的精确度。
本发明实施例中，采用DenseNet中的DenseNet-121对正常影像样本和病变影像样本进行训练，获得第一识别模型，并通过第一识别误差对第一识别模型进行进一步优化。其中，正常影像样本中至少包括：正常影像以及正常影像标签。同理，医疗影像样本中至少包括医疗影像和医疗影像标签。
表1.
Figure PCTCN2019093602-appb-000005
参阅表1所示，为DenseNet-121的网络结构的一种配置参数表。DenseNet-121采用表1所示的配置参数进行配置。DenseNet-121网络结构包含4个稠密块(dense Block)，增长率(growth-rate)设置为12，3个转化层(transition layer)，transition layer的特征压缩比设置为0.5，最后通过分类层(Classification Layer)输出病变识别结果。
参阅图3a所示，为一种第一识别模型的原理示意图。图3a中，终端设备将医疗影像输入至第一识别模型，依次通过卷积层，稠密层-1，卷积层，池化层，稠密层-N，卷积层以及线性层，输出病变识别结果。
例如，参阅图3b所示，为一种食管影像的示例图一，终端设备分别将图3b中的食管影像(a)和食管影像(b)输入至第一识别模型，输出食管影像(a)为正常的医疗影像，食管影像(b)为发生病变的食管影像。
这样，就可以通过第一识别模型，根据提取的医疗影像的各小区域的特征以及整体医疗影像的高级语义特征，对医疗影像进行分类，获得病变识别结果。通过第一识别模型可以对医疗影像进行初步筛选，筛选出发生病变的医疗影像，从而可以通过后续的第二识别模型直接对发生病变的医疗影像进行识别，提高了处理效率。
步骤202：终端设备判断病变识别结果是否为发生病变的医疗影像，若是，则执行步骤203，否则，执行步骤204。
步骤203：终端设备将医疗影像输入至第二识别模型，获得病变程度识别结果。
终端设备通过第二识别模型对医疗影像进行识别具体包括：将病变识别结果为发生病变的医疗影像分割为多个影像块，并分别提取各个影像块的特征信息，以及根据提取的特征信息确定各个影像块的病变程度识别结果。
各实施例中，当所述病变识别结果指示所述医疗影像包括病变时，可以通过第二识别模型对所述医疗影像进行识别，生成用于指示所述病变的程度的病变程度识别结果。具体地，所述第二识别模型可以利用经过训练的第二深度学习网络在所述医疗影像中搜索第一病变程度对应的病变程度特征，根据搜索结果生成所述病变程度识别结果。第二深度学习网络可以是具有多层感知器的神经网络，可以包括，但不限于，卷积神经网络、循环神经网络、深度置信网络等。
病变程度特征为所述第二深度学习网络在训练时从经过标记的不具有所述第一病变程度的器官的第三医疗影像集合和具有所述第一病变程度的器官的第四医疗影像集合中学习得到的、存在于所述第四医疗影像集合中且不存在于所述第三医疗影像集合中的图像特征。为了使学习到的病变程度特征更准确，可在对第二深度学习网络进行训练时，在其目标函数中设置一约束条件，例如，学习到的第一病变程度的病变特征在比第一病变程度低的病变器官的医学影像中的特征响应小于第一阈值(第一阈值可以是接近0的值)，而在具有第一病变程度的病变器官的医学影像中的特征响应分布于病变区域的维度上。
其中，第二识别模型是基于卷积神经网络训练获得的，并且在训练以及识别应用的过程中，采用第二识别误差对第二识别模型进行优化。
其中，病变程度识别结果用于指示医疗影像包括病变的程度。病变程度识别结果包括：每个影像块的病变程度识别结果以及在医疗影像中的区域信息，和/或，根据每一影像块识别结果在其对应区域设置了相应指示信息后的病变程度指示影像。病变程度识别结果还包括医疗影像的病变程度标签，病变程度标签为：医疗影像分割出的多个影像块中病变程度最严重的影像块的第一识别结果；或者，根据所有影像块的特征信息确定的医疗影像的病变程度的第二识别结果，或者，根据第一识别结果和第二识别结果确定的综合结果。
例如，病变程度标签可以为炎症，早期癌症，中期癌症以及晚期癌症，还可以为癌变概率。区域信息可以为坐标或影像块位置名称等信息。病变程度指示影像中的各个区域可以根据不同识别结果设置不同的颜色或图案以进行指示。
其中，终端设备通过第二识别模型确定各个影像块的病变程度识别结果时，可以采用以下几种方式：
第一种方式为：首先，根据每一影像块的特征信息，分别确定每一影像块的癌变概率，并获取癌变概率范围与病变程度标签之间的关联关系；然后，根据该关联关系，确定最高的癌变概率所属的癌变概率范围对应的病变程度标签，并将获得的病变程度标签确定为第一识别结果；接着，根据每一影像块的癌变概率在其对应区域设置相应指示信息，获得病变程度指示影像；最后，将每一影像块的癌变概率和区域信息，病变程度指示影像以及第一识别结果，确定为所述医疗影像的病变程度识别结果。
由于发生癌变的医疗影像中至少有一个影像块是有癌变特征的，非癌变的医疗影像中没有任何影像块是有癌变特征的，因此，通过第一种方式，直接根据病变程度最严重的影像块，确定病变程度标签。
图3c为一种第二识别模型的原理示意图。图3d为一种影像块示意图。参阅图3c所示，终端设备将医疗影像输入至第二识别模型，通过第二识别模型将医疗影像划分为图3d所示的多个影像块，并分别提取每一影像块的特征信息，以及根据提取的特征信息，分别预估每一影像块的癌变概率；然后，终端设备将各影像块的癌变概率进行排序，并将最高癌变概率的影像块的病变程度标签，确定为医疗影像的病变程度标签，进而获得癌变程度识别结果。
第二种方式为：根据各个影像块的特征信息，确定医疗影像的癌变概率，并获取癌变概率范围与病变程度标签之间的关联关系；然后，根据该关联关系，确定该癌变概率所属的癌变概率范围对应的病变程度标签，并将获得的病变程度标签确定为第二识别结果；接着，根据每一影像块的癌变概率在其对应区域设置相应指示信息，获得病变程度指示影像；最后，将每一影像块的癌变概率和区域信息，病变程度指示影像以及第二识别结果，确定为所述医疗影像的病变程度识别结果。
这样，就可以通过整体的医疗影像的特征信息，确定病变程度标签。
第三种方式为：通过上述第一种方式和第二种方式分别获得第一识别结果和第二识别结果，并将第一识别结果和第二识别结果中病变程度最严重的识别结果确定为综合结果，以及将每一影像块的癌变概率和区域信息，病变程度指示影像以及综合结果，确定为所述医疗影像的病变程度识别结果。
例如，图3e为一种食管影像的示例图二，参阅图3e所示，图3e中(c)为一种食管影像的癌症示例图，(d)为一种食管影像的炎症示例图。终端设备分别将图3e中(c)和(d)医疗影像输入至第二识别模型，图3e中(c)对应的病变程度识别结果为发生癌变的医疗影像，图3e中(d)对应的病变程度识别结果为发生炎症的医疗影像。
又例如，图3f为一种食管影像的病变程度指示影像示例图。终端设备分别将图3f的(e)的左图、(f)的左图、(g)的左图以及(h)的左图输入第二识别模型， 则输出的病变程度识别结果分别为：图3f的(e)的右图为图3f的(e)的左图的病变程度指示影像；图3f的(f)的右图为图3f的(f)的左图的病变程度指示影像；图3f的(g)的右图为图3f的(g)的左图的病变程度指示影像；图3f的(h)的右图为图3f的(h)的左图的病变程度指示影像；
这样，用户就可以通过病变程度标签，确定医疗影像的病变程度，根据病变程度指示影像，确定病变程度的判断依据。
其中，第二识别模型是基于卷积神经网络训练获得的，并采用第二识别误差对第二识别模型进行优化。第二识别误差是采用指定的损失函数获得的，损失函数中的交叉熵是基于医疗影像的各个影像块中病变程度最高的影像块的病变程度识别结果确定的。
各实施例中，损失函数可以为最大池损失函数(max pooling loss)，标记分配损失函数(labed assign loss)，稀疏损失函数(sparsity loss)。
表2.
Figure PCTCN2019093602-appb-000006
参阅表2所示，第二识别模型的网络结构的一种配置参数表。表2所示的网络结构中包含两个分支，右边箭头指向的分支为采用上述第一种方式进行医疗影像识别的配置参数，左边箭头指向的分支为采用第二种方式进行医疗影像识别的配置参数。共有层中输出针对每一影像块提取的特征信息，右边箭头指向的分支，通过1*1卷积，根据共有层输出的各影像块的特征信息，获得每一影像块的癌变概率，进而根据各癌变概率，获得病变程度识别结果，左边箭头指向的分支，通过整体影像卷积，根据共有层输出的各影像块的特征信息，获得医疗影像的癌变概率，进而获得病变程度识别结果。
进一步地，还可以根据左边箭头指向的分支输出的第一识别结果和右边箭头指向的分支输出的第二识别结果，确定综合结果。其中，根据第二识别误差对第二识别模型优化时，对于共有层，根据综合结果确定的第二识别误差，对共有层的模型参数进行更新，对于左右箭头指向的分支，分别基于相应的识别结果确定的识别误 差，进行模型参数优化。
步骤204：终端设备输出指示医疗影像为正常的病变识别结果。
进一步地，步骤202用于对医疗影像进行初步判别，以确定医疗影像是否发生病变。各实施例中，步骤201和步骤202也可以不执行，即直接通过后续的步骤203对医疗影像进行癌变识别。
影像识别的具体实施流程还可以为：
首先，终端设备获取待识别的医疗影像。
然后，终端设备将医疗影像输入至第二识别模型，获得病变程度识别结果。具体实施步骤，参见上述步骤203。
这样，可以减少识别的复杂步骤。
参阅图3g所示，为一种医疗系统的结构示意图。医疗系统300包括影像采集装置301，影像识别装置302，以及显示装置303。
影像采集装置301，用于通过内置的照相机或内窥镜等对病人的病灶(如，身体内部)等位置进行拍摄，以采集病人的医疗影像。内窥镜可以为神经镜、尿道膀胱镜、电切镜、腹腔镜、关节镜、鼻窦镜以及喉镜等。
影像识别装置302，用于获取影像采集装置301采集的医疗影像，并通过第一识别模型判别医疗影像是否为发生病变的医疗影像，生成病变识别结果，以及通过第二识别模型对发生病变的医疗影像进行进一步识别，获得病变程度识别结果，以指示医疗影像包括病变的程度。进一步地，影像识别装置302也可以直接采用第二识别模型对发生病变的医疗影像进行识别，获得病变程度识别结果。
显示装置303，用于获取影像识别装置302输出的病变识别结果或病变程度识别结果，并向用户呈现病变识别结果或病变程度识别结果。
这样，医疗系统300可以通过影像采集装置301采集病人的医疗影像，并通过影像识别装置302对采集的医疗影像进行识别，获得病变识别结果或病变程度识别结果，以及通过显示装置303向用户呈现病变识别结果或病变程度识别结果。
基于同一发明构思，本发明实施例中还提供了一种影像识别的装置，由于上述装置及设备解决问题的原理与一种影像识别的方法相似，因此，上述装置的实施可以参见方法的实施，重复之处不再赘述。
如图4a所示，其为本发明实施例提供的一种影像识别的装置的结构示意图一，包括：
获取单元410，用于获取待识别的医疗影像；
判别单元411，用于通过第一识别模型对医疗影像进行判别，生成病变识别结果，第一识别模型用于判别医疗影像是否发生病变；
识别单元412，用于通过第二识别模型对发生病变的医疗影像进行识别，生成病变程度识别结果，第二识别模型用于识别医疗影像包括病变的程度。
一些实施例中，判别单元411可以利用所述第一识别模型中经过训练的深度学 习网络在所述医疗影像中搜索病变特征，根据搜索结果生成所述病变识别结果。其中，所述病变特征为所述深度学习网络在训练时从经过标记的正常器官的第一医疗影像集合和发生病变的器官的第二医疗影像集合中学习得到的、存在于所述第二医疗影像集合中且不存在于所述第一医疗影像集合中的图像特征。
一些实施例中，识别单元412可以利用所述第二识别模型中经过训练的第二深度学习网络在所述医疗影像中搜索第一病变程度对应的病变程度特征，根据搜索结果生成所述病变程度识别结果。其中，所述病变程度特征为所述第二深度学习网络在训练时从经过标记的不具有所述第一病变程度的器官的第三医疗影像集合和具有所述第一病变程度的器官的第四医疗影像集合中学习得到的、存在于所述第四医疗影像集合中且不存在于所述第三医疗影像集合中的图像特征。
一些实施例中，识别单元412具体用于：将病变识别结果为发生病变的医疗影像分割为多个影像块；
分别提取多个影像块的特征信息，根据提取的特征信息确定各个影像块的病变程度识别结果；以及
病变程度识别结果包括：每个影像块的病变程度识别结果及其在医疗影像中的区域信息，和/或，根据每个影像块的病变程度识别结果在其对应区域设置了相应指示信息后的病变程度指示影像。
一些实施例中，病变程度识别结果还包括医疗影像的病变程度标签，医疗影像的病变程度标签为：
病例影像分割出的多个影像块中病变程度最严重的影像块的第一识别结果；或者，
根据所有影像块的特征信息确定的医疗影像的病变程度的第二识别结果；或者，
根据第一识别结果和第二识别结果确定的综合结果。
一些实施例中，第二识别模型，是基于卷积神经网络训练得到的，并且采用第二识别误差对第二识别模型进行优化；
其中，第二识别误差是采用指定的损失函数获得的，损失函数中的交叉熵，是基于医疗影像的各个影像块中病变程度最高的影像块的病变程度识别结果确定的。
一些实施例中，第一识别模型，是基于DenseNet训练获得的，并且采用第一识别误差对第一识别模型进行优化；
其中，第一识别误差是根据各个医疗影像的病变识别结果的交叉熵和相应约束表达式值的加和的平均值，以及正则值获得的，其中，医疗影像的约束表达值是通过预设的约束表达式获得的，约束表达式用于在医疗影像的人工判别结果为正常时获得有效的约束表达值。
如图4b所示，其为本发明实施例提供的一种影像识别的装置的结构示意图二，包括：
输入单元420，用于获取待识别的医疗影像，并通过识别模型对待识别的医疗 影像进行识别；
获得单元421，用于获得识别模型对待识别的医疗影像进行识别后输出的病变程度识别结果，且病变程度识别结果包括：医疗影像包括的多个影像块中，每个影像块的病变程度识别结果及其在医疗影像中的区域信息，和/或根据每个影像块的病变程度识别结果在其对应区域设置了相应指示信息后的病变程度指示影像；
其中，识别模型对医疗影像进行识别具体包括：将医疗影像分割为多个影像块，并针对每个影像块，提取该影像块的特征信息并根据提取的特征信息确定该影像块的病变程度识别结果。
本发明实施例提供的一种影像识别的方法、装置、终端设备和医疗系统中，先通过第一识别模型判别医疗影像是否为发生病变的医疗影像，然后，通过第二识别模型对发生病变的医疗影像进行进一步识别，获得病变程度识别结果，以指示医疗影像所包括的病变的程度。这不需要人工进行分析以及定制特征抽取方案，提高了医疗影像识别的效率和精确度。
基于同一技术构思，本发明实施例还提供了一种终端设备500，参照图5所示，终端设备500用于实施上述各个方法实施例记载的方法，例如实施图2所示的实施例，终端设备500可以包括存储器501、处理器502、输入单元503和显示面板504。
存储器501，用于存储处理器502执行的计算机程序。存储器501可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序等；存储数据区可存储根据终端设备500的使用所创建的数据等。处理器502，可以是一个中央处理单元(central processing unit，CPU)，或者为数字处理单元等等。输入单元503，可以用于获取用户输入的用户指令。显示面板504，用于显示由用户输入的信息或提供给用户的信息，本发明实施例中，显示面板504主要用于显示终端设备中各应用程序的显示界面以及各显示界面中显示的控件实体。各实施例中，显示面板504可以采用液晶显示器(liquid crystal display，LCD)或OLED(organic light-emitting diode，有机发光二极管)等形式来配置显示面板504。
本发明实施例中不限定上述存储器501、处理器502、输入单元503和显示面板504之间的具体连接介质。本发明实施例在图5中以存储器501、处理器502、输入单元503、显示面板504之间通过总线505连接，总线505在图5中以粗线表示，其它部件之间的连接方式，仅是进行示意性说明，并不引以为限。总线505可以分为地址总线、数据总线、控制总线等。为便于表示，图5中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
存储器501可以是易失性存储器(volatile memory)，例如随机存取存储器(random-access memory，RAM)；存储器501也可以是非易失性存储器(non-volatile memory)，例如只读存储器，快闪存储器(flash memory)，硬盘(hard disk drive，HDD)或固态硬盘(solid-state drive，SSD)、或者存储器501是能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质， 但不限于此。存储器501可以是上述存储器的组合。
处理器502，用于实现如图2所示的实施例，包括：
处理器502，用于调用存储器501中存储的计算机程序执行如实施图2所示的实施例。
本发明实施例还提供了一种计算机可读存储介质，存储为执行上述处理器所需执行的计算机可执行指令，其包含用于执行上述处理器所需执行的程序。
在一些可能的实施方式中，本发明提供的一种影像识别的方法的各个方面还可以实现为一种程序产品的形式，其包括程序代码，当程序产品在终端设备上运行时，程序代码用于使终端设备执行本说明书上述描述的根据本发明各种示例性实施方式的一种影像识别的方法中的步骤。例如，终端设备可以执行如实施图2所示的实施例。
程序产品可以采用一个或多个可读介质的任意组合。可读介质可以是可读信号介质或者可读存储介质。可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。
本发明的实施方式的用于一种影像识别的程序产品可以采用便携式紧凑盘只读存储器(CD-ROM)并包括程序代码，并可以在计算设备上运行。然而，本发明的程序产品不限于此，在本文件中，可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。
可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了可读程序代码。这种传播的数据信号可以采用多种形式，包括——但不限于——电磁信号、光信号或上述的任意合适的组合。可读信号介质还可以是可读存储介质以外的任何可读介质，该可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。
可读介质上包含的程序代码可以用任何适当的介质传输，包括——但不限于——无线、有线、光缆、RF等等，或者上述的任意合适的组合。
可以以一种或多种程序设计语言的任意组合来编写用于执行本发明操作的程序代码，程序设计语言包括面向实体的程序设计语言—诸如Java、C++等，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算设备上执行、部分地在用户设备上执行、作为一个独立的软件包执行、部分在用户计算设备上部分在远程计算设备上执行、或者完全在远程计算设备或服务器上执行。在涉及远程计算设备的情形中，远程计算设备可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到用户计算设备，或者，可以 连接到外部计算设备(例如利用因特网服务提供熵来通过因特网连接)。
应当注意，尽管在上文详细描述中提及了装置的若干单元或子单元，但是这种划分仅仅是示例性的并非强制性的。实际上，根据本发明的实施方式，上文描述的两个或更多单元的特征和功能可以在一个单元中具体化。反之，上文描述的一个单元的特征和功能可以进一步划分为由多个单元来具体化。
此外，尽管在附图中以特定顺序描述了本发明方法的操作，但是，这并非要求或者暗示必须按照该特定顺序来执行这些操作，或是必须执行全部所示的操作才能实现期望的结果。附加地或备选地，可以省略某些步骤，将多个步骤合并为一个步骤执行，和/或将一个步骤分解为多个步骤执行。
本领域内的技术人员应明白，本发明的实施例可提供为方法、系统、或计算机程序产品。因此，本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
尽管已描述了本发明的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例做出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本发明范围的所有变更和修改。
显然，本领域的技术人员可以对本发明进行各种改动和变型而不脱离本发明的精神和范围。这样，倘若本发明的这些修改和变型属于本发明权利要求及其等同技术的范围之内，则本发明也意图包含这些改动和变型在内。

Claims (20)

  1. 一种医疗影像的识别方法，包括：
    获取待识别的医疗影像；
    通过第一识别模型对所述医疗影像进行判别，生成用于指示所述医疗影像是否包括病变的病变识别结果；
    当所述病变识别结果指示所述医疗影像包括病变时，通过第二识别模型对所述医疗影像进行识别，生成用于指示所述病变的程度的病变程度识别结果。
  2. 如权利要求1所述的方法，其中，通过第一识别模型对所述医疗影像进行判别，生成用于指示所述医疗影像是否包括病变的病变识别结果包括：
    所述第一识别模型利用经过训练的深度学习网络在所述医疗影像中搜索病变特征，根据搜索结果生成所述病变识别结果；
    其中，所述病变特征为所述深度学习网络在训练时从经过标记的正常器官的第一医疗影像集合和发生病变的器官的第二医疗影像集合中学习得到的、存在于所述第二医疗影像集合中且不存在于所述第一医疗影像集合中的图像特征。
  3. 如权利要求1所述的方法，其中，通过第二识别模型对所述医疗影像进行识别，生成用于指示所述病变的程度的病变程度识别结果包括：
    所述第二识别模型利用经过训练的第二深度学习网络在所述医疗影像中搜索第一病变程度对应的病变程度特征，根据搜索结果生成所述病变程度识别结果；
    其中，所述病变程度特征为所述第二深度学习网络在训练时从经过标记的不具有所述第一病变程度的器官的第三医疗影像集合和具有所述第一病变程度的器官的第四医疗影像集合中学习得到的、存在于所述第四医疗影像集合中且不存在于所述第三医疗影像集合中的图像特征。
  4. 如权利要求1所述的方法，其中，通过第二识别模型对所述医疗影像进行识别包括：
    将所述医疗影像分割为多个影像块；
    分别提取所述多个影像块的特征信息，根据提取的特征信息确定各个影像块的病变程度识别结果；以及
    所述病变程度识别结果包括：每个影像块的病变程度识别结果及其在医疗影像中的区域信息，和/或，根据每个影像块的病变程度识别结果在其对应区域设置了相应指示信息后的病变程度指示影像。
  5. 如权利要求2所述的方法，其中，所述病变程度识别结果还包括医疗影像的病变程度标签，所述医疗影像的病变程度标签为：
    所述病例影像分割出的多个影像块中病变程度最严重的影像块的第一识别结果；或者，
    根据所有影像块的特征信息确定的所述医疗影像的病变程度的第二识别结果；或者，
    根据所述第一识别结果和所述第二识别结果确定的综合结果。
  6. 如权利要求2-5中任一权利要求所述的方法，其中，所述第二识别模型是基于卷积神经网络训练得到的，并且采用第二识别误差对所述第二识别模型进行优化；
    其中，所述第二识别误差是采用指定的损失函数获得的，所述损失函数中的交叉熵，是基于医疗影像的各个影像块中病变程度最高的影像块的病变程度识别结果确定的。
  7. 如权利要求2-5中任一权利要求所述的方法，其中，所述第一识别模型，是基于稠密卷积网络DenseNet训练获得的，并且采用第一识别误差对所述第一识别模型进行优化；
    其中，所述第一识别误差是根据各个医疗影像的病变识别结果的交叉熵和相应约束表达式值的加和的平均值，以及正则值获得的，其中，医疗影像的约束表达值是通过预设的约束表达式获得的，所述约束表达式用于在医疗影像的人工判别结果为正常时获得有效的约束表达值。
  8. 一种影像识别的装置，包括：
    获取单元，用于获取待识别的医疗影像；
    判别单元，用于通过第一识别模型对所述医疗影像进行判别，生成用于指示所述医疗影像是否包括病变的病变识别结果；
    识别单元，用于当所述病变识别结果指示所述医疗影像包括病变时，通过第二识别模型对所述医疗影像进行识别，生成用于指示所述病变的程度的病变程度识别结果。
  9. 如权利要求8所述的装置，其中，所述判别单元用于：
    利用所述第一识别模型中经过训练的深度学习网络在所述医疗影像中搜索病变特征，根据搜索结果生成所述病变识别结果；
    其中，所述病变特征为所述深度学习网络在训练时从经过标记的正常器官的第一医疗影像集合和发生病变的器官的第二医疗影像集合中学习得到的、存在于所述第二医疗影像集合中且不存在于所述第一医疗影像集合中的图像特征。
  10. 如权利要求8所述的装置，其中，所述识别单元用于：
    利用所述第二识别模型中经过训练的第二深度学习网络在所述医疗影像中搜索第一病变程度对应的病变程度特征，根据搜索结果生成所述病变程度识别结果；
    其中，所述病变程度特征为所述第二深度学习网络在训练时从经过标记的不具有所述第一病变程度的器官的第三医疗影像集合和具有所述第一病变程度的器官的第四医疗影像集合中学习得到的、存在于所述第四医疗影像集合中且不存在于所述第三医疗影像集合中的图像特征。
  11. 如权利要求8所述的装置，所述识别单元具体用于：将所述医疗影像分割为多个影像块；
    分别提取所述多个影像块的特征信息，根据提取的特征信息确定各个影像块的 病变程度识别结果；以及
    所述病变程度识别结果包括：每个影像块的病变程度识别结果及其在医疗影像中的区域信息，和/或，根据每个影像块的病变程度识别结果在其对应区域设置了相应指示信息后的病变程度指示影像。
  12. 如权利要求11所述的装置，所述病变程度识别结果还包括医疗影像的病变程度标签，所述医疗影像的病变程度标签为：
    所述病例影像分割出的多个影像块中病变程度最严重的影像块的第一识别结果；或者，
    根据所有影像块的特征信息确定的所述医疗影像的病变程度的第二识别结果；或者，
    根据所述第一识别结果和所述第二识别结果确定的综合结果。
  13. 如权利要求9-12中任一权利要求所述的装置，所述第二识别模型，是基于卷积神经网络训练得到的，并且采用第二识别误差对所述第二识别模型进行优化；
    其中，所述第二识别误差是采用指定的损失函数获得的，所述损失函数中的交叉熵，是基于医疗影像的各个影像块中病变程度最高的影像块的病变程度识别结果确定的。
  14. 如权利要求9-12中任一权利要求所述的装置，所述第一识别模型，是基于稠密卷积网络DenseNet训练获得的，并且采用第一识别误差对所述第一识别模型进行优化；
    其中，所述第一识别误差是根据各个医疗影像的病变识别结果的交叉熵和相应约束表达式值的加和的平均值，以及正则值获得的，其中，医疗影像的约束表达值是通过预设的约束表达式获得的，所述约束表达式用于在医疗影像的人工判别结果为正常时获得有效的约束表达值。
  15. 一种医疗影像的识别方法，包括：
    获取待识别的医疗影像，并通过识别模型对所述待识别的医疗影像进行识别；
    获得所述识别模型对所述待识别的医疗影像进行识别后输出的病变程度识别结果，且所述病变程度识别结果包括：所述医疗影像包括的多个影像块中，每个影像块的病变程度识别结果及其在医疗影像中的区域信息，和/或根据每个影像块的病变程度识别结果在其对应区域设置了相应指示信息后的病变程度指示影像；
    其中，所述识别模型对所述医疗影像进行识别包括：将所述医疗影像分割为多个影像块，并针对每个影像块，提取该影像块的特征信息并根据提取的特征信息确定该影像块的病变程度识别结果。
  16. 一种影像识别的装置，包括：
    输入单元，用于获取待识别的医疗影像，并通过识别模型对所述待识别的医疗影像进行识别；
    获得单元，用于获得所述识别模型对所述待识别的医疗影像进行识别后输出的 病变程度识别结果，且所述病变程度识别结果包括：所述医疗影像包括的多个影像块中，每个影像块的病变程度识别结果及其在医疗影像中的区域信息，和/或根据每个影像块的病变程度识别结果在其对应区域设置了相应指示信息后的病变程度指示影像；
    其中，所述识别模型对所述医疗影像进行识别具体包括：将所述医疗影像分割为多个影像块，并针对每个影像块，提取该影像块的特征信息并根据提取的特征信息确定该影像块的病变程度识别结果。
  17. 一种终端设备，包括至少一个处理单元、以及至少一个存储单元，其中，所述存储单元存储有计算机程序，当所述程序被所述处理单元执行时，使得所述处理单元执行权利要求1～7任一权利要求所述方法的步骤。
  18. 一种医疗系统，包括影像采集装置、影像识别装置和显示装置，其中，
    所述影像采集装置，用于采集病人的医疗影像；
    所述影像识别装置，用于获取所述影像采集装置采集的医疗影像，并通过第一识别模型对所述医疗影像进行判别，生成用于指示所述医疗影像是否包括病变的病变识别结果，以及当所述病变识别结果指示所述医疗影像包括病变时，通过第二识别模型对所述医疗影像进行识别，生成用于指示所述病变的程度的病变程度识别结果；
    所述显示装置，用于呈现所述病变程度识别结果。
  19. 一种医疗系统，包括影像采集装置、影像识别装置和显示装置，其中，
    所述影像采集装置，用于采集病人的医疗影像；
    所述影像识别装置，用于获取所述影像采集装置采集的医疗影像，并通过第二识别模型对包括病变的医疗影像进行识别，生成病变程度识别结果，其中，所述第二识别模型用于识别医疗影像所包括的病变的程度；
    所述显示装置，用于呈现所述病变程度识别结果。
  20. 一种计算机可读存储介质，存储有计算机程序，当所述程序被一个或多个处理器执行时，使得所述一个或多个处理器执行权利要求1～7任一权利要求所述方法的步骤。
PCT/CN2019/093602 2018-10-30 2019-06-28 一种影像识别的方法、装置、终端设备和医疗系统 Ceased WO2020087960A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP19879486.9A EP3876192B1 (en) 2018-10-30 2019-06-28 Image recognition method and device
JP2020561046A JP7152513B2 (ja) 2018-10-30 2019-06-28 画像認識方法、装置、端末機器及び医療システム、並びにそのコンピュータプログラム
US17/078,878 US11410306B2 (en) 2018-10-30 2020-10-23 Method, apparatus, system, and storage medium for recognizing medical image
US17/856,043 US11610310B2 (en) 2018-10-30 2022-07-01 Method, apparatus, system, and storage medium for recognizing medical image

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811278418.3 2018-10-30
CN201811278418.3A CN109427060A (zh) 2018-10-30 2018-10-30 一种影像识别的方法、装置、终端设备和医疗系统

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/078,878 Continuation US11410306B2 (en) 2018-10-30 2020-10-23 Method, apparatus, system, and storage medium for recognizing medical image

Publications (1)

Publication Number Publication Date
WO2020087960A1 true WO2020087960A1 (zh) 2020-05-07

Family

ID=65514793

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/093602 Ceased WO2020087960A1 (zh) 2018-10-30 2019-06-28 一种影像识别的方法、装置、终端设备和医疗系统

Country Status (5)

Country Link
US (2) US11410306B2 (zh)
EP (1) EP3876192B1 (zh)
JP (1) JP7152513B2 (zh)
CN (1) CN109427060A (zh)
WO (1) WO2020087960A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112233194A (zh) * 2020-10-15 2021-01-15 平安科技（深圳）有限公司 医学图片优化方法、装置、设备及计算机可读存储介质
CN113130050A (zh) * 2021-04-20 2021-07-16 皖南医学院第一附属医院（皖南医学院弋矶山医院） 一种医学信息显示方法及显示系统
CN115311268A (zh) * 2022-10-10 2022-11-08 武汉楚精灵医疗科技有限公司 食管内窥镜图像的识别方法及装置

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109427060A (zh) 2018-10-30 2019-03-05 腾讯科技（深圳）有限公司 一种影像识别的方法、装置、终端设备和医疗系统
CN110432977A (zh) * 2019-08-07 2019-11-12 杭州睿笛生物科技有限公司 一种电脉冲消融设备以及适用其仿真方法
CN110610489B (zh) * 2019-08-30 2021-11-23 西安电子科技大学 基于注意力机制的光学喉镜图像病变区标注方法
TWI705458B (zh) * 2019-09-19 2020-09-21 沐恩生醫光電股份有限公司 一種醫療影像辨識的方法與系統
CN111488912B (zh) * 2020-03-16 2020-12-11 哈尔滨工业大学 一种基于深度学习神经网络的喉部疾病诊断系统
CN111860209B (zh) * 2020-06-29 2024-04-26 北京字节跳动网络技术有限公司 手部识别方法、装置、电子设备及存储介质
CN112037167B (zh) * 2020-07-21 2023-11-24 苏州动影信息科技有限公司 一种基于影像组学和遗传算法的目标区域确定系统
CN112509688B (zh) * 2020-09-25 2024-06-11 卫宁健康科技集团股份有限公司 压疮图片自动分析系统、方法、设备和介质
CN113159238B (zh) * 2021-06-23 2021-10-26 安翰科技(武汉)股份有限公司 内窥镜影像识别方法、电子设备及存储介质
CN116342859B (zh) * 2023-05-30 2023-08-18 安徽医科大学第一附属医院 一种基于影像学特征识别肺部肿瘤区域的方法及系统
WO2025150130A1 (ja) * 2024-01-11 2025-07-17 富士通株式会社 病変識別方法および病変識別プログラム

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107609503A (zh) * 2017-09-05 2018-01-19 刘宇红 智能癌变细胞识别系统及方法、云平台、服务器、计算机
CN108573490A (zh) * 2018-04-25 2018-09-25 王成彦 一种针对肿瘤影像数据的智能读片系统
WO2018189551A1 (en) * 2017-04-12 2018-10-18 Kheiron Medical Technologies Ltd Malignancy assessment for tumors
CN109427060A (zh) * 2018-10-30 2019-03-05 腾讯科技（深圳）有限公司 一种影像识别的方法、装置、终端设备和医疗系统

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105997128A (zh) * 2016-08-03 2016-10-12 上海联影医疗科技有限公司 利用灌注成像识别病灶的方法及系统
CA3040518C (en) * 2016-10-21 2023-05-23 Nantomics, Llc Digital histopathology and microdissection
CN108095683A (zh) * 2016-11-11 2018-06-01 北京羽医甘蓝信息技术有限公司 基于深度学习的处理眼底图像的方法和装置
CN107203778A (zh) * 2017-05-05 2017-09-26 平安科技（深圳）有限公司 视网膜病变程度等级检测系统及方法
JP6890184B2 (ja) * 2017-09-15 2021-06-18 富士フイルム株式会社 医療画像処理装置及び医療画像処理プログラム
CN107563123A (zh) * 2017-09-27 2018-01-09 百度在线网络技术（北京）有限公司 用于标注医学图像的方法和装置
WO2019087790A1 (ja) * 2017-10-31 2019-05-09 富士フイルム株式会社 検査支援装置、内視鏡装置、検査支援方法、及び検査支援プログラム
CN108305249B (zh) * 2018-01-24 2022-05-24 福建师范大学 基于深度学习的全尺度病理切片的快速诊断和评分方法
CN108346154B (zh) * 2018-01-30 2021-09-07 浙江大学 基于Mask-RCNN神经网络的肺结节分割装置的建立方法
CN108510482B (zh) * 2018-03-22 2020-12-04 姚书忠 一种基于阴道镜图像的宫颈癌检测装置
CN108710901B (zh) * 2018-05-08 2022-03-01 广州市新苗科技有限公司 一种基于深度学习的脊柱畸形筛查系统及方法
CN108665457B (zh) * 2018-05-16 2023-12-19 腾讯医疗健康(深圳)有限公司 图像识别方法、装置、存储介质及计算机设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018189551A1 (en) * 2017-04-12 2018-10-18 Kheiron Medical Technologies Ltd Malignancy assessment for tumors
CN107609503A (zh) * 2017-09-05 2018-01-19 刘宇红 智能癌变细胞识别系统及方法、云平台、服务器、计算机
CN108573490A (zh) * 2018-04-25 2018-09-25 王成彦 一种针对肿瘤影像数据的智能读片系统
CN109427060A (zh) * 2018-10-30 2019-03-05 腾讯科技（深圳）有限公司 一种影像识别的方法、装置、终端设备和医疗系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3876192A4 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112233194A (zh) * 2020-10-15 2021-01-15 平安科技（深圳）有限公司 医学图片优化方法、装置、设备及计算机可读存储介质
CN112233194B (zh) * 2020-10-15 2023-06-02 平安科技（深圳）有限公司 医学图片优化方法、装置、设备及计算机可读存储介质
CN113130050A (zh) * 2021-04-20 2021-07-16 皖南医学院第一附属医院（皖南医学院弋矶山医院） 一种医学信息显示方法及显示系统
CN113130050B (zh) * 2021-04-20 2023-11-24 皖南医学院第一附属医院（皖南医学院弋矶山医院） 一种医学信息显示方法及显示系统
CN115311268A (zh) * 2022-10-10 2022-11-08 武汉楚精灵医疗科技有限公司 食管内窥镜图像的识别方法及装置
CN115311268B (zh) * 2022-10-10 2022-12-27 武汉楚精灵医疗科技有限公司 食管内窥镜图像的识别方法及装置

Also Published As

Publication number Publication date
EP3876192B1 (en) 2026-02-25
EP3876192A1 (en) 2021-09-08
EP3876192A4 (en) 2021-11-17
CN109427060A (zh) 2019-03-05
JP2021521553A (ja) 2021-08-26
US11610310B2 (en) 2023-03-21
US20220343502A1 (en) 2022-10-27
US11410306B2 (en) 2022-08-09
JP7152513B2 (ja) 2022-10-12
US20210042920A1 (en) 2021-02-11

Similar Documents

Publication Publication Date Title
WO2020087960A1 (zh) 一种影像识别的方法、装置、终端设备和医疗系统
CN111340819B (zh) 图像分割方法、装置和存储介质
Younas et al. A deep ensemble learning method for colorectal polyp classification with optimized network parameters: A deep ensemble learning method for colorectal polyp classification with optimized network parameters
Yue et al. Automated endoscopic image classification via deep neural network with class imbalance loss
Ay et al. Automated classification of nasal polyps in endoscopy video-frames using handcrafted and CNN features
WO2020088288A1 (zh) 内窥镜图像的处理方法、系统及计算机设备
CN108268870A (zh) 基于对抗学习的多尺度特征融合超声图像语义分割方法
CN110689025A (zh) 图像识别方法、装置、系统及内窥镜图像识别方法、装置
CN110363768A (zh) 一种基于深度学习的早期癌病灶范围预测辅助系统
CN109948671B (zh) 图像分类方法、装置、存储介质以及内窥镜成像设备
Meer et al. Deep convolutional neural networks information fusion and improved whale optimization algorithm based smart oral squamous cell carcinoma classification framework using histopathological images
Xie et al. Optic disc and cup image segmentation utilizing contour-based transformation and sequence labeling networks
Sornapudi et al. DeepCIN: attention-based cervical histology image classification with sequential feature modeling for pathologist-level accuracy
Haj‐Manouchehri et al. Polyp detection using CNNs in colonoscopy video
CN117218129B (zh) 食道癌图像识别分类方法、系统、设备及介质
Chen et al. Deep transfer learning for histopathological diagnosis of cervical cancer using convolutional neural networks with visualization schemes
CN112668668B (zh) 一种术后医学影像评估方法、装置、计算机设备及存储介质
Bhandari et al. Improved diabetic retinopathy severity classification using squeeze-and-excitation and sparse light weight multi-level attention u-net with transfer learning from xception
CN118657993B (zh) 一种具备可解释性的病灶感知眼底图像分类方法及系统
WO2024016691A1 (zh) 一种图像检索方法、模型训练方法、装置及存储介质
WO2022160070A1 (en) Machine learning enabled system for skin abnormality interventions
Deepa et al. Pre-Trained Convolutional Neural Network for Automated Grading of Diabetic Retinopathy
CN115908224A (zh) 目标检测模型的训练方法、目标检测方法和训练装置
Yang et al. An automatic method for sublingual image segmentation and color analysis
Asif et al. SFI-ensemble: Sugeno fuzzy integral-based ensemble of CNN models with meta-heuristic fuzzy measures for mouth and oral disease detection

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19879486

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020561046

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019879486

Country of ref document: EP

Effective date: 20210531

WWG Wipo information: grant in national office

Ref document number: 2019879486

Country of ref document: EP