WO2020003992A1 - Dispositif d'apprentissage et procédé d'apprentissage, et dispositif de traitement d'image médicale - Google Patents

Dispositif d'apprentissage et procédé d'apprentissage, et dispositif de traitement d'image médicale Download PDF

Info

Publication number
WO2020003992A1
WO2020003992A1 PCT/JP2019/022910 JP2019022910W WO2020003992A1 WO 2020003992 A1 WO2020003992 A1 WO 2020003992A1 JP 2019022910 W JP2019022910 W JP 2019022910W WO 2020003992 A1 WO2020003992 A1 WO 2020003992A1
Authority
WO
WIPO (PCT)
Prior art keywords
learning
image
medical
model
endoscope
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2019/022910
Other languages
English (en)
Japanese (ja)
Inventor
麻依子 遠藤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Corp
Original Assignee
Fujifilm Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujifilm Corp filed Critical Fujifilm Corp
Priority to JP2020527359A priority Critical patent/JP7382930B2/ja
Publication of WO2020003992A1 publication Critical patent/WO2020003992A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/045—Control thereof
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present invention relates to a learning device, a learning method, and a medical image processing device, and in particular, to a learning device and a learning method for generating a model for performing image recognition on a medical image, and a medical device using the model.
  • the present invention relates to an image processing device.
  • a technology for automatically detecting a lesion from a medical image or classifying a lesion by type using an image recognition model generated by machine learning is known.
  • image recognition such as detection and classification can be performed by learning a large number of images corresponding to a problem.
  • Patent Document 1 discloses a method of generating a highly accurate image recognition model even when the number of images for learning is small, by performing pre-learning using an image group similar to the characteristics of the image group to be learned,
  • a method of performing actual learning using a group of images to be learned Specifically, pre-learning is performed based on a group of images in which the shape of the subject is similar, a group of images obtained by capturing a living body phantom, a group of images in which the tissue structure of the subject is similar, and a group of images obtained by capturing a simulated organ using the same imaging system.
  • a method of performing main learning using a group of images to be learned has been proposed.
  • Japanese Patent Application Laid-Open No. H11-163873 performs first learning with a first image group captured at a first frame rate, and then performs a second learning with a second image group captured at a second frame rate lower than the first frame rate. A method for performing the second learning has been proposed.
  • the present invention has been made in view of such circumstances, and provides a learning device, a learning method, and a medical image processing device capable of efficiently generating a model for performing image recognition on a medical image having a specific image quality.
  • the purpose is to do.
  • a first learning unit that generates a first model that performs image recognition on a medical image of the first image quality by learning using a first medical image group including medical images of the first image quality; And performing image recognition on a medical image of the second image quality by learning using a second medical image group composed of medical images of a second image quality different from the first image quality based on the first model.
  • a second learning unit that generates two models.
  • the first model is generated by performing the first learning on the first medical image group of the first image quality.
  • the second model is generated by performing the second learning on the second medical image group of the second image quality based on the first model. Since the second learning is performed based on the first learning result, an accurate model can be generated even when the number of learning images is small. Therefore, when generating a model for performing image recognition on a medical image having a specific image quality, the first learning is performed on the first medical image group having a large number of learning images, and then the medical image having the target image quality is obtained. By performing the second learning on the image group (second medical image group), it is possible to efficiently generate a model for performing image recognition on a medical image of a target image quality.
  • the images constituting the medical image include both moving images and still images.
  • a moving image can be regarded as a time-series image group including a plurality of frames.
  • the image quality here means the image quality of an image constituting one frame in a moving image.
  • the first medical image group is composed of medical images of the first resolution
  • the second medical image group is composed of medical images of the second resolution different from the first resolution. Accordingly, when a model for performing image recognition on a medical image having a specific resolution is generated, an accurate model can be efficiently generated even when the number of learning images of the corresponding resolution is small. .
  • the second medical image group is configured by medical images having a lower resolution (second resolution) than the resolution (first resolution) of the medical images forming the first medical image group. Accordingly, when a model for performing image recognition on a medical image with a specific resolution is generated, even when the number of learning images with the corresponding resolution is small, a model with high accuracy can be efficiently generated.
  • the first medical image group is composed of medical images having a resolution of 4K or more
  • the second medical image group is composed of medical images having a resolution of less than 4K. Accordingly, when a model for performing image recognition on a medical image with a specific resolution of less than 4K is generated, even if the number of learning images of the corresponding resolution is small, a model with high accuracy can be efficiently created. Can be generated.
  • the 4K image refers to a high-definition image having about 4000 pixels on the long side. In particular, it refers to an image having about 4000 ⁇ 2000 pixels in the horizontal and vertical directions.
  • “4K UHDTV (Ultra High Definition Television: Ultra High Definition Television)” and “DCI 4K” which are generally known are included in “4K” in this specification.
  • “4K @ UHDTV” is 4K defined by the International Telecommunication Union (ITU), which is 3840 (horizontal) x 2160 (vertical) pixels.
  • DCI @ 4K is 4K defined by ⁇ Digital Cinema ⁇ Initiatives (DCI) to which movie companies and the like belong, and is an image of 4096 (width) ⁇ 2160 (height) pixels.
  • the first medical image group is composed of medical images having a resolution of 8K or more
  • the second medical image group is composed of medical images having a resolution of less than 8K. Accordingly, when a model for performing image recognition on a medical image having a specific resolution of less than 8K is generated, even if the number of learning images of the corresponding resolution is small, an accurate model can be efficiently generated. Can be generated.
  • the 8K image is a high-definition image having about 8000 pixels on the long side. In particular, it refers to an image in which the number of horizontal and vertical pixels is about 8000 ⁇ 4000.
  • the commonly known “8K UHDTV” also referred to as 8K Ultra-high-definition television, 8K Ultra HDTV, 8K UHDTV, 8K UHD, Super Hi-Vision 8K, etc.
  • 8K @ UHDTV is 8K defined by the International Telecommunication Union and is an image of 7680 horizontal ⁇ 4320 vertical pixels.
  • the second medical image group is configured by medical images having a higher resolution (second resolution) than the resolution (first resolution) of the medical images forming the first medical image group. Accordingly, when a model for performing image recognition on a medical image with a specific resolution is generated, even when the number of learning images with the corresponding resolution is small, a model with high accuracy can be efficiently generated.
  • the first medical image group is configured by medical images having a resolution of less than 4K
  • the second medical image group is configured by medical images having a resolution of 4K or more. Accordingly, when a model for performing image recognition on a medical image with a specific resolution of 4K or more is generated, even if the number of learning images of the corresponding resolution is small, an accurate model can be efficiently created. Can be generated.
  • the first medical image group is configured by medical images having a resolution of less than 8K
  • the second medical image group is configured by medical images having a resolution of 8K or more. Accordingly, when a model for performing image recognition on a medical image with a specific resolution of 8K or more is generated, even if the number of learning images of the corresponding resolution is small, a model with high accuracy can be efficiently created. Can be generated.
  • the first medical image group is constituted by medical images having less noise than the medical images constituting the second medical image group.
  • the first medical image group is configured by medical images having a larger amount of noise than the medical images forming the second medical image group.
  • the first medical image group is configured by medical images having a wider angle of view than the medical images forming the second medical image group. This makes it possible to generate a model that performs image recognition on a medical image having a specific angle of view without learning from an image that has been cut out or the like.
  • the first medical image group is composed of medical images captured by an endoscope
  • the second medical image group is medical images captured by an endoscope different from the endoscope capturing the first medical image group.
  • the learning device according to the above (1) comprising:
  • the first medical image group is configured by medical images captured by the endoscope
  • the second medical image group is configured by an endoscope different from the endoscope capturing the first medical image group. It is composed of captured medical images. Accordingly, when a model for performing image recognition on a medical image captured by a specific endoscope is generated, even if the number of learning images captured by the endoscope is small, the accuracy can be improved. Model can be generated efficiently.
  • the second medical image group is configured by medical images captured by an endoscope having a specification different from that of the endoscope capturing the medical image forming the first medical image group.
  • the first model and the second model are configured by a convolutional neural network.
  • a model configured by a medical image acquisition unit for acquiring a medical image and a second model generated by the learning device according to any one of (1) to (14), and performing image recognition on the medical image
  • a medical image processing apparatus comprising:
  • the model for performing image recognition on the medical image is configured by the second model generated by any one of the learning devices (1) to (14). Thereby, it is possible to accurately perform image recognition on a medical image having a specific image quality.
  • the medical image processing apparatus further including: a plurality of models; and a model switching unit that switches a model to be used.
  • a plurality of models for performing image recognition are provided, and are switched and used.
  • image recognition can be performed using an appropriate model according to the image quality.
  • An endoscope information acquisition unit for acquiring information of an endoscope that has taken a medical image is further provided, and the plurality of models are configured to use a second medical image group captured by different endoscopes to obtain a second model.
  • a plurality of models for performing image recognition are provided.
  • Each model is generated by learning by the second learning unit using the second medical image group captured by different endoscopes, and is used depending on the endoscope capturing the medical image to be recognized. Models to be switched automatically.
  • the plurality of models are generated by learning in the second learning unit using the second medical image group captured by the endoscope having different specifications from each other.
  • the plurality of models are generated by learning in the second learning unit using the second medical image group captured by the endoscope having different resolutions or different amounts of noise.
  • the first model is generated by performing the first learning on the first medical image group of the first image quality.
  • the second model is generated by performing the second learning on the second medical image group of the second image quality based on the first model. Since the second learning is performed based on the first learning result, an accurate model can be generated even when the number of learning images is small. Therefore, when generating a model for performing image recognition on a medical image having a specific image quality, the first learning is performed on the first medical image group having a large number of learning images, and then the medical image having the target image quality is obtained. By performing the second learning on the image group (second medical image group), it is possible to efficiently generate a model for performing image recognition on a medical image of a target image quality.
  • a model for performing image recognition on a medical image having a specific image quality can be efficiently generated.
  • FIG. 2 is a block diagram illustrating an embodiment of a configuration of a learning device.
  • Schematic diagram showing an example of the configuration of a CNN Conceptual diagram of setting of CNN constituting the first model and the second model Diagram showing an example of a hardware configuration of a learning device
  • Flow chart showing the procedure of learning performed by the learning device
  • Flowchart showing the learning procedure based on the learning result of a group of high-resolution learning images
  • the flowchart which shows the procedure of the learning based on the learning result by the low-resolution learning image group
  • the flowchart which shows the procedure of the learning based on the learning result by the learning image group of low noise.
  • a flowchart showing a learning procedure based on a learning result by a group of high-noise learning images.
  • a flowchart showing a learning procedure based on a learning result by a group of wide-angle learning images is a block diagram illustrating an embodiment of a configuration of an endoscope image processing device.
  • the figure which shows an example of the hardware constitutions of an endoscope image processing apparatus Block diagram showing a modified example of the endoscope image processing device Block diagram showing another modification of the endoscope image processing device
  • FIG. 1 is a block diagram showing an embodiment of the configuration of the learning device.
  • the learning device 1 is configured as a device that generates a model for performing image recognition on an endoscope image obtained by endoscopy by machine learning.
  • An endoscope image is an example of a medical image.
  • the learning device 1 of the present embodiment is configured as a device that generates a model for performing image recognition on an endoscope image having a specific image quality by machine learning.
  • the image recognition performed here is, for example, detection of a lesion included in the image, classification of each type of lesion, and the like.
  • a learning device 1 includes a first learning unit 10 that generates a first model M1 that performs image recognition on an endoscopic image of a first image quality by machine learning, A second learning unit that generates a second model that performs image recognition on an endoscopic image having a second image quality based on the first model that is generated by the first learning unit; And a learning control unit 30 that performs overall control of the operation. Further, a first learning data set 12 for learning by the first learning unit 10 and a second learning data set 22 for learning by the second learning unit 20 are provided.
  • the first learning unit 10 performs learning using the first learning data set 12 to generate a first model M1 that performs image recognition on an endoscopic image of the first image quality.
  • the first model M1 is composed of, for example, a convolutional neural network (CNN).
  • CNN convolutional neural network
  • the first learning data set 12 includes a first endoscope image group which is a learning image group.
  • the first endoscope image group is an example of a first medical image group, and includes an endoscope image of the first image quality.
  • the images constituting the endoscope image include both still images and moving images.
  • a moving image can be regarded as a time-series image group including a plurality of frames.
  • the data forming the image is data having intensity values (luminance values) of red (Red, R), green (Green, G) and blue (Blue, B) in pixel units.
  • the image quality means the image quality of an image constituting one frame.
  • the second learning unit 20 learns the endoscope image of the second image quality by learning using the second learning data set 22 based on the first model M1 generated by the first learning unit 10.
  • a second model M2 for performing image recognition is generated.
  • the second model M2 is composed of, for example, CNN.
  • the second model M2 generated by the second learning unit 20 is a model that performs image recognition on an endoscopic image having a specific image quality. That is, image recognition processing is performed using the second model M2 as a learned model.
  • the second learning unit 20 optimizes the weight parameter of each layer of the CNN constituting the second model M2 by learning.
  • the second learning data set 22 includes a second endoscope image group that is a learning image group.
  • the second endoscope image group is an example of a second medical image group, and includes an endoscope image having a second image quality different from the first image quality.
  • the image quality (second image quality) of the endoscope images constituting the second endoscope image group is set to the same image quality (including the same level of image quality) as that of the image recognition target.
  • the first learning data set 12 is composed of an endoscope image having an image quality (first image quality) different from the second image quality. Therefore, the image quality of the endoscope image to be subjected to image recognition is different from the image quality of the endoscope image. Specifically, it is configured by an endoscope image of higher image quality or lower image quality.
  • the learning control unit 30 controls the operations of the first learning unit 10 and the second learning unit 20, and controls the overall operation of the learning device 1. Further, the learning control unit 30 sets the CNNs that form the first model M1 and the second model M2.
  • FIG. 2 is a schematic diagram showing an example of the configuration of the CNN.
  • the CNN is configured by a multilayer neural network configured by stacking a convolutional layer, a normalization layer, a pooling layer, and the like.
  • FIG. 3 is a conceptual diagram of the setting of the CNN constituting the first model and the second model.
  • (A) is a diagram showing an example of a CNN constituting a first model M1
  • (B) is a diagram showing an example of a CNN constituting a second model M2.
  • the learning control unit 30 sets the CNN that forms the first model M1, and sets the CNN that forms the second model M2 based on the learned first model M1.
  • the CNN of the second model M2 is set by resetting the weight parameters of some layers of the CNN constituting the learned first model M1.
  • the layers for which the weight parameters are reset are some layers close to the output.
  • the weight parameters of the last three layers (all connected layers, all connected layers, and Softmax layer) surrounded by the broken line BL are reset, and the CNN of the second model M2 is set.
  • the weight parameters of the learned first model M1 are set as initial values.
  • FIG. 4 is a diagram illustrating an example of a hardware configuration of the learning device.
  • the learning device 1 is composed of computers such as a server computer and a client computer, and includes a CPU (Central Processing Unit) 51, a ROM (Read Only Memory) 52, a RAM (Random Access Memory) 53, an HDD (Hard Disk Drive) 54, and communication. An interface 55 and an input / output interface 56 are provided.
  • the learning device 1 includes an input device 57, a display device 58, and the like.
  • the CPU 51 controls each unit of the learning device 1 by executing the program, and realizes each function of the learning device 1.
  • the ROM 52 stores various programs executed by the CPU 51, various data, and the like.
  • the RAM 53 provides the CPU 51 with a work area.
  • the HDD 54 stores various programs executed by the CPU 51 and various data.
  • the communication interface 55 is an interface (interface; I / F) for connecting the learning device 1 to a network 59 such as a LAN (Local Area Network).
  • the learning device 1 communicates with an external device via the communication interface 55.
  • the input / output interface 56 is an interface for connecting external devices such as the input device 57 and the display device 58 to the learning device 1.
  • the input device 57 inputs information corresponding to an operation by a user to the learning device 1.
  • the input device 57 includes, for example, a keyboard, a mouse, and the like.
  • the display device 58 displays various information.
  • the display device 58 is configured by, for example, a liquid crystal display, an organic EL (Electro Luminescence) display, or the like.
  • the functions of the first learning unit 10, the second learning unit 20, and the learning control unit 30 that constitute the learning device 1 are realized by the CPU 51 executing a predetermined program. Further, the first learning data set 12 and the second learning data set 22 are stored in the HDD 54.
  • FIG. 5 is a flowchart illustrating a learning procedure performed by the learning device.
  • the CNN constituting the first model M1 is set (step S1).
  • the first learning is performed on the set CNN using the first learning data set 12 (step S2). That is, learning is performed on a first endoscopic image group composed of endoscopic images of the first image quality. As a result, a first model M1 for performing image recognition on the endoscopic image of the first image quality is generated.
  • the CNN constituting the second model M2 is set based on the learned first model M1 (step S3).
  • the weight parameters of some layers close to the output of the learned first model M1 are reset, and the CNN of the second model M2 is set (see FIG. 3).
  • step S4 second learning is performed on the set CNN using the second learning data set 22 (step S4). That is, learning is performed using a second endoscope image group including endoscope images of the second image quality.
  • a second model M2 for performing image recognition on the endoscopic image of the second image quality is generated.
  • the second endoscope image group used in the second learning has the same image quality (the same level of image quality) as the endoscope image (endoscope image having a specific image quality) to be subjected to image recognition. (Including image quality).
  • the second model M2 generated by the second learning is a model capable of performing image recognition on an endoscopic image having the same image quality as the endoscope image to be subjected to image recognition.
  • the endoscope of the second image quality is performed based on the first learning result.
  • the second learning is performed on the image group. Since the second learning is performed based on the first learning result, an accurate model can be generated even when the number of learning images is small. Therefore, for example, when generating a model for performing image recognition on a medical image having a specific image quality, after performing the first learning with abundant image quality for a learning image, a medical image group having a target image quality is obtained. Performs the second learning. Thereby, a target model can be efficiently generated.
  • Example ⁇ Learning with a group of learning images with different resolutions>
  • a first image is generated using an endoscope image group having a different resolution (first resolution) from the endoscope image for performing image recognition.
  • the second learning is performed using an endoscope image group having the same resolution (second resolution) as the endoscope image for performing image recognition.
  • (1) first learning is performed using an endoscope image group having a higher resolution (first resolution) than an endoscope image for which image recognition is performed, and based on the result, image learning is performed.
  • a method of performing the second learning using an endoscope image group having the same resolution (second resolution) as the endoscope image, and (2) an endoscope having a lower resolution (first resolution) than the endoscope image performing image recognition A method of performing a first learning using a group of mirror images, and performing a second learning with an endoscope image group having the same resolution (second resolution) as an endoscope image for performing image recognition based on the result. , There is. Hereinafter, the cases (1) and (2) will be described separately.
  • a model for performing image recognition on an endoscope image having a specific resolution is generated.
  • the first learning is performed using a group of endoscope images having a higher resolution than the resolution, and based on the result, the same resolution (including the same level of resolution) as the resolution of the endoscope image for performing image recognition is used.
  • the second learning is performed on the endoscope image group.
  • FIG. 6 is a flowchart showing a learning procedure based on a learning result of a high-resolution learning image group.
  • the CNN configuring the first model M1 is set (step S11).
  • the first learning is performed on the set CNN using the first learning data set 12 (step S12).
  • the first learning data set 12 includes an endoscope image having a relatively higher resolution than the resolution of the endoscope image to be subjected to image recognition.
  • a first model M1 that performs image recognition on an endoscopic image having a relatively high resolution is generated.
  • the CNN constituting the second model M2 is set based on the learned first model M1 (step S13).
  • the second learning is performed on the set CNN using the second learning data set 22 (step S14).
  • the second learning data set 22 includes an endoscope image having the same resolution (including the same level of resolution) as the resolution of the endoscope image to be subjected to image recognition.
  • a model (second model M2) capable of performing image recognition on the endoscope image having the target resolution is generated.
  • an endoscope image group having a resolution of 4K or more for example, an endoscope image group having a 4K resolution or an 8K resolution.
  • the first learning is performed, and then the second learning is performed on an endoscope image group having a target resolution (for example, 2K resolution) based on the first learning result.
  • a target resolution for example, 2K resolution
  • first learning is performed using a group of endoscopic images having a 4K resolution, and then 2K based on the first learning result.
  • the second learning is performed on the endoscope image group having the resolution.
  • a 2K image is a high-definition image in which the number of pixels on the long side is about 2000. In particular, it refers to an image in which the number of horizontal ⁇ vertical pixels is about 2000 ⁇ 1000. Therefore, general full high-definition (1920 ⁇ 1080) is included in 2K here.
  • an endoscope image group having a resolution of 8K or more (for example, an endoscope image group having an 8K resolution) is used. It is conceivable that the first learning is performed, and then the second learning is performed on an endoscope image group having a target resolution (for example, 2K resolution or 4K resolution) based on the first learning result.
  • a target resolution for example, 2K resolution or 4K resolution
  • a first learning is performed on a group of endoscopic images having an 8K resolution, and then a 4K resolution is set based on the first learning result.
  • the second learning is performed on the endoscope image group having the resolution.
  • the Optimize models at cost even when a model for performing image recognition on an endoscope image taken by a low-resolution endoscope used in a small hospital or the like is generated, the Optimize models at cost.
  • an endoscope image for performing image recognition is generated.
  • the first learning is performed using an endoscope image group having a resolution lower than the resolution, and based on the result, the same resolution (including the same level of resolution) as the resolution of the endoscope image for performing image recognition is used.
  • the second learning is performed on the endoscope image group.
  • FIG. 7 is a flowchart showing a learning procedure based on a learning result of a low-resolution learning image group.
  • the CNN constituting the first model M1 is set (step S21).
  • the first learning is performed on the set CNN using the first learning data set 12 (step S22).
  • the first learning data set 12 includes an endoscope image having a resolution relatively lower than the resolution of the endoscope image to be subjected to image recognition.
  • a first model M1 that performs image recognition on an endoscopic image having a relatively low resolution is generated.
  • the CNN constituting the second model M2 is set based on the learned first model M1 (step S23).
  • the second learning is performed on the set CNN using the second learning data set 22 (step S24).
  • the second learning data set 22 includes an endoscope image having the same resolution (including the same level of resolution) as the resolution of the endoscope image to be subjected to image recognition.
  • a model (second model M2) capable of performing image recognition on the endoscope image having the target resolution is generated.
  • the first group of endoscope images having a resolution of less than 4K (for example, an endoscope image group having a 2K resolution) is used.
  • the second learning is performed on an endoscope image group having a target resolution (for example, 4K resolution) based on the first learning result.
  • a target resolution for example, 4K resolution
  • a first learning is performed using a group of endoscopic images having a 2K resolution, and then a 4K resolution is set based on the first learning result.
  • the second learning is performed on the endoscope image group having the resolution.
  • an endoscope image group having a resolution of less than 8K for example, an endoscope image group having a 4K resolution or a 2K resolution.
  • the first learning may be performed, and then the second learning may be performed on the endoscope image group having the target resolution (for example, 8K resolution) based on the first learning result.
  • the target resolution for example, 8K resolution
  • a first learning is performed on a group of 4K resolution endoscope images, and then, based on the first learning result, an 8K resolution is obtained.
  • the second learning is performed on the endoscope image group having the resolution.
  • a model having a target resolution can be efficiently generated by using an existing learning image group even when a learning image having a target resolution is insufficient.
  • first learning is performed using an endoscope image group having a different noise amount from the endoscope image for performing image recognition.
  • the second learning is performed with an endoscope image group having the same amount of noise (including the same amount of noise) as the amount of noise of the endoscope image to be subjected to image recognition.
  • (1) the first learning is performed using an endoscope image group having a smaller amount of noise than the endoscope image for which image recognition is performed, and based on the result, the endoscope image for which image recognition is performed is performed.
  • a method of performing the second learning with an endoscope image group having the same noise amount (including the same amount of noise) as the noise amount, and (2) an endoscope having a larger amount of noise than an endoscope image performing image recognition The first learning is performed using the mirror image group, and based on the result, the endoscope image group having the same noise amount (including the same amount of noise) as the noise amount of the endoscope image for performing image recognition is used. And a second learning method.
  • the cases (1) and (2) will be described separately.
  • the learning is performed using an endoscope image that performs image recognition.
  • the second learning is performed on the endoscope image group.
  • FIG. 8 is a flowchart showing a learning procedure based on the learning result of the low-noise learning image group.
  • the CNN constituting the first model M1 is set (step S31).
  • the first learning is performed on the set CNN using the first learning data set 12 (step S32).
  • the first learning data set 12 is composed of an endoscope image having a relatively smaller noise amount than the noise amount of the endoscope image to be subjected to image recognition. For example, when generating a model for performing image recognition on an endoscope image captured by a low-end endoscope, the first learning image is generated using an endoscope image captured by a lower-end high-end endoscope.
  • the data set 12 is configured. By this first learning, a first model M1 that performs image recognition on an endoscopic image with relatively low noise is generated.
  • the CNN constituting the second model M2 is set based on the learned first model M1 (step S33).
  • the second learning data set 22 is composed of an endoscope image having the same amount of noise (including the same amount of noise) as the amount of noise of the endoscope image to be subjected to image recognition.
  • a second learning data set 22 is generated using an endoscope image captured by the low-end endoscope. Is composed.
  • a model capable of performing image recognition on the endoscope image having the target noise amount is generated.
  • large hospitals such as university hospitals use endoscopes with a relatively small amount of noise (so-called high-end endoscopes), and small hospitals such as clinics have a relatively large amount of noise.
  • An endoscope is used.
  • small hospitals have a problem that it is difficult to collect learning images because the number of examinations is smaller than that of large hospitals. For this reason, there is a case where the learning image of the target noise amount is insufficient.
  • a model for performing image recognition on an endoscope image captured by an endoscope (an endoscope having a relatively large amount of noise) used in a small hospital or the like is generated. Even if it does, the model can be optimized at low cost.
  • FIG. 9 is a flowchart showing a learning procedure based on a learning result of a group of high-noise learning images.
  • the CNN constituting the first model M1 is set (step S41).
  • the first learning is performed on the set CNN using the first learning data set 12 (step S42).
  • the first learning data set 12 includes an endoscope image having a relatively larger noise amount than the noise amount of the endoscope image to be subjected to image recognition.
  • a first model M1 that performs image recognition on an endoscopic image with relatively high noise is generated.
  • the CNN constituting the second model M2 is set based on the learned first model M1 (step S43).
  • the second learning is performed on the set CNN using the second learning data set 22 (step S44).
  • the second learning data set 22 includes an endoscope image having the same amount of noise (including the same amount of noise) as the amount of noise of the endoscope image to be subjected to image recognition.
  • a model capable of performing image recognition on the endoscope image having the target noise amount is generated.
  • the model of the target noise amount is efficiently generated using the existing learning image group. it can.
  • ⁇ Learning based on learning results using wide-angle learning images> When generating a model for performing image recognition on an endoscope image having a specific angle of view, the first learning is performed using an endoscope image group having a wider angle of view than the endoscope image for performing image recognition. Then, based on the result, the second learning is performed on an endoscope image group having the same angle of view (including substantially the same angle of view) as the angle of view of the endoscope image to be subjected to image recognition.
  • FIG. 10 is a flowchart showing a learning procedure based on a learning result of a wide-angle learning image group.
  • the CNN constituting the first model M1 is set (step S51).
  • the first learning is performed on the set CNN using the first learning data set 12 (step S52).
  • the first learning data set 12 includes an endoscope image having an angle of view relatively wider than the angle of view of the endoscope image to be subjected to image recognition.
  • a first model M1 that performs image recognition on an endoscope image having a wider angle than the target endoscope image is generated.
  • a CNN constituting the second model M2 is set based on the learned first model M1 (step S53).
  • the second learning is performed on the set CNN using the second learning data set 22 (step S54).
  • the second learning data set 22 includes an endoscope image having the same angle of view (including substantially the same angle of view) as the angle of view of the endoscope image to be subjected to image recognition.
  • an image recognition model (second model M2) optimized for use with a specific endoscope is generated.
  • ⁇ Learning based on learning results from a group of learning images captured by another endoscope> When generating a model that performs image recognition on an endoscope image captured by a specific endoscope, first learning is performed using an endoscope image group captured by another endoscope. Based on the result, the second learning is performed on an endoscope image group captured by an endoscope that performs image recognition.
  • FIG. 11 is a flowchart showing a learning procedure based on a learning result of a learning image group captured by different endoscopes.
  • the CNN constituting the first model M1 is set (step S61).
  • the first learning data set 12 includes an endoscope image captured by an endoscope (another endoscope) different from an endoscope that performs image recognition.
  • an endoscope that performs image recognition is an endoscope photographed by an endoscope having different specifications (image sensor size, image sensor resolution, image sensor type, imaging optical system configuration, light source type, etc.). It consists of.
  • a first model M1 that performs image recognition on the endoscope image captured by the other endoscope is generated.
  • the CNN configuring the second model M2 is set based on the learned first model M1 (step S63).
  • the second learning is performed on the set CNN using the second learning data set 22 (step S64).
  • the second learning data set 22 includes an endoscope image captured by the same endoscope (including an endoscope having the same model and the same specification) as the endoscope that performs image recognition.
  • a model capable of performing image recognition on the target endoscopic image is generated.
  • the learning method of this aspect even when the learning image of the endoscope to be image-recognized is insufficient, using the abundant other endoscope learning image group that exists abundantly, It is possible to efficiently generate a model for performing image recognition of an endoscope image captured by a specific endoscope.
  • an image recognition model of a specific endoscope can be efficiently generated even when there is an individual difference.
  • the functions of the first learning unit 10 and the second learning unit 20 are realized by the same computer, but the functions may be realized by a plurality of computers.
  • the functions of the first learning unit 10 and the second learning unit 20 can be realized by separate computers.
  • the model for performing image recognition is configured by CNN, but the configuration of the model for performing image recognition is not limited to this. Any model generated by machine learning may be used.
  • the CNN of the second model M2 is set by resetting the weight parameters of some layers of the CNN constituting the learned first model M1, but the second model M2 is set.
  • the technique is not limited to this.
  • a method of re-learning the entire CNN a method of performing the second learning by replacing the input layer and the output layer of the learned first model M1
  • Various methods such as a method of fixing weight parameters of some layers (for example, a layer for performing feature extraction) of the learned first model M1 and learning only other layers (for example, a layer for performing recognition) Can be adopted.
  • the learning coefficient may be changed in each layer.
  • the second learning may be performed by setting a learning coefficient larger than that in the other layers so that the learning proceeds faster.
  • the second learning including the setting of the second model can employ a so-called transfer learning (also referred to as fine tuning) method.
  • FIG. 12 is a block diagram illustrating an embodiment of the configuration of the endoscope image processing device.
  • the endoscope image processing device 100 is an example of a medical image processing device.
  • the endoscope image processing apparatus 100 acquires an endoscopic image having a specific image quality, and performs image recognition on the acquired endoscopic image (detection of a lesion included in the image, classification of each type of lesion, and the like). And output the result.
  • image recognition an image recognition model generated by the learning device 1 is used.
  • an endoscope image processing apparatus 100 includes an endoscope image acquisition unit 110 that acquires an endoscope image to be recognized, and an image recognition unit that performs image recognition on the acquired endoscope image. 112, a recognition result output unit 114 for outputting a recognition result, and an image processing control unit 116 for controlling the whole.
  • the endoscope image acquisition unit 110 is an example of a medical image acquisition unit, and acquires an endoscope image (medical image) to be recognized.
  • This endoscope image is an endoscope image having a specific image quality.
  • the image recognition unit 112 performs a process of image recognition (detection of a lesion included in the image, classification of each type of lesion, and the like) on the endoscopic image acquired by the endoscopic image acquiring unit 110.
  • the image recognizing unit 112 is configured by an image recognition model (learned model) generated by the learning device 1. Therefore, the first learning is performed with a learning image group (first endoscopic image group) having an image quality different from the target image quality, and based on the learning result, a learning image group (second image group) with the target image quality is obtained. (A group of endoscope images) and a model (second model) generated by learning.
  • the recognition result output unit 114 outputs the recognition result by the image recognition unit 112 in a predetermined format.
  • the data is output to a monitor in a predetermined display format.
  • the image processing control unit 116 controls the operation of each unit.
  • FIG. 13 is a diagram illustrating an example of a hardware configuration of the endoscope image processing device.
  • the endoscope image processing apparatus 100 is configured by a computer such as a server computer and a client computer, and includes a CPU 121, a ROM 122, a RAM 123, an HDD 124, a communication interface 125, an input / output interface 126, and the like.
  • the learning device 1 includes an input device 127, a display device 128, and the like.
  • the CPU 121 controls each unit of the endoscope image processing device 100 by executing the program, and realizes each function of the endoscope image processing device 100.
  • the ROM 122 stores various programs executed by the CPU 121, various data, and the like.
  • the RAM 123 provides the CPU 121 with a work area.
  • the HDD 124 stores various programs executed by the CPU 121 and various data.
  • the communication interface 125 is an interface for connecting the endoscope image processing device 100 to a network 59 such as a LAN.
  • the endoscope image processing device 100 communicates with an external device via the communication interface 125.
  • the input / output interface 126 is an interface for connecting external devices such as the input device 127 and the display device 128 to the endoscope image processing device 100.
  • the input device 127 inputs information according to a user operation to the endoscope image processing device 100.
  • the input device 127 includes, for example, a keyboard, a mouse, and the like.
  • the display device 128 displays various information.
  • the display device 128 includes, for example, a liquid crystal display, an organic EL display, and the like.
  • the functions of the endoscope image acquisition unit 110, the image recognition unit 112, and the recognition result output unit 114 are realized by the CPU 121 executing a predetermined program.
  • the endoscope image to be recognized is stored in, for example, the HDD 124 and acquired from the HDD 124. Alternatively, it is stored in an external storage device connected via the network 59, and acquired from the external storage device via the network 59. Alternatively, it is acquired via the network 59 from an endoscope device connected via the network 59.
  • the endoscope image acquisition unit 110 acquires an endoscope image to be recognized from a designated acquisition source under the control of the image processing control unit 116.
  • the recognition result is displayed on the display device 128 in a predetermined display format, for example.
  • the recognition result output unit 114 outputs the recognition result of the image recognition unit 112 to the display device 128 in a predetermined format under the control of the image processing control unit 116.
  • the endoscope image acquiring unit 110 acquires an endoscope image to be recognized.
  • This endoscope image is an endoscope image of a specific image quality.
  • the image recognition section 112 performs image recognition on the obtained endoscope image.
  • the recognition result output unit 114 outputs the recognition result.
  • image recognition is performed using a model optimized for a specific image quality, so that highly accurate image recognition can be performed.
  • FIG. 14 is a block diagram illustrating a modified example of the endoscope image processing device.
  • the endoscope image processing apparatus 100A of the present embodiment differs from the endoscope image processing apparatus 100 of the above-described embodiment in further including a model switching unit 130 for switching a model used for image recognition. I do.
  • the image recognition unit 112 is provided with a plurality of models for performing image recognition on the endoscope image, and the model to be used is switched by the model switching unit 130.
  • This model is a model optimized by performing the second learning.
  • the prepared plural models are stored in, for example, the ROM 122 or the HDD 124.
  • the model switching unit 130 switches a model to be used in accordance with an instruction from the image processing control unit 116.
  • the image processing control unit 116 switches the model to be used according to an instruction from the user.
  • the endoscope since the endoscope may have individual differences even with the same model, prepare a model optimized for each endoscope and use it for image recognition according to the endoscope used for the inspection. Switch models. This enables more accurate image recognition.
  • a processor device a device that processes image signals output from the endoscope and generates image data
  • a processor device a device that processes image signals output from the endoscope and generates image data
  • FIG. 15 is a block diagram illustrating another modified example of the endoscope image processing device.
  • the endoscope image processing apparatus 100 ⁇ / b> B of the present modification further includes an endoscope information acquisition unit 140 that acquires information of an endoscope that has captured an endoscope image of a recognition target. This is different from the image processing apparatus 100A.
  • a plurality of models that are optimized by the image recognition unit 112 for image recognition are prepared for each endoscope used for inspection.
  • the endoscope information acquisition unit 140 acquires information on the endoscope that has captured the endoscopic image to be recognized, and outputs the information to the image processing control unit 116.
  • the image processing control unit 116 instructs the model switching unit 130 to switch based on the acquired endoscope information so that the corresponding model is used.
  • the model switching unit 130 switches the model to be used according to an instruction from the image processing control unit 116. For example, a table in which the type (model) of the endoscope is associated with the corresponding model is prepared, and model switching is performed with reference to the table.
  • a model suitable for image recognition is automatically switched, so that high-precision image recognition is always possible.
  • a plurality of models to be switched are used in addition to a mode in which a plurality of models corresponding to the specifications of the endoscope are prepared, and a model generated by performing the second learning with a group of learning images having different resolutions from each other.
  • a model generated by performing the second learning with a group of learning images having different amounts of noise from each other, a model generated by performing a second learning with a group of learning images having different combinations thereof, and the like are prepared. . Then, an appropriate model is selected according to the application.
  • “Medical images” to which the present invention can be applied include, besides endoscope images, CT (Computerized Tomography) images, X-ray images, ultrasound diagnostic images, MRI (Magnetic Resonance Imaging) images, PET (Positron Emission Tomography).
  • CT Computerized Tomography
  • X-ray images X-ray images
  • ultrasound diagnostic images MRI (Magnetic Resonance Imaging) images
  • PET PET
  • SPECT Single Photon Emission Computed Tomography
  • fundus image a fundus image.
  • the medical image processing device of the present disclosure can be used as a diagnosis support device that supports medical examination, treatment, diagnosis, or the like by a doctor or the like.
  • diagnosis support includes the concept of consultation support and / or treatment support.
  • Hardware for realizing the learning device and the medical image processing device can be configured with various processors as described below.
  • processors include general-purpose processors that execute programs and function as various processing units, such as CPUs (Central Processing Units) and FPGAs (Field Programmable Gate Arrays).
  • CPUs Central Processing Units
  • FPGAs Field Programmable Gate Arrays
  • a dedicated electric circuit which is a processor having a circuit configuration specifically designed to execute a specific process such as a certain programmable logic device (Programmable Logic Device: PLD) or an ASIC (Application Specific Integrated Circuit), is included.
  • PLD programmable logic device
  • ASIC Application Specific Integrated Circuit
  • One processing unit may be configured by one of these various processors, or may be configured by two or more processors of the same type or different types.
  • one processing unit may be configured by a plurality of FPGAs or a combination of a CPU and an FPGA.
  • a plurality of processing units may be configured by one processor.
  • configuring a plurality of processing units with one processor first, as represented by a computer such as a client or a server, one processor is configured by a combination of one or more CPUs and software. There is a form in which a processor functions as a plurality of processing units.
  • SoC system-on-chip
  • a form using a processor that realizes the function of the entire system including a plurality of processing units with one integrated circuit (IC) chip is used.
  • IC integrated circuit
  • the various processing units are configured using one or more of the various processors described above as a hardware structure.
  • circuitry in which circuit elements such as semiconductor elements are combined.
  • Endoscope The endoscope is not limited to a flexible endoscope, but may be a rigid endoscope or a capsule endoscope.
  • observation light As the observation light (illumination light) of the endoscope, white light, light in one or a plurality of specific wavelength bands, or light in various wavelength bands according to the observation purpose such as a combination thereof is selected.
  • the white light is light in a white wavelength band or light in a plurality of wavelength bands.
  • the “specific wavelength band” is a band narrower than the white wavelength band. A specific example regarding a specific wavelength band will be described below.
  • a first example of the specific wavelength band is, for example, a blue band or a green band in a visible region.
  • the wavelength band of the first example includes a wavelength band of 390 nm or more and 450 nm or less or a wavelength band of 530 nm or more and 550 nm or less, and the light of the first example is within the wavelength band of 390 nm or more and 450 nm or less or 530 nm or more and 550 nm or less. It has a peak wavelength within the wavelength band.
  • a second example of the specific wavelength band is, for example, a red band in a visible region.
  • the wavelength band of the second example includes the wavelength band of 585 nm to 615 nm or the wavelength band of 610 nm to 730 nm, and the light of the second example is within the wavelength band of 585 nm to 615 nm or 610 nm to 730 nm.
  • the third example of the specific wavelength band includes a wavelength band in which the extinction coefficient differs between oxyhemoglobin and reduced hemoglobin, and the light of the third example peaks in a wavelength band in which the extinction coefficient differs between oxyhemoglobin and reduced hemoglobin. Having a wavelength.
  • the wavelength band of the third example includes a wavelength band of 400 ⁇ 10 nm, a wavelength band of 440 ⁇ 10 nm, a wavelength band of 470 ⁇ 10 nm, or a wavelength band of 600 nm or more and 750 nm or less. It has a peak wavelength in a wavelength band of 10 nm, 440 ⁇ 10 nm, 470 ⁇ 10 nm, or 600 nm or more and 750 nm or less.
  • the fourth example of the specific wavelength band is used for observation of fluorescence emitted from a fluorescent substance in a living body (fluorescence observation), and is a wavelength band of excitation light for exciting this fluorescent substance, for example, 390 nm to 470 nm.
  • a fifth example of the specific wavelength band is a wavelength band of infrared light.
  • the wavelength band of the fifth example includes a wavelength band of 790 nm or more and 820 nm or less, or a wavelength band of 905 nm or more and 970 nm or less, and the light of the fifth example is within the wavelength band of 790 nm or more and 820 nm or less or 905 nm or more and 970 nm or less. It has a peak wavelength within the wavelength band.
  • a laser light source As the type of the light source, a laser light source, a xenon light source, an LED light source (LED: Light-Emitting Diode), or an appropriate combination thereof can be adopted.
  • the type of light source, the wavelength, the presence or absence of a filter, and the like are preferably configured according to the type of subject, the purpose of observation, and the like.Also, at the time of observation, the wavelength of illumination light depends on the type of subject, the purpose of observation, and the like. It is preferable to combine and / or switch.
  • the wavelength of the light to be irradiated is switched by rotating a disk-shaped filter (rotary color filter) provided in front of the light source and provided with a filter that transmits or blocks light of a specific wavelength. You may.
  • the image sensor used for the endoscope is not limited to a color image sensor in which a color filter is provided for each pixel, but may be a monochrome image sensor.
  • a monochrome image sensor it is possible to sequentially change the wavelength of the illumination light to perform image capturing in a plane-sequential (color sequential) manner.
  • the wavelength of the emitted illumination light may be sequentially switched between violet, blue, green, and red, or may be irradiated with a broadband light (white light) and rotated by a rotary color filter (red, green, blue, etc.).
  • the wavelength of the emitted illumination light may be switched.
  • the wavelength of the illumination light emitted by the rotary color filter by irradiating one or a plurality of narrow band lights may be switched.
  • the narrow band light may be infrared light having two or more different wavelengths.
  • the processor device that processes the image of the endoscope may generate a special light image having information of a specific wavelength band based on the normal light image obtained by imaging using white light. Note that the generation here includes the concept of “acquisition”.
  • the processor unit 16 converts the signal of the specific wavelength band into red (R), green (G), and blue (B), cyan (Cyan, C), and magenta (Magenta, M) included in the normal light image. ), And can be obtained by performing an operation based on the color information of yellow (Yellow, Y).
  • a program for causing a computer to implement the functions of the learning device and the medical image processing device described in the above-described embodiment is recorded on an optical disk, a magnetic disk, or a computer-readable medium that is a non-transitory information storage medium such as a semiconductor memory or other tangible material.
  • a program it is possible to provide a program through this information storage medium.
  • the program signal can be provided as a download service using an electric communication line such as the Internet.
  • Reference Signs List 1 learning device 10 first learning unit 12 first learning data set 16 processor device 20 second learning unit 22 second learning data set 30 learning control unit 51 CPU 52 ROM 53 RAM 54 HDD 55 Communication interface 56 Input / output interface 57 Input device 58 Display device 59 Network 100 Endoscope image processing device 100A Endoscope image processing device 100B Endoscope image processing device 110 Endoscope image acquisition unit 112 Image recognition unit 114 Recognition result Output unit 116 Image processing control unit 121 CPU 122 ROM 123 RAM 124 HDD 125 Communication Interface 126 Input / Output Interface 127 Input Device 128 Display Device 130 Model Switching Unit 140 Endoscope Information Acquisition Unit M1 First Model M2 Second Model S1 to S4 Learning Procedures S11 to S14 Learning with High-Resolution Learning Image Group Learning Procedures Based on Results S21 to S24 Learning Procedures Based on Learning Results with Low-Resolution Learning Image Group S31 to S34 Learning Procedures Based on Learning Result with Low-Noise Learning Image Group From Learning Procedure S41 S44: Learning procedure based on the learning result

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Optics & Photonics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Endoscopes (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)
  • Image Analysis (AREA)

Abstract

La présente invention concerne un dispositif d'apprentissage et un procédé d'apprentissage au moyen desquels un modèle pour effectuer une reconnaissance d'image pour une image médicale ayant une qualité d'image spécifique peut être généré de manière efficace, ainsi qu'un dispositif de traitement d'image médicale. Un dispositif d'apprentissage (1) comporte : une première unité d'apprentissage (10) pour générer un premier modèle (M1) pour effectuer une reconnaissance d'image pour une image médicale ayant une première qualité d'image par apprentissage à l'aide d'un premier groupe d'images médicales constitué d'images médicales ayant la première qualité d'image ; et une seconde unité d'apprentissage (20) pour générer un second modèle (M2) pour effectuer une reconnaissance d'image pour une image médicale ayant une seconde qualité d'image par apprentissage à l'aide d'un second groupe d'images médicales constitué d'images médicales ayant la seconde qualité d'image, sur la base du premier modèle (M1) généré par la première unité d'apprentissage (10). Le second groupe d'images médicales est constitué d'images médicales ayant la même qualité d'image qu'une image médicale à reconnaître, et le premier groupe d'images médicales est constitué d'images médicales ayant une qualité d'image différente de la seconde qualité d'image. La seconde unité d'apprentissage (20) apprend sur la base d'un résultat d'apprentissage de la première unité d'apprentissage (10), et peut donc générer un modèle extrêmement précis même lorsque le second groupe d'images médicales est petit.
PCT/JP2019/022910 2018-06-28 2019-06-10 Dispositif d'apprentissage et procédé d'apprentissage, et dispositif de traitement d'image médicale Ceased WO2020003992A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2020527359A JP7382930B2 (ja) 2018-06-28 2019-06-10 医療画像処理装置

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2018123443 2018-06-28
JP2018-123443 2018-06-28

Publications (1)

Publication Number Publication Date
WO2020003992A1 true WO2020003992A1 (fr) 2020-01-02

Family

ID=68986461

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/022910 Ceased WO2020003992A1 (fr) 2018-06-28 2019-06-10 Dispositif d'apprentissage et procédé d'apprentissage, et dispositif de traitement d'image médicale

Country Status (2)

Country Link
JP (1) JP7382930B2 (fr)
WO (1) WO2020003992A1 (fr)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021156974A1 (fr) * 2020-02-05 2021-08-12 オリンパス株式会社 Dispositif de traitement d'image, procédé de traitement d'image, programme de traitement d'image, dispositif de commande d'affichage et dispositif endoscopique
JPWO2021240656A1 (fr) * 2020-05-26 2021-12-02
WO2022065504A1 (fr) * 2020-09-28 2022-03-31 ＬｅａｐＭｉｎｄ株式会社 Programme d'exécution de processus de reconnaissance et dispositif d'exécution de processus de reconnaissance
JP2022069737A (ja) * 2020-10-26 2022-05-12 キヤノン株式会社 情報処理システム、情報端末、情報処理方法及びプログラム
JPWO2022180786A1 (fr) * 2021-02-26 2022-09-01
CN115023171A (zh) * 2020-01-20 2022-09-06 奥林巴斯株式会社 学习用医疗图像数据生成装置、学习用医疗图像数据生成方法以及程序
JP2023011215A (ja) * 2021-07-12 2023-01-24 富士フイルム株式会社 画像処理装置及び医療画像処理装置
WO2023042273A1 (fr) * 2021-09-14 2023-03-23 日本電気株式会社 Dispositif de traitement d'image, procédé de traitement d'image, et support de stockage
JP2023043051A (ja) * 2021-09-15 2023-03-28 富士フイルム株式会社 画像処理装置、学習装置、画像処理方法、学習方法、画像処理プログラム、及び学習プログラム
CN116507270A (zh) * 2020-09-08 2023-07-28 全球先进临床解决方案有限公司 用于利用统计学分析和人工智能分析进行癫痫发作检测的系统和方法
CN116744854A (zh) * 2021-01-25 2023-09-12 株式会社岛津制作所 X射线透视摄影装置
WO2025009147A1 (fr) * 2023-07-06 2025-01-09 オリンパスメディカルシステムズ株式会社 Système, dispositif et procédé d'assistance à l'imagerie diagnostique
JP2025514224A (ja) * 2022-04-27 2025-05-02 ジャイラス エーシーエムアイ インク ディー／ビー／エー オリンパス サージカル テクノロジーズ アメリカ 複数の順序付けされた分類モジュールを有するシステム

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017175282A1 (fr) * 2016-04-04 2017-10-12 オリンパス株式会社 Procédé d'apprentissage, dispositif de reconnaissance d'image et programme
JP2017221486A (ja) * 2016-06-16 2017-12-21 ソニー株式会社 情報処理装置、情報処理方法、プログラム及び医療用観察システム

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008302075A (ja) 2007-06-08 2008-12-18 Olympus Corp 画像処理装置並びにスコープおよびこれらを備える内視鏡装置
JP6900190B2 (ja) 2016-01-14 2021-07-07 キヤノン株式会社 認識学習装置、認識学習方法及びプログラム
JP6750854B2 (ja) 2016-05-25 2020-09-02 キヤノン株式会社 情報処理装置および情報処理方法

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017175282A1 (fr) * 2016-04-04 2017-10-12 オリンパス株式会社 Procédé d'apprentissage, dispositif de reconnaissance d'image et programme
JP2017221486A (ja) * 2016-06-16 2017-12-21 ソニー株式会社 情報処理装置、情報処理方法、プログラム及び医療用観察システム

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
SONOYAMA, SHOJI ET AL.: "Transfer Learning for Bag-of-Visual words approach to NBI endoscopic image classification", 2015 37TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY, 25 August 2015 (2015-08-25), pages 785 - 788, XP032810308, DOI: 10.1109/EMBC.2015.7318479 *
SONOYAMA, SHOJI: "Transfer Learning for Endoscopic Image Classification", IEICE TECHNICAL REPORT, vol. 115, no. 401, January 2016 (2016-01-01), pages 223 - 227, XP055666694 *

Cited By (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115023171A (zh) * 2020-01-20 2022-09-06 奥林巴斯株式会社 学习用医疗图像数据生成装置、学习用医疗图像数据生成方法以及程序
WO2021156974A1 (fr) * 2020-02-05 2021-08-12 オリンパス株式会社 Dispositif de traitement d'image, procédé de traitement d'image, programme de traitement d'image, dispositif de commande d'affichage et dispositif endoscopique
US12137879B2 (en) 2020-02-05 2024-11-12 Olympus Corporation Image processing apparatus, image processing method, and endoscope apparatus
JPWO2021240656A1 (fr) * 2020-05-26 2021-12-02
WO2021240656A1 (fr) * 2020-05-26 2021-12-02 日本電気株式会社 Dispositif de traitement d'image, procédé de commande et support de stockage
US12530877B2 (en) 2020-05-26 2026-01-20 Nec Corporation Display candidate area information according to display mode determined for decision-making based on evaluation result by machine learning model
JP7529021B2 (ja) 2020-05-26 2024-08-06 日本電気株式会社 画像処理装置、制御方法及びプログラム
CN116507270A (zh) * 2020-09-08 2023-07-28 全球先进临床解决方案有限公司 用于利用统计学分析和人工智能分析进行癫痫发作检测的系统和方法
WO2022065504A1 (fr) * 2020-09-28 2022-03-31 ＬｅａｐＭｉｎｄ株式会社 Programme d'exécution de processus de reconnaissance et dispositif d'exécution de processus de reconnaissance
US12518516B2 (en) 2020-09-28 2026-01-06 Maxell, Ltd. Recognition process execution program and recognition process execution device
JP7630952B2 (ja) 2020-10-26 2025-02-18 キヤノン株式会社 情報処理システム、情報端末、情報処理方法及びプログラム
JP2022069737A (ja) * 2020-10-26 2022-05-12 キヤノン株式会社 情報処理システム、情報端末、情報処理方法及びプログラム
CN116744854A (zh) * 2021-01-25 2023-09-12 株式会社岛津制作所 X射线透视摄影装置
US12602779B2 (en) 2021-02-26 2026-04-14 Nec Corporation Image processing device, image processing method and storage medium
JP7485193B2 (ja) 2021-02-26 2024-05-16 日本電気株式会社 画像処理装置、画像処理方法及びプログラム
WO2022180786A1 (fr) * 2021-02-26 2022-09-01 日本電気株式会社 Dispositif et procédé de traitement d'image, et support de stockage
JPWO2022180786A1 (fr) * 2021-02-26 2022-09-01
JP2023011215A (ja) * 2021-07-12 2023-01-24 富士フイルム株式会社 画像処理装置及び医療画像処理装置
JP7653853B2 (ja) 2021-07-12 2025-03-31 富士フイルム株式会社 画像処理装置
WO2023042273A1 (fr) * 2021-09-14 2023-03-23 日本電気株式会社 Dispositif de traitement d'image, procédé de traitement d'image, et support de stockage
JPWO2023042273A1 (fr) * 2021-09-14 2023-03-23
JP2023043051A (ja) * 2021-09-15 2023-03-28 富士フイルム株式会社 画像処理装置、学習装置、画像処理方法、学習方法、画像処理プログラム、及び学習プログラム
JP7703408B2 (ja) 2021-09-15 2025-07-07 富士フイルム株式会社 画像処理装置、学習装置、画像処理方法、学習方法、画像処理プログラム、及び学習プログラム
US12524874B2 (en) * 2021-09-15 2026-01-13 Fujifilm Corporation Image processing device, learning device, image processing method, learning method, image processing program, and learning program that can accurately detect a lesion from a medical image using a lesion detection model
JP2025514224A (ja) * 2022-04-27 2025-05-02 ジャイラス エーシーエムアイ インク ディー／ビー／エー オリンパス サージカル テクノロジーズ アメリカ 複数の順序付けされた分類モジュールを有するシステム
WO2025009147A1 (fr) * 2023-07-06 2025-01-09 オリンパスメディカルシステムズ株式会社 Système, dispositif et procédé d'assistance à l'imagerie diagnostique

Also Published As

Publication number Publication date
JPWO2020003992A1 (ja) 2021-06-24
JP7382930B2 (ja) 2023-11-17

Similar Documents

Publication Publication Date Title
JP7382930B2 (ja) 医療画像処理装置
JP7346285B2 (ja) 医療画像処理装置、内視鏡システム、医療画像処理装置の作動方法及びプログラム
JP7801393B2 (ja) 光不足環境におけるツール追跡を伴うハイパースペクトル撮像
US11727542B2 (en) Super resolution and color motion artifact correction in a pulsed hyperspectral, fluorescence, and laser mapping imaging system
JP7383105B2 (ja) 医療画像処理装置及び内視鏡システム
JP7289373B2 (ja) 医療画像処理装置、内視鏡システム、診断支援方法及びプログラム
US12268469B2 (en) Medical imaging systems and methods that facilitate use of different fluorescence imaging agents
CN113994661A (zh) 具有固定模式噪声消除的超光谱、荧光和激光标测成像
WO2020054543A1 (fr) Dispositif et procédé de traitement d'image médicale, système d'endoscope, dispositif de processeur, dispositif d'aide au diagnostic et programme
CN114173641A (zh) 内窥镜荧光成像系统中的图像旋转
WO2020017213A1 (fr) Appareil de reconnaissance d'image d'endoscope, appareil d'apprentissage d'image d'endoscope, procédé d'apprentissage d'image d'endoscope et programme
US20250359760A1 (en) Medical imaging systems and methods
WO2019039259A1 (fr) Système d'aide au diagnostic, système d'endoscope, processeur et procédé d'aide au diagnostic
JP7335399B2 (ja) 医用画像処理装置及び内視鏡システム並びに医用画像処理装置の作動方法
WO2019220801A1 (fr) Dispositif de traitement d'image d'endoscope, procédé de traitement d'image d'endoscope et programme
CN114175620A (zh) 内窥镜激光标测成像系统中的图像旋转
WO2021044590A1 (fr) Système d'endoscope, système de traitement, procédé de fonctionnement de système d'endoscope et programme de traitement d'image
CN214231268U (zh) 内窥镜成像装置和电子设备
US20240212104A1 (en) Systems and methods for low-light image enhancement
JP7593796B2 (ja) 医用画像処理システム、医用画像処理システムの作動方法
KR101281802B1 (ko) 감량 영상 구성장치
US20240013392A1 (en) Processor device, medical image processing device, medical image processing system, and endoscope system
JP2023011303A (ja) 医療画像処理装置及びその作動方法
WO2024166306A1 (fr) Dispositif médical, système d'endoscope, procédé de commande, programme de commande et dispositif d'apprentissage

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19825158

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020527359

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19825158

Country of ref document: EP

Kind code of ref document: A1