WO2004036283A1 - 顕微鏡画像処理システム、顕微鏡画像処理方法、プログラム、および、記録媒体 - Google Patents
顕微鏡画像処理システム、顕微鏡画像処理方法、プログラム、および、記録媒体 Download PDFInfo
- Publication number
- WO2004036283A1 WO2004036283A1 PCT/JP2003/011367 JP0311367W WO2004036283A1 WO 2004036283 A1 WO2004036283 A1 WO 2004036283A1 JP 0311367 W JP0311367 W JP 0311367W WO 2004036283 A1 WO2004036283 A1 WO 2004036283A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- information
- image
- individual
- shooting
- magnification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
- G02B21/365—Control or image processing arrangements for digital or video microscopes
Definitions
- the present invention relates to a microscope image processing system, a microscope image processing method, a program, and a recording medium, and in particular, to a microscope image processing system capable of automating microscope image capturing as intended by an operator, and a microscope image processing. Methods, programs, and recording media. Background art
- the conventional system has many problems, and as a result, it is inconvenient for both the user and the administrator of the image analysis system using the microscope, and> The usage efficiency was poor.
- the problem to be solved by the conventional technology described above is not limited to the case where a specimen or the like arranged on a multi-well plate or the like is photographed with a microscope. Similar considerations can be made for this system.
- the present invention has been made in view of the above problems, and provides a microscope image processing system, a microscope image processing method, a program, and a recording medium that can automate the photographing of a microscope image as intended by an operator. It is intended to provide. Disclosure of the invention
- a microscope image processing system includes: a batch photographing unit that collectively photographs each specimen holding unit of a specimen holder at a predetermined magnification using a microscope; A thumbnail display means for displaying a thumbnail of the image file photographed by the photographing means; and An individual imaging information specifying means for specifying individual imaging information relating to a position and a magnification; and individual imaging means for individually imaging the inside of the specimen holding unit at the position and the magnification specified by the individual imaging information.
- each specimen holder for example, each well of a multiwell plate
- a specimen holder for example, a multiwell plate, etc.
- the image file that was shot is displayed as a thumbnail, and the image file selected by the user from the displayed image files is individually associated with the position and magnification of the sample that the user wants to shoot. Since the imaging information is specified, and the inside of the sample holding unit is individually imaged at the position and the magnification specified by the individual imaging information, it is possible to automate the imaging of a desired location of each cell in the plate.
- a place to which the user pays attention may be preferentially photographed simultaneously or concurrently with simultaneous photographing of the inside of each well.
- a microscope image processing system is the microscope image processing system according to the above, wherein the individual imaging information designating means is a monitor for displaying an individual imaging information designation screen for allowing a user to designate the individual imaging information. And displaying the individual photographing information from the coordinate information and the magnification information designated by the user on the individual photographing information designation screen via the input device.
- the individual imaging information designating means is a monitor for displaying an individual imaging information designation screen for allowing a user to designate the individual imaging information. And displaying the individual photographing information from the coordinate information and the magnification information designated by the user on the individual photographing information designation screen via the input device.
- an individual shooting information designation screen for allowing the user to specify individual shooting information is displayed on the monitor, and the coordinate information specified by the user on the individual shooting information designation screen via the input device is provided. Since the individual shooting information is created from the magnification information and the magnification information, the user can specify the desired position and perform the individual shooting.
- Various input support functions such as the well-known map 'viewing function (keys to move in 8 directions and 16 directions, magnification setting buttons, etc.) are displayed on the individual shooting information designation screen displayed on the monitor by this system.
- the user may specify individual shooting information using such a function.
- a microscope image processing system is the microscope image processing system described above, wherein the individual imaging information designating means automatically determines coordinate information and / or magnification information for imaging by an image processing method.
- the above individual shooting information It is characterized in that it is created.
- individual coordinate information is created by automatically determining coordinate information Z or magnification information to be captured by an image processing method.
- an image processing method for example, a known image processing method (for example, calculation of a center of gravity, etc.)
- the shooting position for individual shooting can be automatically determined.
- the present invention relates to a microscope image processing method, and a microscope image processing method according to the present invention relates to a method of collectively photographing each sample holding section of a sample holder at a predetermined magnification using a microscope.
- a photographing step, a thumbnail display step for displaying a thumbnail of the image file photographed in the batch photographing step, and a user photographing the image file selected by the user from the image file displayed in the thumbnail display step An individual imaging information specifying step of specifying individual imaging information relating to the position and magnification of a sample to be performed; and an individual imaging step of individually imaging the inside of the sample holding unit at the position and magnification specified by the individual imaging information. It is characterized by including.
- each sample holder for example, each plate of a multi-well plate
- a sample holder for example, a multiwell plate
- the captured quotation file is displayed as a thumbnail, and the image file selected by the user is displayed from the displayed image file.
- the individual imaging information relating to the ratio is specified, and the inside of the sample holding unit is individually imaged at the position and magnification specified by the individual imaging information, so that the imaging of the desired location in each cell on the plate can be automated Become like
- the user may preferentially photograph a place of interest at the same time or concurrently with photographing the inside of each cell collectively.
- a microscope image processing method is the microscope image processing method described above, wherein the individual photographing information designating step comprises: transmitting the individual photographing information to a user. Displaying an individual photographing information designation screen for designating the individual photographing information from the coordinate information and the magnification information designated on the individual photographing information designation screen by the user via the input device; And
- an individual shooting information specifying screen for allowing the user to specify individual shooting information is displayed on the monitor, and the coordinate information and magnification information specified by the user on the individual shooting information specifying screen via the input device. Since the individual shooting information is created from the user, the user can specify the desired position and perform the individual shooting.
- various inputs such as the known map 'viewing function (keys to move in 8 directions or 16 directions, magnification setting change buttons, etc.) It may have a support function, and the user may specify individual shooting information using such a function.
- the individual photographing information designating step automatically determines coordinate information to be photographed and z or magnification information by an image processing method. Then, the individual photographing information is created.
- individual coordinate information is created by automatically determining coordinate information and / or magnification information to be captured by an image processing method. Therefore, for example, individual image capturing is performed by a known image processing method (for example, calculation of a center of gravity). The shooting position can be automatically determined.
- the present invention relates to a program, and the program according to the present invention includes a batch photographing step of photographing each specimen holder of the specimen holder at a predetermined magnification at a time using a microscope; A thumbnail display step of displaying a thumbnail of the image file photographed in the step; and a sample of the sample that the user wants to photograph for the image file selected by the user from the image file displayed in the thumbnail display step. Specify the individual shooting information for the position and magnification. And an individual imaging step of individually imaging the specimen holding unit at the position and magnification designated by the individual imaging information.
- each sample holder eg, each well of a multi-well plate
- a sample holder eg, a manhole plate
- the thumbnails of the captured image files are displayed.
- Individual imaging information relating to the position and magnification of the sample that the user wants to capture for the image file selected by the user from the displayed image files Is designated, and the specimen holder is individually photographed at the position and magnification designated by the individual photographing information, so that photographing of a desired place in each well in the plate can be automated.
- a place where the user pays attention may be preferentially photographed at the same time as or simultaneously with photographing each LELL.
- the program according to the next invention is the program according to the above, wherein the individual photographing information designation step displays an individual photographing information designation screen for allowing a user to designate the individual photographing information on a monitor, and the user
- the individual photographing information is created from coordinate information and magnification information specified on the individual photographing information specifying screen via an input device.
- an individual shooting information specifying screen for allowing the user to specify individual shooting information is displayed on the monitor, and the coordinate information and magnification specified by the user on the individual shooting information specifying screen via the input device. Since the individual shooting information is created from the information, the user can specify the desired position and perform the individual shooting.
- the program according to the next invention is the program according to the above, wherein the individual photographing information designating step is to automatically determine coordinate information and / or magnification information to be photographed by an image processing method to create the individual photographing information. It is characterized by doing.
- coordinate information to be photographed and Z or magnification information are automatically determined by an image processing method and individual photographing information is created.
- a known image processing method for example, calculation of a center of gravity, etc. This enables the shooting position of individual shooting to be automatically determined.
- the present invention relates to a recording medium, and the recording medium according to the present invention has the program described above recorded thereon.
- the program described above can be realized by using a computer by causing a computer to read and execute the program recorded on the recording medium. The same effect can be obtained.
- FIG. 1 is a principle configuration diagram showing a system configuration of the present invention.
- FIG. 2 is a block diagram showing an example of a system configuration of an image analysis device 100 to which the present invention is applied.
- FIG. 4 is a block diagram illustrating an example of a system configuration of an image management device 20.0 to which the present invention is applied.
- FIG. 4 is a flowchart illustrating an example of a main process of the present system in the present embodiment. Yes, FIG. 5 is a diagram showing an example of the configuration of the present system in the present embodiment, and
- FIG. 6 is a diagram showing an example of a shooting setting screen of an “image view” (user interface) in the present embodiment.
- FIG. 7 is a diagram showing an example of a processing outline of a macro program in the present embodiment.
- FIG. 1 is a principle configuration diagram showing a system configuration of the present invention.
- FIG. 2 is a block diagram showing an example of a system configuration of an image analysis device 100 to which the present invention is applied.
- FIG. 8 is a diagram showing an example of a processing outline of an image view II in the embodiment. Is the initial setting in this embodiment.
- the processing performed by the macro program FIG. 10 is a flowchart showing an example.
- FIG. 10 is a flowchart showing an example of processing executed by the calibration macro program in the present embodiment.
- FIG. 11 is a photographing macro program in the present embodiment.
- FIG. 12 is a flowchart showing an example of a process executed by the image viewer according to the present embodiment. 1 3 figures in the present embodiment, a flow chart illustrating an example of processing performed by the image-bi Yuwa during individual shooting, the first 4 figures in this embodiment, the image during Kiyari play Deployment run
- FIG. 15 is a flowchart showing an example of the processing executed by the viewer II.
- FIG. FIG. 16 is a flowchart showing an example of a process executed by the image view ⁇ at the time of executing the embedded image processing.
- FIG. 17 is a flowchart showing an example of a cooperative operation at the time of calibration setting in the present embodiment.
- FIG. 18 is a chart showing a low magnification in the present embodiment.
- FIG. 19 is a processing sequence showing an example of a cooperative operation at the time of batch photographing.
- FIG. 19 is a processing sequence showing an example of a cooperative operation at the time of individual photographing in this embodiment.
- FIG. 21 is a flowchart illustrating an example of a calibration initial setting process of the present system according to the present embodiment.
- FIG. 21 is a flowchart illustrating an example of a batch photographing process of the present system according to the present embodiment.
- FIG. 22 shows an example of a thumbnail display screen displayed on the monitor.
- FIG. 23 shows another example of an individual shooting information designation screen displayed on the monitor.
- FIG. 24 is a diagram showing an example of a thumbnail list screen of an image view (user interface) in the present embodiment.
- FIG. 25 is an image view of the image view (user interface) in the present embodiment.
- FIG. 26 is a diagram showing an example of a full-size image screen of the user interface).
- FIG. 26 is a diagram showing an example of two captured calibration image files.
- FIG. 28 shows an example of an image file of two caliper images.
- FIG. 29 is a diagram illustrating an example of a case where image files are superimposed and displayed.
- FIG. 29 is a diagram illustrating an example of an individual shooting information designation screen displayed on the monitor.
- the present invention is not limited to this case. The same can be applied to all cases of shooting with.
- FIG. 1 is a principle configuration diagram showing a system configuration of the present invention, and conceptually shows only a portion related to the present invention in the configuration.
- the microscope image processing system of the present invention generally includes an image analysis device 100 that controls a microscope device 400 to capture a microscope image, and an image management device 20. 0 and are communicably connected via a network 300 (however, the surface image analysis device 100 and the image management device 100 are configured to be realized in the same housing. ;).
- the image analysis device 100 collectively photographs the inside of each cell at a low magnification.
- the image management device 200 communicates with the image analysis device 100 via the network 300, acquires a low-magnification image that has been captured, and displays a thumbnail image.
- the operator specifies the position of the sample to be imaged for each well on the image through the image management device 200.
- the image management device 200 notifies the image analysis device 100 of a shooting instruction according to the specified condition.
- the image analysis device 100 stores the image capturing conditions specified by the image management device 200 Shoot individually with.
- the image analysis apparatus 100 may collectively photograph the inside of each cell at an arbitrary magnification (for example, a high magnification), not limited to a low magnification.
- an arbitrary magnification for example, a high magnification
- the image analysis apparatus 100 may preferentially photograph a place of interest of the worker simultaneously or concurrently with photographing the inside of each well collectively.
- the image analyzing apparatus 100 may, for example, combine the shot images and shoot the entire image at a high resolution.
- the image management device 200 can display images captured at a high resolution by, for example, a well-known map viewing function (for example, a well-known map reference function (for example, keys for moving in eight directions and sixteen directions, Input and display may be performed using any input support function according to the shooting target and search purpose, such as a button for changing the magnification setting.
- a well-known map viewing function for example, a well-known map reference function (for example, keys for moving in eight directions and sixteen directions, Input and display may be performed using any input support function according to the shooting target and search purpose, such as a button for changing the magnification setting.
- a well-known map viewing function for example, a well-known map reference function (for example, keys for moving in eight directions and sixteen directions, Input and display may be performed using any input support function according to the shooting target and search purpose, such as a button for changing the magnification setting.
- a well-known map viewing function for example, a well-known map reference function (for example, keys for moving in
- FIG. 2 is a block diagram showing an example of a system configuration of the image analysis apparatus 100 to which the present invention is applied, and conceptually shows only a portion of the configuration relating to the present invention.
- the image analysis device 100 includes a control unit 102 such as a CPU for generally controlling the entire image analysis device 100, a router connected to a communication line, and the like.
- Communication control interface 104 connected to communication devices (not shown), input / output control interface 108 connected to input devices 112 and output devices 114, and various databases
- a storage unit 106 for storing tables and the like, and these units are communicably connected via an arbitrary communication path.
- the image analysis device 100 is communicably connected to a network 300 via a communication device such as a router and a wired or wireless communication line such as a dedicated line.
- Various databases and tables (image file information database 106a to calibration initial setting file 106e) stored in the storage unit 106 are storage means such as a fixed disk device. Stores various programs, tables, files, databases, web page files, etc. used for processing.
- the image file information database 106a stores image files (including image IDs, image data, etc.) and image information (image IDs, plate IDs, and cell IDs). , Coordinate information, magnification information, organism name, etc.).
- the system setting information file 1 b contains microscope hardware configuration information (information on the magnification of the objective lens installed), shooting parameter information (exposure time, brightness, light source voltage, etc. for each objective lens). This is a method for storing system setting information that stores information about the parameters of the above, and plate format information (information about the arrangement of multiwell plates, information about the distance between wells, etc.). '
- the status information file 106c stores the work status of each plate (during shooting, waiting for a shooting instruction, etc.), image information for a shot image file, and the like.
- the individual shooting information file 106 d sets various conditions for individual shooting, such as how to shoot individual coordinates using which objective lens and how many times, and under what name. For storing individual shooting information. ⁇
- the calibration initial setting file 106 e stores a calibration image file and a calibration information file.
- a communication control interface unit 104 performs + communication control between the image analysis device 100 and the network 300 (or a communication device such as a router). That is, the communication control interface unit 104 has a function of communicating data with another terminal via a communication line.
- an input / output control interface unit 108 controls the input device 112 and the output device 114.
- the output device 114 in addition to a monitor (including a home television), a speaker can be used (hereinafter, the output device 114 may be described as a monitor).
- a keyboard, a mouse, a microphone, or the like can be used as the input device 112.
- the monitor also realizes a pointing device function in cooperation with the mouse. '
- the control unit 102 includes a control program such as an OS (Operating System), a program defining various processing procedures, and an internal memory for storing required data. Then, information processing for executing various processes is performed by these programs and the like.
- the control unit 102 is functionally conceptualized as a batch shooting unit 102 a, individual shooting unit 102 b, microscope stage initial setting unit 102 c, calibration initial setting file creation unit 10 2 d and a status management unit 102 e.
- the batch photographing unit 102a uses a microscope to set each specimen holder (for example, each well of a multiwell plate) of a specimen holder (for example, a multiwell plate) at a predetermined magnification (lower This is a batch photographing means for photographing all at once with any one of magnification or high magnification.
- the individual imaging unit 102b is an individual imaging unit that individually images the inside of the specimen holding unit (for example, each well of a multiwell plate) at a position and magnification specified by the individual imaging information.
- the microscope stage initial setting unit 102 c is a microscope stage initial setting unit that obtains the center position of the jewel coordinates of the multi-well plate for high predication and sets the center position as the coordinate origin.
- the calibration initial setting file creating section 102d is a calibration initial setting file creating means for creating a calibration initial setting file.
- the status management unit 102 e is a status management unit that manages status information indicating the progress of work on each plate.
- FIG. 3 is a block diagram showing an example of a system configuration of the image management apparatus 200 to which the present invention is applied, and conceptually shows only a part related to the present invention in the configuration.
- an image management device 200 is a control unit 202 such as a CPU that controls the entire image management device 200 as a whole, a router connected to a communication line, and the like.
- Communication interface unit 204 connected to the communication device (not shown), input / output control interface unit 208 connected to the input device 212 and the output device 214, and various databases And a storage section 206 for storing tables and the like, and these sections are communicably connected via an arbitrary communication path.
- the image management device 200 is communicably connected to a network 300 via a communication device such as a router and a wired or wireless communication line such as a dedicated line.
- Various databases and tables (individual photographing information file 206a) stored in the storage unit 206 are storage means such as a fixed disk device, and are used to store various programs (tape files and file files) used for various processes. "Stores files for database pages.
- the individual imaging information file 206a contains individual imaging information specified by the user (for example, including the position (coordinate information) of the sample to be imaged and the imaging magnification, etc.). ) Is a database that stores
- a communication control interface unit 204 performs communication control between the image management device 200 and the network 300 (or a communication device such as a router). That is, the communication control interface unit 204 has a function of communicating data with another terminal via a communication line.
- an input / output control interface unit 208 controls the input device 212 and the output device 214.
- the output device 214 a speaker can be used in addition to a motor (including a home television) (the output device 214 may be described as a monitor in the following).
- a keyboard, a mouse, a microphone, or the like can be used as the input device 211. The monitor also realizes the pointing device function in cooperation with the mouse.
- control unit 202 has a control program such as an operating system (OS), a program that defines various processing procedures, and an internal memory for storing required data. Then, information processing for executing various processes is performed by these programs and the like.
- the control unit 202 is configured to include a thumbnail display unit 202 a, an individual shooting information designating unit 202 b, and a calibration initial setting unit 202 c in terms of function concept. .
- the thumbnail display section 202a is a thumbnail display section for displaying a thumbnail of the image file shot by the batch shooting section.
- the individual imaging information designating section 202 b is an individual imaging information designating section which designates individual imaging information relating to a position and a magnification of a sample which the user wants to photograph for an image file selected by the user from the image files displayed on the thumbnail display means. This is a means for specifying shooting information.
- the individual shooting information specifying section 202b displays an individual shooting information specifying screen for allowing the user to specify the individual shooting information on the monitor, and the user selects the individual shooting information specifying screen via the input device.
- Individual imaging information may be created from the coordinate information and magnification information specified above.
- the individual shooting information designating unit 202b automatically determines coordinate information and Z or magnification information to be shot by a known image processing method (for example, calculation of a center of gravity, etc.) and creates individual shooting information. May be.
- the calibration initial setting unit 202c is a calibration initial setting unit that determines a parameter for converting a pixel on the image view II into a stage movement amount. The details of the processing performed by these units will be described later.
- the network 300 has a function of interconnecting the image analysis device 100 and the image management device 200, and is, for example, the Internet.
- FIG. 20 is a flowchart showing an example of a calibration initial setting process of the present system in the present embodiment.
- the image analysis device 100 obtains the center position of the jewel coordinates, which is the reference of the multi-well plate, by the processing of the microscope stage initial setting unit 102c, and performs the work of setting the coordinates as the origin (step SB).
- the microscope stage initial setting unit 102c sets the center point of the reference level (for example, the first level) as the origin. And set.
- the image analysis device 100 creates a calibration initialization file by the processing of the calibration initialization file creating section 102d (step SB-2). Specifically, when the user sets a multi-wall plate for calibration initial setting (for example, a plate in which an adjustment image such as a crosshair is inserted at the center of the first tool) in the microscope apparatus 400, The calibration initialization file creator 102 d captures two images with the X and y coordinates shifted for the adjustment image. Then, the calibration initial setting file creation unit 102 d generates two captured image files (calibration image files) and calibration information files such as coordinate information. Is created as a calibration initialization file 106 e.
- FIG. 26 and FIG. 27 are diagrams showing an example of two captured calibration image files.
- the two images shown in Fig. 26 and Fig. 27 are 3 mm in the X direction of the calibration initial setting multiwall plate with the same calibration initial setting file creation unit 102d. This is an image taken 3 mm in the y direction.
- the initial calibration file creation unit 102d stores the file names of these two images, the magnification of the objective lens used for shooting, and information about the actual distance between the two images.
- Setting file 106 Store in the calibration information file of e.
- the image management device 200 receives the calibration initial setting file 106 e from the image analysis device 100 by the processing of the calibration initial setting section 202 c. (Step SB-3).
- the image management device 200 displays the two calibration image files included in the acquired calibration initial setting file in a superimposed manner by the processing of the calibration initial setting unit 202 c,
- the displacement distance (the number of display pixels) between the two adjustment images is measured (step SB-4).
- FIG. 28 is a diagram showing an example of a case where two calibration image files are displayed in an overlapping manner.
- the calibration initial setting section 202 Measure the number of pixels between the two center points. .
- the calibration initial setting unit 202c automatically or manually moves one image file so that the display positions of the crosshairs of the two image files match, and measures the number of pixels at that distance. May be.
- the image management device 200 performs the processing of the calibration initial setting unit 202c to determine the distance between the two adjustment images (the number of display pixels) and the calibration information.
- Difference between X and Y coordinates stage movement distance
- the ratio between the number of display pixels and the stage movement distance is calculated based on the ratio, and the value of the ratio is converted from the number of pixels on the image view to the stage movement distance.
- the data is stored in a predetermined storage area (step SB-5). This completes the calibration initial setting process.
- FIG. 4 is a flowchart showing an example of a main process of the present system in the present embodiment.
- the image analyzer 100 collectively photographs each cell at a low magnification using the microscope device 400 by the processing of the collective photographing unit 102a (step SA-1).
- FIG. 21 is a flowchart showing an example of the batch photographing process of the present system in the present embodiment. .
- the image analysis apparatus 100 displays a plate ID input screen on the monitor by the processing of the status management unit 102e, and prompts the user to input the plate ID (step SC-1). Then, the image analysis device 100 stores the plate ID input by the user in a predetermined storage area of the status information file 106c by the processing of the status management unit 102e.
- the image analysis apparatus 100 accesses the status information file 106c by the processing of the status management unit 102e, and updates the status information of the plate ID to “first batch shooting” ( Step SC—2).
- the image analysis apparatus 100 performs low-magnification photographing of each well in order from the first well of the multi-well plate set in the microscope apparatus 400 by the processing of the batch photographing unit 102a. Step SC—3).
- the image analysis apparatus 100 determines whether or not all the wells have been shot when one or a predetermined number of shots have been completed by the processing of the batch shooting unit 102a.
- Judge (Step SC-4).
- the status management unit 102 e accesses the status information file 106 c and updates the status information of the plate ID to “summary batch imaging”. Update to (Step SC-5), return to Step SC-3, and continue taking the next level.
- step SC-4 when the imaging of all the wells is completed in step SC-4, the image analysis apparatus 100 accesses the status information file 106 c by the processing of the status management unit 102 e, and The status information of the plate ID is updated to “Batch shooting completed” (step SC-6).
- the status information of the plate ID is "Restart batch shooting", so that the next batch shooting process is executed. Only unphotographed jewels can be taken.
- the image management device 200 processes the image file and image information taken from the image analysis device 100 via the network 300 by the processing of the thumbnail display portion 202a. (Step SA-2) and display a thumbnail on the monitor (Step SA-3).
- FIG. 22 is a diagram showing an example of a thumbnail display screen displayed on the monitor.
- the thumbnail display screen consists of a plate ID input area MA-1, a plate ID selection button MA-2, a thumbnail image acquisition button MA_3, a thumbnail image display area MA-4, and an individual shooting button. Including MA-7.
- the user inputs a desired plate ID into the plate ID input area MA-1 via the input device 2 1 2 and selects the thumbnail image acquisition button MA-3 by clicking the mouse or the like with the mouse or the like. Then, the thumbnail display section 202a displays, in the thumbnail image display area MA-4, a low-magnification image (MA-5) of each batch image corresponding to the plate ID and related information such as a file ID. (MA-6) is displayed.
- a low-magnification image MA-5 of each batch image corresponding to the plate ID and related information such as a file ID.
- the thumbnail display section 202 a displays a window for displaying a list of plate IDs. open.
- the image management apparatus 200 allows the user to select a desired image file from the image files displayed as thumbnails on the monitor by the processing of the individual shooting information specifying unit 202b. .
- the image management device 200 causes the user to specify individual imaging information relating to the position of the sample to be imaged and the imaging magnification for the image file selected by the user by the processing of the individual imaging information specifying unit 202 b. Display screen for specifying individual shooting information on the monitor (step SA-4).
- FIG. 29 is a diagram showing an example of the individual shooting information designation screen displayed on the monitor.
- the individual shooting information designation screen has a display area MG-1 for the image selected by the user, a display area MG-2 for the information related to the image file such as the plate ID and the ID, and the selection area.
- the shooting frame list display area MG-3 on the left side of the screen displays a list of individual shots set in the selected image file.
- shooting frame information (MG-9 to MG-12) of each shooting is superimposed and displayed on the image display area.
- the user can check the list of the shooting frame information of the set individual shooting for each column.
- the user can select an item corresponding to desired shooting frame information from the shooting frame list display area MG-3, and any of shooting frame information (MG-9 to MG-12) in the image display area (MG-1).
- the shooting condition and position information related to the shooting frame information of the selected individual shooting are displayed in the shooting condition display area MG-4 and the shooting position information display area MG-5 on the right side of the screen, respectively. .
- the shooting frame information for four individual shots (10 times X 1 (MG-11), 20 times X 2 (MG-9 and MG-10), 40 times x 1 (MG-11)) has been set, and frame 1 (MG-9) has been selected by the user.
- the user can arbitrarily change the shooting condition information by selecting and editing the shooting condition display area MG-4 and the shooting position information display area MG-5 while the shooting frame information for each individual shooting is selected.
- the photographing position is changed, it can be changed by directly moving the photographing frame information by dragging or the like on the image display area (MG-1).
- the user can delete desired shooting frame information by selecting the individual shooting delete button MG-6 with the shooting frame information selected.
- a new shooting is performed by editing the shooting condition display area MG14 and selecting the placement button MG-7 or the automatic placement button MG-8.
- Frame information can be created.
- the user selects the arrangement button MG-7, the user can specify a desired shooting position on the image.
- the user selects the automatic distribution button MG-8, the photographing frame information is automatically arranged at the position of the center of gravity of the desired image using the existing image processing method.
- the shooting conditions for each individual shooting can be freely set for each shooting frame information via the shooting condition display area MG-4.
- the “objective lens magnification” of the photographing condition display area MG-4 is used to select the objective lens magnification used for photographing.
- the image analysis apparatus 100 switches to the predetermined objective lens designated here when performing individual imaging, and performs imaging.
- “Focus” in the shooting condition display area MG-4 is used to select whether to automatically perform the Z-axis adjustment or manually.
- the image analysis apparatus 100 captures an image while shifting the Z-axis, finds an optimal Z coordinate, and executes photographing.
- the automatic force function is provided in many commercially available microscope devices and its attached image analysis device, Function may be used.
- the image analyzer 100 moves to the predetermined X and Y coordinates at the time of executing the individual shooting, then temporarily stops and sets the focus to the user. Display a prompt message.
- the user After the user operates the microscope apparatus 400 to adjust the Z coordinate, the user notifies the image analysis apparatus 100 of "resume individual imaging".
- Shooting condition display area MG-4 “Exposure” sets the exposure time at the time of shooting, and determines whether the exposure is determined by “Auto” or that the “specified” exposure time is used. it can.
- the “photographing position” of the photographing condition display area MG-4 uses the distance from the origin as a coordinate value, and is displayed in micrometer units in the example of FIG.
- FIG. 23 is a diagram showing another example of the individual shooting information designation screen displayed on the monitor.
- the individual shooting information designation screen has a display area MB—1 for the image file selected by the user, and a display area MB—for the information related to the image file such as the plate ID and the ID.
- Magnification selection button MB-3 for specifying the magnification for individual shooting
- Wenore shooting button MB-4 Auto button MB-5 for selecting automatic shooting mode
- MB-6 for selecting a semi-automatic shooting mode that can perform the operation
- canceller button MB_7 for canceling the processing.
- the individual photographing information specifying section 202 b is set to the image file.
- the shooting frame information corresponding to the selected magnification is superimposed and displayed on the display area MB-1 of.
- two pieces of photographing frame information MB_8 corresponding to 40 times and one piece of photographing frame information MB-9 corresponding to 20 times are displayed. Then, the user can specify the shooting position (coordinate information) by moving the shooting frame information displayed via the input device 211.
- the individual photographing information setting unit 202b is operated by the user. Creates the specified shooting position (coordinate information), magnification information, and shooting mode information indicating either automatic shooting or semi-automatic shooting as individual shooting information.
- shooting can be performed based on the individual shooting information for the screen: c displayed on the screen. That is, before performing individual imaging based on the individual imaging information specified for all the cells, it is possible to perform real-time imaging based on the individual imaging information specified for the displayed tools.
- the image management device 200 notifies the image analysis device 100 of the individual shooting information by the processing of the individual shooting information designating section 202b (step S A-5). 'After this step, if the level shooting button MB-4 is selected in Fig. 23, it is executed for each specified level. On the other hand, if the key shooting button MB-7 is not selected, after the individual shooting information is set for each cell, the selection of the individual shooting button MA-7 shown in FIG. This is executed for all the cells for which the individual shooting information is set.
- the image analyzing apparatus 100 performs the processing of the individual photographing unit 102b to specify the position designated by the individual photographing information notified from the image managing apparatus 200 at the designated magnification by the designated magnification.
- the microscope device 400 is controlled according to the photographing conditions such as the photographing mode (automatic, semi-automatic), etc. to photograph individually (step SA-6).
- the image analyzer 100 may collectively photograph the inside of each well at an arbitrary magnification (for example, a high magnification), not limited to the low magnification.
- an arbitrary magnification for example, a high magnification
- the image analysis device 100 may preferentially photograph a place of interest of a worker simultaneously or concurrently with photographing the inside of each cell at once.
- the image analysis device 100 may, for example, combine the photographed wells and photograph the entire well at a high resolution.
- the image management device 200 can display an image captured at a high resolution, for example, by using a known map viewing function (for example, a well-known map reference function (for example, a key that moves in 8 directions and 16 directions, Click to change the magnification setting)))
- a known map viewing function for example, a well-known map reference function (for example, a key that moves in 8 directions and 16 directions, Click to change the magnification setting)
- Any input support depending on the search purpose Input and display may be performed using various functions. For example, when observing and photographing a brain slice fixed to each well of a microplate, it is necessary to look at the whole image of the slice. , Will be able to be synthesized automatically. This ends the main processing.
- FIG. 5 is a diagram showing an example of the configuration of the present system in the present embodiment.
- the image analysis device 100 includes an initial setting Mac mouth program (corresponding to the microscope stage initial setting section 102c in FIG. 2), a calibration macro program (the calibration initial setting in FIG. 2).
- File creation unit 102 d) shooting macro program (corresponding to the batch shooting unit 102 a, individual shooting unit 102 b, and status management unit 102 e in Fig. 2), FTPD (Corresponding to the communication control interface section 104 in FIG. 2), the photographed image directory (corresponding to the image file information database 106a in FIG. 2), and the system file (the system in FIG. 2).
- the system files of the image analyzer 100 include a plate ID file, a status file, a log file, an individual shooting setting file, an instruction notification file, a system setting file, and the like.
- the plate ID file stores the plate ID currently being worked on (current ID).
- the status file stores the work status of each plate (during shooting, waiting for shooting instructions, etc.) and information on shot image files.
- the log file is for storing the operation log of the control unit. It is a file for debugging and is not essential for the operation of this system.
- the individual shooting setting file stores information such as how to shoot which coordinates of which columns and how many times with the objective lens, and under what name.
- the instruction notification file is a file for synchronizing the start of individual shooting.
- the system setting file contains microscope hardware configuration information (such as the magnification of the installed objective lens), shooting parameters (such as exposure time, brightness, and light source voltage for each objective lens), and plate format information (such as Information such as the array information of the multi-well plate and the distance between the wells is stored.
- microscope hardware configuration information such as the magnification of the installed objective lens
- shooting parameters such as exposure time, brightness, and light source voltage for each objective lens
- plate format information such as Information such as the array information of the multi-well plate and the distance between the wells is stored.
- Image View II is a program for image browsing and photographing support (in FIG. 3, thumbnail display section 202a, individual photographing information designating section 202b, and calibration initial setting section). (Corresponding to 202c) and files (corresponding to the individual shooting information file 206a in Fig. 3).
- the network 300 has a function of interconnecting the image analysis device 100 and the image processing device 200, and is, for example, a local area network (LAN).
- LAN local area network
- the microscope device 400 is a microscope device under the control of the image analysis device 100 for taking a sample.
- the microscope system (DM-name) of Leica Microsystems (company name) in this embodiment is used.
- IRE 2 product name).
- the shooting is performed in the following two phases.
- Image View II uses the Image View II application to view captured images and instruct individual macro photography programs to capture individual images.
- Image View II performs the operation according to the state of the plate ID work. Plate ID If the is the plate currently set on the microscope, the captured images can be sequentially acquired and displayed, and individual photography settings and photography instructions can be issued. If the plate is not the one currently set in the microscope, the image stored locally can be displayed and viewed.
- the macro program and Image View II operate in cooperation via a file. All the files (communication files, configuration files, image files) on the microscope system (DM-IRE2) (product name) are placed in a directory that can be accessed by FTP, and Image View II can access these files via FTP. Access files. The access from the FTP client can be restricted by the FTP server. Depending on the settings, the microscope system (DM-IRE2) (product name) can be accessed simultaneously from multiple clients running Image View II.
- the macro program records the processing status in the status file, reads the status file at startup, and if the operation was interrupted during the low-magnification batch shooting operation, performs the processing from the continuation (recovery processing). If the operation is waiting for an individual shooting instruction or is interrupted during the shooting, restart the operation while waiting for the individual shooting instruction.
- the captured image is saved in the same format as the file saved by the macro program, with the directory and file name saved.
- the image file is imported and displayed on the image view ⁇ .
- Image View II accesses the microscope system (DM-IRE2) (product name) via FTP and transfers any new files locally.
- DM-IRE2 microscope system
- Image View II reads the status file and operates according to the work status of the macro program. If the macro program is taking a picture, the status file is monitored, and image files that have been taken are sequentially transferred, thumbnails are created, and a list is displayed.
- the user sets individual shooting for each gel while viewing the low-magnification shot image in the full size window. Multiple magnifications and shooting area specifications can be set for each well. When the setting is completed, an instruction to start individual shooting is issued.
- the macro program monitors the instruction notification file and waits for an individual shooting instruction. After confirming the shooting instructions, the shooting settings are obtained from the file and the individual images are shot again. '
- Image View II monitors the status file and sequentially acquires and displays the images that have been captured.
- FIG. 6 is a diagram illustrating an example of a shooting setting screen of an image view (user interface) in the present embodiment.
- the shooting setting screen of Image View II displays the shooting frame information corresponding to the set individual shooting that is superimposed and displayed on the image file display area MC-1 and the image file display area MC-1 selected by the user.
- the shooting frame information MC-2 corresponding to 40x and the shooting frame information MC-3 corresponding to 20x are displayed.).
- the shooting setting screen of the image view displays images of low-magnification batch shooting in full size (only this screen can be set for individual shooting).
- the user Drag the mouse to display a frame corresponding to the shooting magnification. Then, the user drags the frame to specify a portion to be photographed. Then, the user presses the photographing button after completing the setting of the photographing positions of all the necessary cells, and photographs are taken.
- the user can take multiple shots under different conditions after the shooting is completed. Also, multiple shooting settings can be made for one well in one shot.
- FIG. 24 is a diagram showing an example of a thumbnail list screen of the image view (user interface) in the present embodiment.
- the thumbnail list screen of the image view is the same as the plate ID input area MD-1, plate ID selection button MD-2, thumbnail display area MD-3, thumbnail display area MD-3, and the image MD-4 displayed in the thumbnail display area MD-3.
- El display button MD-5 for displaying thumbnails of all images taken above
- Erase button MD-6 for erasing all images taken within the selected ⁇
- to start individual shooting Capture button MD-7 image display area MD-8 for displaying all captured images of the same level
- image display area MD- 8 includes an erasing button MD-10 for erasing a selected image from the images displayed in FIG.
- the user inputs a desired plate ID into the plate ID input area MD-1 via the input device 2 1 2, or selects a plate ID selection button (“Select Pect. 1 ate ID ”button) Select MD-2 by clicking on it with a mouse or the like, and enter the plate ID from the window that displays a list of plate IDs that is opened by processing the thumbnail display section 202a. ) Then, when the user selects the magnification specifying tab (in the example of FIG. 24, the magnification can be specified as 2.5 times, 5 times, 10 times, 20 times, and 40 times).
- thumbnails of the images (image MD-4) of the same magnification taken at each level are displayed in the thumbnail display area MD-3 (when multiple images match at the same level and at the same magnification, the latest image is displayed). Show thumbnails). Then, the user selects the thumbnail and clicks the ⁇ el display button (“show We 1 1 s "button) Press MD-5 to open the sheet (image display area MD-8) and display thumbnails (image MD-9) of all images shot on the same page on the sheet .
- the image display area ME-1 in FIG. 25 described below the image display area ME-1 in FIG. 25 described below.
- FIG. 25 is a diagram showing an example of a full-size image screen of an image view (user interface) in the present embodiment.
- the full-size image screen of the image view II includes the image display area ME-1.
- the user can use the cursor keys of the input device 211 to switch and display images of the same magnification of other cells on the full-size image screen of the image view.
- the user presses both the shift key and the cursor key of the input device 211 to switch and display a plurality of images taken at the same magnification at different magnifications on the full-size image screen of the image viewer. be able to. '
- FIG. 7 is a diagram showing an example of a macro program processing outline in the present embodiment.
- the initial setting macro program sets the origin of the stage (step SD-1).
- the calibration macro program captures an image for the view calibration (step SD-2).
- the photographing macro program performs plate photographing and the like (step SD-3).
- step SD-3 details of each processing shown in FIG. 7 will be described later.
- FIG. 8 is a diagram showing an example of an outline of processing of the image view II in this embodiment.
- Image View II displays thumbnail images and sets individual shooting. Here, the details of each process shown in FIG. 8 will be described later.
- FIG. 9 is a flowchart showing an example of processing executed by the initialization macro program in this embodiment.
- the user sets a multi-well plate on the microscope apparatus 400 (step SF-1).
- the user activates the initial setting macro program by performing a predetermined operation of the surface image analyzer 100 (step SF-2).
- the initial setting macro program displays a microscope confirmation dialog on the output device 114 (step SF-3). Then, after the user has tilted the column of the microscope apparatus 400 (step SF-4), the initialization macro program initializes the microscope stage, specifically, moves to the center of the cell coordinates (step SF-4). SF-5).
- the user fine-adjusts the center of the cell coordinates while viewing the display image output to the output device 114 of the image analysis device 100 (step SF-6).
- the initial setting macro program sets the origin of the multiwell plate coordinates to the center of the jewel coordinates (step SF-7).
- FIG. 10 is a flowchart showing an example of processing executed by the calibration macro program in the present embodiment.
- the user sets a calibration plate (for example, a crosshair drawn at the center of the jewel coordinates) on the microscope apparatus 4 (step SG-1).
- the user activates the calibration macro program by performing a predetermined operation of the image analysis device 100 (step SG-2).
- the calibration macro program Instruct the microscope device 400 to move the stage and then take two images with the X and Y coordinates shifted (step SG-3). Then, the calibration macro program creates a view ⁇ calibration information file and an image file (step SG-4).
- FIG. 11 is a flowchart showing an example of a process executed by the photographing macro program in the present embodiment.
- the user sets the plate on the microscope device 400 (step SH_1).
- the user starts the photographing macro program by performing a predetermined operation of the image analysis device 100 (step SH-2).
- the photographing macro program displays a plate ID input dialog on the output device 114 (step SH-3). Then, the user inputs the plate ID to the plate ID input dialog displayed on the output device 114 via the input device 112 (step SH-4).
- the photographing macro program After receiving an instruction from the user that the plate ID input has been completed, the photographing macro program refers to the status file (step SH-5). Then, the photographing macro program determines whether or not the photographing to be performed is the first time or the time when the low magnification photographing is restarted, based on the referenced status file (step SH-6).
- the shooting macro program determines that the shooting to be performed is the first time or when resuming low-magnification shooting, the shooting macro program performs low-magnification batch shooting, records the status in the status file for each 1-mm shot, Save the shot image to the shot image directory (Step SH-7). Then, upon receiving the completion of the low-magnification shooting, the shooting macro program monitors the instruction notification file and receives the instruction (step SH-8).
- step SH-9 the photographing macro program performs individual photographing, and sets a status flag for each photographing. Record in the aisle (Step SH-10).
- the shooting macro program monitors the instruction notification file and continues receiving instructions (step SH-8), step SH-9, and, if necessary, step SH-10. Is performed.
- FIG. 12 is a flowchart showing an example of a process executed by an image view at the time of image acquisition and browsing in this embodiment.
- the user activates the image view II.
- the image view II displays a window on the output device 214 (step SJ-2).
- step SJ-3 the operator inputs the plate ID to the window displayed on the output device 214 via the input device 212.
- the image view II acquires the current ID from the plate ID file (step S J-4).
- the image view II determines whether the input plate ID is the same as the acquired current ID (step SJ-5). Then, if the input plate ID and the acquired current ID are not the same (step SJ-15), the image view II displays the acquired local image on the output device 214 (step SJ-16). In this case, the individual shooting setting is not performed.
- step SJ-5 the image view II acquires the system settings from the system setting file (step SJ-7). Then, the image view II captures an image of the same plate ID (step SJ-8).
- step SJ-9 'image view' acquires the status of the shooting macro program from the status file (step SJ-9). Then, based on the acquired status of the photographing macro program, the image view II determines whether the photographing force is being taken or the instruction is waiting (step SJ-10). If the image is being shot (step SJ-10), the image view ⁇ monitors the status, successively acquires and displays the shot image (step SJ-11). Then, when waiting for an instruction (step SJ-10), the image view II displays the acquired image (step SJ-12). In this case, individual shooting settings can be made.
- FIG. 13 is a flowchart showing an example of a process executed by the image view during individual photographing in the present embodiment.
- the image view II displays an image on the output device 214 (step SK-1).
- individual shooting settings are possible.
- the user selects an individual shooting target element (step SK1-2).
- the image view II displays the low magnification photographed image on the output device 214 in a full size screen (step SK-3). Then, the user specifies the imaging location and the magnification, and performs the imaging setting for each well (step SK-4).
- the setting of the shooting location and the magnification may be performed according to the following procedure.
- the photographing frame information centered on the position (FIG. 6) (MC_2 in, etc.) is superimposed on the display screen.
- shooting frame information of different magnifications may be displayed depending on the dragging distance (for example, dragging a short distance will display 80 times the shooting frame information, dragging a medium distance will display a 40 times shooting frame information, Drag it to get 20x shooting frame information).
- the user sets the shooting location by adjusting the shooting frame information to a desired position.
- step SK-5 the image view II waits for a shooting setting operation.
- the image view II checks the status of the shooting macro program against the status file (step SK-7). Then, the image view ⁇ determines whether the status of the shooting macro program is “shooting” or “waiting for shooting instructions” (step Step SK—8) If “shooting”, the process returns to the playback step SK—5. Then, when the state of the photographing macro program is “waiting for photographing instruction”, the image view ⁇ instructs the microscope device 400 to start photographing (step SK-9). Then, the image view II monitors the status and sequentially acquires and displays the captured images (step SK-10).
- FIG. 14 is a flowchart illustrating an example of processing executed by the image view when performing calibration in the present embodiment.
- the user starts the calibration function of the image view II by a predetermined process (step SL-1).
- Image View II reads the calibration information file (Step SL-2).
- the image view II acquires a calibration image file (step SL-3). Then, the image view II superimposes the two images and displays them on the output device 214 (step SL-4). Then, the user clicks the reference point (for example, the center point of the crosshair) of the two displayed images (step SL-5). Then, the image view II calculates the ratio of the number of displayed pixels and the moving distance of the stage, and stores the ratio in a file (step SL-6).
- FIG. 15 is a flowchart showing an example of a process executed by the image view ⁇ ⁇ ⁇ ⁇ when executing the capture of the designated plate ID image in the present embodiment.
- the image view II displays an image on the output device 214 (step SM-1).
- individual shooting settings can be made.
- the user designates the “image synchronization” button via the input device 212 (step SM-2).
- Image View II searches the captured image directory of the microscope system (DM-IRE2) (product name) (Step SM-3). Then, the image view ⁇ determines whether there is a new image in the shooting plane image directory (step SM-4), and if there is a new image, obtains a new image (step SM-5), and stores the thumbnail as a thumbnail. It is created and displayed on the corresponding magnification specification tab (step SM-6).
- FIG. 16 is a view showing an image view ⁇ 'when executing erasure of a photographed image in this embodiment.
- 5 is a flowchart illustrating an example of a process executed by a process.
- the image view II displays an image on the output device 214 (step SN-1).
- the user selects an image to be deleted on the thumbnail screen (step SN-2).
- the user specifies a “delete” button via the input device 212 (step SN-3).
- the image view II opens a deletion confirmation dialog on the output device 214 (step SN-4). Then, the user specifies the “OK” button of the deletion confirmation dialog opened on the output device 214 via the input device 212 (step SN-5).
- the image view II deletes the local designated image (step SN-6). Then, the image view II deletes the designated image of the microscope system (DM-IRE2) (product name) from the photographed image directory (step SN-7).
- DM-IRE2 designated image of the microscope system
- FIG. 17 is a processing sequence showing an example of a cooperative operation at the time of calibration setting in the present embodiment.
- the image view II acquires the settings from the system setting file (step SR-1).
- the user sets the dedicated plate on the microscope device 400 (Step SR-2).
- the user starts the macro program by performing a predetermined operation of the image analysis device 100 (step SR-3).
- the macro program obtains the settings from the system setting file (step SR-4).
- the macro program captures the image (step SR-5).
- the macro program saves the photographed image in the photographed image directory (step SR-6). Then, the macro program saves the calibration settings in the calibration setting file (step SR-7). Then, the user instructs the image view ⁇ to start the calibration (step SR-8). Then, the image view II acquires the settings from the calibration setting file (step SR-9). 'Then, the image view II acquires an image from the photographed image directory (step SR-10). Then, the image view II displays the double copied image (step SR-11).
- step SR-12 the user clicks the reference points of the two images via the input device 212 (step SR-12).
- the image view ⁇ calculates and stores the ratio of the stage movement amount to the pixel (step SR-13).
- FIG. 18 is a processing sequence illustrating an example of a cooperative operation at the time of low-magnification batch shooting in the present embodiment.
- the image view II refers to the settings in the system setting file (step SP-1).
- step S-2 the user sets the plate on the mirroring device 400 (step S-2).
- the user starts the macro program by performing a predetermined operation of the image analysis device 100 (step sp-3).
- the macro program refers to the settings in the system setting file (step SP-4).
- the user inputs the plate ID to the macro program via the input device 112 (step SP-5).
- the macro program saves the plate ID in the plate ID file (Step SP-6). Then, the macro program obtains the status from the status file (step SP-7). Then, the user inputs the print ID to the image view via the input device 212 (step SP-7). — 8).
- the image view II acquires the plate ID from the plate ID file (step SP-9). Then, the image view II acquires status information from the status file (step SP-10). Then, the macro program takes a low-magnification image (step SP—: I 1).
- the macro program saves the captured images sequentially to the captured image directory (Step SP— 1 2). Then, the macro program records the status in the status file (step SP-13). Then, the image view II acquires the low magnification image from the photographed image directory (step SP-14), creates a thumbnail, and displays the thumbnail (step SP-15).
- FIG. 19 is a processing sequence showing an example of a cooperative operation at the time of individual photographing in the present embodiment.
- the macro program monitors the instruction notification file to confirm the instruction from the image view II (step SQ-1).
- the user inputs individual photographing settings via the input device 212 of the image view II (step SQ-2).
- the image view II stores the input settings (step SQ-3).
- step SQ-4 the user issues a shooting start instruction via the input device 212 (step SQ-4).
- Image View II writes the shooting setting information to the shooting setting file (FTP transfer) (Step SQ-5).
- Image View II writes the individual shooting instruction in the instruction notification file (FTP transfer) (step SQ-6).
- the macro program receives the photographing instruction from the instruction notification file in which the photographing instruction from the image view II is written (step SQ-7).
- step SQ-8 the photographed image directory to confirm whether or not there is a photographed image
- the macro program acquires shooting setting information from the shooting setting file (step SQ-9). Then, the macro program captures an individual pixel image (step SQ-10). Then, the macro program sequentially saves the captured images in the captured image directory (step SQ-11).
- the image view II sequentially acquires the photographed images from the photographed image directory (step SQ-12). Then, the image view II creates a thumbnail and displays it on the output device 214 (step SQ-13). Then, the user browses the captured image displayed on the output device 214 of the image view II (step SQ-14). Steps SQ-1 to SQ-14 are individual shooting cycles. And repeat;
- the processing is performed in response to a request from a client terminal configured in a separate housing from the image analysis apparatus 100, and the processing is performed.
- the result may be returned to the client terminal.
- the image analysis device 100 and the image management device 200 may be configured to be realized as the same housing.
- the microscope system (DM-IRE2) (product name) of Leica Microsystems (company name) is used.
- the present invention is not limited to this. Instead, there are alternative models equivalent to this microscope, such as Carl Zeiss (company name) Ax iovert 200M (product name), Nikon (company name) TE 2000E (product name), and Olympus (company name) IX81. (Product name) or the like may be used.
- the image analysis apparatus 100 is a PC attached to a microscope system of Leica Microsystems (company name), image measurement software Qwin (product name), and a macrofloat mounted thereon.
- the present invention is realized using a group of programs, but the present invention is not limited to these.
- the present invention is not limited to these.
- software corresponding to Qwin there is a Universal Imaging Corporation (Universal Imaging Corporation).
- Company name MetaMorp (product name) and Mitani Corporation (company name) Lum ina Vision (product name) may be used.
- the image analysis apparatus 100 of the present invention is not an essential condition to be mounted on such off-the-shelf image measurement software or the like.
- step SA-2 The acquisition of the low-magnification image (step SA-2) and the notification of the individual photographing information (step SA-5) described above may be realized by using the existing e-mail transmission technology.
- the function provided by the website provided by the device 100 or the image management device 200 allows the user or the like to input information by presenting a predetermined input format and transmit the input information.
- it may be realized by existing file transfer technology such as FTP.
- all or a part of the processes described as being performed automatically can be manually performed, or the processes described as being performed automatically can be performed. All or a part of the processing can be automatically performed by a known method.
- each illustrated component is a functional concept, and does not necessarily need to be physically configured as illustrated.
- each unit or each device of the image analysis device 100 are transferred to a CPU (Central Processing Unit) and a CPU (Central Processing Unit). It can be realized by a program interpreted and executed by the CPU, or can be realized as hardware by wired logic. The program is recorded on a recording medium described later, and is read mechanically by the image analysis device 100 as necessary.
- a CPU Central Processing Unit
- CPU Central Processing Unit
- instructions are given to the CPU in cooperation with the OS (Operating System) in the storage unit 106 such as the ROM or the HD, and various processes are performed.
- a computer program for recording is recorded. This computer program is executed by being loaded into a RAM or the like, and forms a control unit 102 in cooperation with a CPU. Also, this computer program may be recorded in an application program server connected to the image analysis apparatus 100 via an arbitrary network 1 or network 300, and all or part of the computer program may be downloaded as necessary. It is also possible to download.
- the program according to the present invention can be stored in a computer-readable recording medium.
- this “recording medium” refers to any “portable physical medium” such as a flexible disk, a magneto-optical disk, ROM, EPROM, EEPROM, CD-ROM, MO, DVD, and various computer systems. Any ⁇ fixed physical medium '' such as built-in ROM, RAM, HD, etc., or a communication line or carrier wave when transmitting a program via a network represented by LAN, WAN, Internet And “communications media” that hold programs for a short period of time.
- program is a data processing method described in an arbitrary language or description method, regardless of the format of source code or binary code.
- the “program” is not necessarily limited to a single program, but can be distributed as multiple modules / libraries, or can cooperate with a separate program such as an OS (Operating System). And those that achieve that function. It should be noted that a known configuration or procedure can be used for a specific configuration for reading the recording medium in each device described in the embodiment, a reading procedure, an installation procedure after reading, and the like.
- Various databases and the like stored in the storage unit 106 are storage devices such as a memory device such as a RAM and a ROM, a fixed disk device such as a hard disk, a flexible disk, and an optical disk, and are used for various processes and for providing a website. Stores various programs such as tape files, file databases, and web page files.
- the image analysis apparatus 100 connects a peripheral device such as a printer monitor or an image scanner to an information processing apparatus such as an information processing terminal such as a known personal computer or a workstation, and connects the information processing apparatus of the present invention to the information processing apparatus. This may be achieved by implementing software (including programs, data, etc.) that realizes the above.
- each database may be independently configured as an independent database device, and a part of the processing may be realized by using CGI (Common Gateway Inte rfa ace).
- CGI Common Gateway Inte rfa ace
- the tooth management device 200 is a printer for information processing devices such as known personal computers, workstations, home game devices, Internet TVs, PHS terminals, mobile terminals, mobile communication terminals, and information processing terminals such as PDAs. ⁇ ⁇
- information processing devices such as known personal computers, workstations, home game devices, Internet TVs, PHS terminals, mobile terminals, mobile communication terminals, and information processing terminals such as PDAs. ⁇
- peripheral devices such as monitors and image scanners as necessary, and installing software (including programs, data, etc.) for implementing a web information browsing function and an e-mail function in the information processing device It may be realized.
- the control unit 202 of the image management device 200 can realize the whole or any part of the control unit by a CPU and a program interpreted and executed by the CPU.
- ROM or the HD a computer program for giving instructions to the CPU in cooperation with the OS (OperaintSystem) and performing various processes is recorded.
- This computer program is executed by being loaded into the RAM, and forms a control unit in cooperation with the CPU.
- this computer program may be recorded in an application program server connected to the image management device 200 via an arbitrary network, and all or a part of the computer program may be downloaded as necessary. It is. Alternatively, all or any part of each control unit can be It can also be realized as hardware such as logic.
- the network 300 has a function of interconnecting the image analysis device 100 and the image management device 200, and includes, for example, the Internet, an intranet, and LAN (including both wired / wireless). ), VAN, PC communication network, public telephone network (including both analog and digital), leased line network (including both analog and digital), CATV network, IMT 200 0 system, GSM system or PDC / PDC-P system, etc., mobile circuit switching network / mobile bucket switching network, wireless calling network, local wireless network such as B 1 uetooth, PHS network, CS, BS or Any of satellite communication networks such as ISDB may be included. That is, the present system can transmit and receive various data via any network, whether wired or wireless.
- each sample holding part (for example, each well of a multiwell plate) of a sample holder (for example, a multiwell plate) is set to a predetermined magnification (for example, using a microscope).
- an individual shooting information designation screen for allowing the user to specify individual shooting information is displayed on the monitor, and the coordinate information and the coordinates designated on the individual shooting information designation screen by the user via the input device.
- the present invention provides a microscope image processing system, a microscope image processing method, a program, and a recording medium that allow a user to specify a desired position to perform individual shooting because individual shooting information is created from magnification information. thing Can be.
- various inputs such as a well-known map / viewing function (keys for moving in eight directions and sixteen directions, magnification setting change buttons, etc.) on the individual shooting information designation screen displayed on the monitor according to the present invention.
- a support function may be provided, and the user may specify individual shooting information using such a function.
- the coordinate information to be photographed and the Z or magnification information are automatically determined by the image processing method and the individual photographing information is created, for example, a known image processing method (for example, calculation of the center of gravity or the like) is used. It is possible to provide a microscope image processing system, a microscope image processing method, a program, and a recording medium that can automatically determine a shooting position of individual shooting.
- the microscope image processing system can automate the photographing of the microscope image as intended by the operator.
- the present invention can be widely practiced in many industrial fields, particularly in the fields of pharmaceuticals, foods, cosmetics, and medical treatments, and is extremely useful.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Microscoopes, Condenser (AREA)
- Processing Or Creating Images (AREA)
Abstract
Description
Claims
Priority Applications (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2004544908A JPWO2004036283A1 (ja) | 2002-09-06 | 2003-09-05 | 顕微鏡画像処理システム、顕微鏡画像処理方法、プログラム、および、記録媒体 |
| AU2003261965A AU2003261965C1 (en) | 2002-09-06 | 2003-09-05 | Microscope image processing system, microscope image processing method, program, and recording medium |
| EP03808878A EP1548480A4 (en) | 2002-09-06 | 2003-09-05 | MICROSCOPE PROCESSING SYSTEM, MICROSCOPE PROCESSING PROCESS, PROGRAM AND RECORDING MEDIUM |
| US10/526,485 US20050270370A1 (en) | 2002-09-06 | 2003-09-05 | Microscope image processing system, microscope image processing method, program, and recording medium |
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2002262013 | 2002-09-06 | ||
| JP2002-262013 | 2002-09-06 | ||
| JP2002-330233 | 2002-11-14 | ||
| JP2002330233 | 2002-11-14 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2004036283A1 true WO2004036283A1 (ja) | 2004-04-29 |
Family
ID=32109435
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/JP2003/011367 Ceased WO2004036283A1 (ja) | 2002-09-06 | 2003-09-05 | 顕微鏡画像処理システム、顕微鏡画像処理方法、プログラム、および、記録媒体 |
Country Status (4)
| Country | Link |
|---|---|
| US (1) | US20050270370A1 (ja) |
| EP (1) | EP1548480A4 (ja) |
| JP (1) | JPWO2004036283A1 (ja) |
| WO (1) | WO2004036283A1 (ja) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008058396A (ja) * | 2006-08-29 | 2008-03-13 | Olympus Corp | 顕微鏡撮像システム、顕微鏡撮像方法、及び、記録媒体 |
| JP2008197992A (ja) * | 2007-02-14 | 2008-08-28 | Nikon Corp | 顕微鏡画像処理システムおよび顕微鏡画像処理方法 |
| JP2009008846A (ja) * | 2007-06-27 | 2009-01-15 | Nikon Corp | 多次元画像の閲覧装置 |
| JP2009244561A (ja) * | 2008-03-31 | 2009-10-22 | Olympus Corp | 顕微鏡撮影システム |
| JP2019017316A (ja) * | 2017-07-19 | 2019-02-07 | 大日本印刷株式会社 | 画像表示装置、プログラム及び制御方法 |
| JP2019523897A (ja) * | 2016-05-12 | 2019-08-29 | ライフ テクノロジーズ コーポレーション | 画像の取り込みおよび表示のためのシステム、方法、ならびに装置 |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7711990B1 (en) * | 2005-12-13 | 2010-05-04 | Nvidia Corporation | Apparatus and method for debugging a graphics processing unit in response to a debug instruction |
| US7681081B2 (en) * | 2006-09-15 | 2010-03-16 | Hong Fu Jin Precision Industry (Shenzhen) Co., Ltd. | Test device and method for testing stability of computer |
| DE102008014030B4 (de) * | 2008-03-12 | 2017-01-26 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Verfahren zum Kalibrieren eines Bühne-Kamera-Systems sowie Bühne-Kamera-System und Mikroskop mit derartigem Bühne-Kamera-System |
| US9151944B2 (en) * | 2012-07-19 | 2015-10-06 | Sony Corporation | Method and apparatus for navigating stacked microscopy images |
| US9797767B2 (en) * | 2014-08-26 | 2017-10-24 | General Electric Company | Calibration of microscopy systems |
| JP6790734B2 (ja) * | 2016-11-02 | 2020-11-25 | 株式会社ニコン | 装置、方法、およびプログラム |
| US11112952B2 (en) * | 2018-03-26 | 2021-09-07 | Microscopes International, Llc | Interface for display of multi-layer images in digital microscopy |
| CN213245476U (zh) * | 2019-10-25 | 2021-05-21 | 尹毅 | 一种艺术品信息采集点交装置 |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH063597A (ja) * | 1992-06-22 | 1994-01-14 | Olympus Optical Co Ltd | 顕微鏡静止画像観察システム |
| JPH06222281A (ja) * | 1993-01-22 | 1994-08-12 | Olympus Optical Co Ltd | 顕微鏡遠隔観察システム |
| WO1998008185A1 (en) * | 1996-08-23 | 1998-02-26 | Bacus Research Laboratories, Inc. | Method and apparatus for testing a progression of neoplasia including cancer chemoprevention testing |
| JPH1138324A (ja) * | 1997-07-23 | 1999-02-12 | Nikon Corp | レーザ走査顕微鏡 |
| JP2002014288A (ja) * | 2000-06-29 | 2002-01-18 | Nikon Corp | 顕微鏡装置及びその使用方法 |
| JP2002196257A (ja) * | 2000-12-25 | 2002-07-12 | Nikon Corp | 画像処理装置、コンピュータ読取可能な記録媒体、及び顕微鏡システム。 |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5655029A (en) * | 1990-11-07 | 1997-08-05 | Neuromedical Systems, Inc. | Device and method for facilitating inspection of a specimen |
| US5257182B1 (en) * | 1991-01-29 | 1996-05-07 | Neuromedical Systems Inc | Morphological classification system and method |
| US6148096A (en) * | 1995-09-15 | 2000-11-14 | Accumed International, Inc. | Specimen preview and inspection system |
| CA2236268A1 (en) * | 1995-11-30 | 1997-06-05 | Chromavision Medical Systems, Inc. | Method and apparatus for automated image analysis of biological specimens |
-
2003
- 2003-09-05 WO PCT/JP2003/011367 patent/WO2004036283A1/ja not_active Ceased
- 2003-09-05 US US10/526,485 patent/US20050270370A1/en not_active Abandoned
- 2003-09-05 EP EP03808878A patent/EP1548480A4/en not_active Ceased
- 2003-09-05 JP JP2004544908A patent/JPWO2004036283A1/ja active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH063597A (ja) * | 1992-06-22 | 1994-01-14 | Olympus Optical Co Ltd | 顕微鏡静止画像観察システム |
| JPH06222281A (ja) * | 1993-01-22 | 1994-08-12 | Olympus Optical Co Ltd | 顕微鏡遠隔観察システム |
| WO1998008185A1 (en) * | 1996-08-23 | 1998-02-26 | Bacus Research Laboratories, Inc. | Method and apparatus for testing a progression of neoplasia including cancer chemoprevention testing |
| JPH1138324A (ja) * | 1997-07-23 | 1999-02-12 | Nikon Corp | レーザ走査顕微鏡 |
| JP2002014288A (ja) * | 2000-06-29 | 2002-01-18 | Nikon Corp | 顕微鏡装置及びその使用方法 |
| JP2002196257A (ja) * | 2000-12-25 | 2002-07-12 | Nikon Corp | 画像処理装置、コンピュータ読取可能な記録媒体、及び顕微鏡システム。 |
Non-Patent Citations (1)
| Title |
|---|
| See also references of EP1548480A4 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008058396A (ja) * | 2006-08-29 | 2008-03-13 | Olympus Corp | 顕微鏡撮像システム、顕微鏡撮像方法、及び、記録媒体 |
| US8106943B2 (en) | 2006-08-29 | 2012-01-31 | Olympus Corporation | Microscope image pickup system, microscope image pickup method and recording medium |
| JP2008197992A (ja) * | 2007-02-14 | 2008-08-28 | Nikon Corp | 顕微鏡画像処理システムおよび顕微鏡画像処理方法 |
| JP2009008846A (ja) * | 2007-06-27 | 2009-01-15 | Nikon Corp | 多次元画像の閲覧装置 |
| JP2009244561A (ja) * | 2008-03-31 | 2009-10-22 | Olympus Corp | 顕微鏡撮影システム |
| JP2019523897A (ja) * | 2016-05-12 | 2019-08-29 | ライフ テクノロジーズ コーポレーション | 画像の取り込みおよび表示のためのシステム、方法、ならびに装置 |
| USD951283S1 (en) | 2016-05-12 | 2022-05-10 | Life Technologies Corporation | Microscope system display screen with graphical user interface |
| JP7300834B2 (ja) | 2016-05-12 | 2023-06-30 | ライフ テクノロジーズ コーポレーション | 画像の取り込みおよび表示のためのシステム、方法、ならびに装置 |
| JP2019017316A (ja) * | 2017-07-19 | 2019-02-07 | 大日本印刷株式会社 | 画像表示装置、プログラム及び制御方法 |
Also Published As
| Publication number | Publication date |
|---|---|
| US20050270370A1 (en) | 2005-12-08 |
| AU2003261965B2 (en) | 2007-03-22 |
| AU2003261965A1 (en) | 2004-05-04 |
| EP1548480A4 (en) | 2007-02-14 |
| JPWO2004036283A1 (ja) | 2006-02-16 |
| EP1548480A1 (en) | 2005-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2004036283A1 (ja) | 顕微鏡画像処理システム、顕微鏡画像処理方法、プログラム、および、記録媒体 | |
| RU2437169C2 (ru) | Устройство отображения изображения, устройство съемки изображения | |
| EP1686533B1 (en) | Method and apparatus for creating a virtual microscope slide | |
| JP7838045B2 (ja) | デジタル撮像システムおよび方法 | |
| CA2398736C (en) | Method and apparatus for internet, intranet, and local viewing of virtual microscope slides | |
| CN101542353B (zh) | 图像处理装置、图像处理方法和观察系统 | |
| JP2008052227A (ja) | 観察装置 | |
| JP5015381B2 (ja) | 顕微鏡写真撮影装置 | |
| JP5822345B2 (ja) | ホールスライドイメージ作成装置 | |
| JP2002258163A (ja) | 顕微鏡制御装置、顕微鏡制御システム、顕微鏡の制御方法、プログラム、及び記録媒体 | |
| WO2008065751A1 (fr) | Dispositif de microscope | |
| JP7115508B2 (ja) | 病理画像表示システム、病理画像表示方法及びプログラム | |
| EP1860482A2 (en) | Microscope system and method for synthesizing microscopic images | |
| JP2011083032A (ja) | データ構造を記憶したコンピュータ読み取り可能な記録媒体 | |
| JP2008139487A (ja) | 顕微鏡装置 | |
| US20140063072A1 (en) | Information processing apparatus, information processing method, and information processing program | |
| JP2005077253A (ja) | 拡大観察装置、画像ファイル生成装置、画像ファイル生成プログラムおよびコンピュータで読み取り可能な記録媒体 | |
| US7865007B2 (en) | Microscope system, observation method and observation program | |
| CN114005112A (zh) | 细胞位置标定方法、装置、计算机设备和存储介质 | |
| JP4847101B2 (ja) | 顕微鏡システム | |
| JPH063597A (ja) | 顕微鏡静止画像観察システム | |
| JP3698737B2 (ja) | 顕微鏡遠隔観察システム及び遠隔観察方法 | |
| JPH05241075A (ja) | 顕微鏡システム | |
| JP2009210773A (ja) | 顕微鏡用画像の情報処理装置 | |
| JP3704529B2 (ja) | 顕微鏡観察システムおよび顕微鏡画像の表示方法 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| AK | Designated states |
Kind code of ref document: A1 Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW |
|
| AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG |
|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
| WWE | Wipo information: entry into national phase |
Ref document number: 2004544908 Country of ref document: JP |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 2003808878 Country of ref document: EP Ref document number: 10526485 Country of ref document: US Ref document number: 2003261965 Country of ref document: AU |
|
| WWP | Wipo information: published in national office |
Ref document number: 2003808878 Country of ref document: EP |
|
| WWG | Wipo information: grant in national office |
Ref document number: 2003261965 Country of ref document: AU |