JPH02141069A - Picture input device - Google Patents

Abstract

PURPOSE:To facilitate inspection or observation by deciding a notable area from a picture image-picked-up by means of a low resolution image pickup means, and photographing only the area by means of a high resolution image pickup device. CONSTITUTION:A notable area deciding circuit 40 decides the notable area from the contents of a converting picture memory 39, obtains the central position coordinates of the area, supplies them to a stage control circuit 42, simultaneously obtains the multiplying factor of a multiplying factor variable lens corresponding to the rectangular area, and supplies it to a focus/multiplying factor control circuit 44. The stage control circuit 42 controls an X-Y stage 22 so that the central position coordinates of the image pickup element may be the central position coordinates applied by the notable area deciding circuit 40. The focus/multiplying factor control circuit 44 regulates the multiplying factor of the multiplying factor variable lens so that a visual field from the viewpoint of the image pickup element may be the area, and it regulates focus as well. Thus, the picture photographed by the image pickup element is electrically scanned and read, amplified, digitally converted, and written to a picture memory 48. Thus, the desired information can be obtained in a short time without a waste.

Description

[Detailed Description of the Invention] [Table of Contents] Overview Industrial Field of Application Conventional Technology Problems to be Solved by the Invention Means for Solving the Problems Action Embodiment 1st Embodiment (Figs. 1 to 5) 2nd Implementation Example (Figures 6 to 8) Effects of the extended invention [Summary] Image input that is used in image recognition devices, image inspection devices, measurement devices, etc., and has a low-resolution imaging device and a high-resolution imaging device, and has the advantages of both. Regarding the device, the aim is to automatically determine the visual field of interest without human judgment and to image that visual field with high resolution. a high-resolution imaging device that captures an image at a higher resolution than the low-resolution imaging device; A field of view moving means for moving within the field of view and an image captured by the low-resolution imaging means or an image obtained by converting the image, and a region including a portion where a change between adjacent pixel data is equal to or greater than a predetermined value is determined to be a region of interest. attention area determining means; attention area determining means for determining a subjective contour by interpolation filter processing on an image captured by the low-resolution imaging means or an image obtained by converting the image; and at least an attention area determining means for determining, as an attention area, an area including a portion where a change over time in each pixel data is equal to or greater than a predetermined value in an image captured by the low-resolution imaging means or an image obtained by converting the image. It has one attention area determination means and a control means that controls the field of view moving means to make the second field of view correspond to the attention area, and is configured to image the attention area with high resolution. .

[Industrial Application Field] The present invention relates to an image input device that is used in an image recognition device, an image inspection device, a measurement device, or the like, and includes a low-resolution image pickup device and a high-resolution image pickup device, and has the advantages of both.

[Prior Art] In an image input device used in an image recognition device or the like, in order to obtain an image with a wide imaging field of view and high resolution, the capacity of the image memory becomes large and the image processing time becomes long. For example, if the number of pixels in the vertical and horizontal columns of an image is increased by 10 times, the image memory capacity and image processing time will be 100 times each.6 For image recognition, a certain resolution or higher is required depending on the recognition target. It is necessary to obtain images of Recognition may become impossible.

Therefore, an image input device has been proposed that combines the advantages of both, comprising a low-resolution imaging device that can image a wide field of view with low resolution, and a high-resolution imaging device that has a narrow imaging field of view but high resolution. (Japanese Unexamined Patent Publication No. 62-74179).

[Problems to be Solved by the Invention] However, in conventional image input devices, it is necessary to determine the field of view of the high-resolution imaging device based on human judgment, so the object to be recognized must be constantly monitored visually. There was a perfect spot.

SUMMARY OF THE INVENTION In view of these problems, an object of the present invention is to provide an intelligent image input device that can automatically determine a visual field of interest without human judgment and image that visual field with high resolution. .

[Means for Solving the Problem] In order to achieve this object, the image input device according to the first invention includes a low-resolution imaging device that images a first field of view, and a second field of view that is narrower than the first field of view. A high-resolution imaging device that captures an image at a higher resolution than the resolution imaging device, a field of view moving means that moves the second field of view within the first field of view, and an image taken by the low-resolution imaging means or a converted image. an attention area determining means for determining an area of the image including a portion where a change between adjacent pixel data is equal to or greater than a predetermined value as an attention area; and a field of view that controls the visual field moving means to change the second visual field to the visual field corresponding to the attention area. and control means to

In the second invention, instead of the attention area determining means of the first invention, a subjective contour is obtained by interpolation filter processing for an image captured by the low-resolution imaging means or an image obtained by converting the image, and the subjective contour is A region of interest determining means is provided for determining the area within the region of interest to be a region of interest.

In the third invention, instead of or in addition to the attention area determining means of the first or second invention, each pixel data is provided for an image captured by the low-resolution imaging means or an image obtained by converting the image. The apparatus includes an attention area determining means for determining an area including a portion where the change over time of is equal to or greater than a predetermined value as an attention area.

[Operation] Depending on the object to be recognized, inspected, or observed, there may be cases where it is desired to mainly know in detail an area with large spatial changes or an area with large temporal changes.

According to the first or second aspect of the invention, since regions with large spatial changes are determined from images captured by a low-resolution imaging device, it is possible to automatically determine a region of interest in a short time. Further, since only this area is photographed with a high-resolution imaging device, recognition, inspection, or observation becomes easy, and the desired information can be obtained in a short time without wasting any waste. Since such processing is performed without human judgment, there is no need to constantly monitor the recognition target, etc. (and labor can be saved).

According to the third aspect of the present invention, since regions with large temporal changes are determined from images captured by a low-resolution imaging device, it is possible to automatically determine a region of interest in a short time. Other points are above! Or it is the same as the case of the second invention.

Therefore, according to the present invention, an image input device can be efficiently made intelligent.

[Example] Hereinafter, the present invention will be explained in detail based on the drawings.

(1) First embodiment For example, when monitoring a wide field of view with TV cameras for fire prevention purposes, if you want to detect small fire sources early, it is necessary to install a large number of TV cameras and to correspond to each TV camera. It is necessary to have an image processing device. A first embodiment of the present invention will be described assuming such a case.

No.! The figure shows the optical system of the image input device. /% Uzing 1
An imaging lens I2 is provided on one side of the housing 10, and an imaging element 14 is fixed to the inner surface of the housing 10 facing the imaging lens 12 at the imaging position of the imaging lens 12.

The image sensor 14 can image a wide field of view, but has low resolution, and the number of pixels is, for example, 256×256.

A half mirror 16 is fixed in the optical path between the imaging lens 12 and the image sensor 4 with its surface inclined at 45 degrees to the optical axis of the imaging lens 12. An imaging device 20 is arranged below the half mirror 16 via a variable magnification lens 18, and a portion of the light beam that passes through the imaging lens 12 and enters the housing 10 is reflected downward by the half mirror 16. The image is then magnified by the variable magnification lens 18 and formed on the image sensor 20. The number of pixels of the image sensor 20 is <256 x 256, which is the same as the image sensor 14, for example, but the image taken by the image sensor 20 is magnified by the variable magnification lens 18, so the image sensor! The resolution is higher than that of the image taken by No. 4.

The variable magnification lens 18 is integrated into an image sensor 20 so as to be movable in its front axis direction, and the image sensor 20 is held on an X-Y moving stage 22λ, and the x-Y moving stage 22a is fixed to the bottom surface of the housing 10. X-Y fixed stage 2
It is movable on 2b by a drive mechanism (not shown).

Here, if the fire determination standard is set low (high sensitivity) and a fire is determined by processing the low-resolution image taken by the image sensor 14, the probability of false fire detection increases; A wide field of view can be monitored using one image input device, and high-speed processing can be performed. Therefore, the image sensor 20 uses the area that has been determined to be a fire as the area of interest.
If you process the high-resolution images and re-determine whether there is a fire, you can use one image input device to capture many TVs.
It can play the role of a camera, and requires only one image processing device. For this purpose, it is necessary to automatically perform fire determination and control the x-y moving stage 20a to image the determined area.

Next, the image input device shown in FIG. 2 will be explained with reference to FIGS. 3 to 5.

The image taken by the image sensor I4 at time t is electrically scanned and read by the image input circuit 26,
The image is amplified and digitally converted and written into the image memory 28, and transferred to the delayed image memory 32 by the image transfer circuit 30. Image sensor at time (t+d)! 4 is similarly written into the image memory 28 via the image input circuit 26.

The contents of the image memory 28 are filtered for each pixel by a spatial differentiation circuit 34 using a differential filter, for example a 5 obel differential filter, and the results are sequentially supplied to the evaluation circuit 38. Further, the difference in pixel data for each corresponding pixel in the image memory 28 and the delayed image memory 32 is determined by the time differentiation circuit 36, and the results are sequentially supplied to the evaluation circuit 38. The evaluation circuit 38 adds a preset weight to each output value of the spatial differentiation circuit 34 and the temporal differentiation circuit 36 for each pixel, calculates the sum of both, and writes the sum into the converted image memory 39. The attention area determination circuit 40 determines the attention area from the contents of the converted image memory 39.

For example, if the images at time and time (t+d) are the same as shown in FIG. 3, the output value of the time differentiation circuit 36 is 0 for each pixel, and the output value of the spatial differentiation circuit 34 takes a large value within the rectangular areas A and B shown in FIG. Therefore, the attention area determination circuit 4
0 determines that rectangular areas A and B are the areas of interest. The attention area determination circuit 40 determines the center position coordinates of the area A and supplies it to the stage control circuit 42. At the same time, the attention area determination circuit 40 determines the magnification of the variable magnification lens 18 corresponding to the rectangular area A and controls the focus/magnification control circuit. 44. The stage control circuit 42 controls the x-y stage 22 so that the center position coordinates of the image sensor 20 become the center position coordinates given by the attention area determination circuit 40. Further, the focus/magnification control circuit 44 adjusts the magnification of the variable magnification lens 18 and adjusts the focus so that the field of view seen from the image sensor 20 is the area A. After confirming that the processing in the stage control circuit 42 and focus/magnification control circuit 44 has been completed, the attention area determination circuit 40 supplies an image input command to the image input circuit 46. In response to this, the image input circuit 46 electrically scans and reads the image of the area A taken by the image sensor 20, amplifies it, converts it into digital data, and stores it in the image memory 48. This image is processed by the image recognition circuit 50, recognition is performed according to the purpose, and the results are displayed on the display 52. Further, this image is displayed as it is on the display 52 and used for monitoring from a remote location. Next, area B shown in FIG.
Similar processing is performed for .

Immediately after the processing in the spatial differentiation circuit 34 and the time differentiation circuit 36 is completed, the image transfer circuit 30 transfers the contents of the image memory 28 to the delayed image memory 32, and then the time (
At t+2d), the image photographed by the image sensor 14 is written into the image memory 28 via the image input circuit 26.

If this image is as shown in FIG. 4, there is a region in which the output value of the temporal differentiator 3B is not 0, and the 0 value evaluated by the evaluation circuit 38, for example, is added to the output value of the spatial differentiator 34. If the weight attached to the output value of the time differentiator circuit 36 is considerably larger than the weight given to the output value of the time differentiation circuit 36, the attention area determined by the attention area determination circuit 40 becomes the rectangular area C shown in FIG. This rectangular area C is photographed by the image sensor 20 and displayed on the display 52, and the image recognition circuit 50 recognizes whether or not there is a fire. Whether or not there is a fire can be determined based on the color and the presence or absence of fluctuations characteristic of fire.

In this way, a region with large spatial and temporal changes can be automatically imaged by the image sensor 20 without human judgment, and can be used for fire detection and monitoring.

In addition, regarding the image memory 28 and the delayed image memory 32, pixel data such as brightness is converted into features such as the area value of each figure (the area value of this figure is given to each pixel in t figures), and the The transformed image may be processed by the spatial differentiation circuit 34 and the temporal differentiation circuit 36.

(2) Second embodiment Next, a second embodiment of the present invention will be described.

Depending on the subject and the distance to the subject, the image captured by the imaging device 14 may become a low-resolution image in which a plurality of figures are scattered and aggregated, as shown in FIG. For example, if various products are packaged in boxes and transported on a belt conveyor, and one or more rows of characters written on the surfaces of six boxes being transported are photographed from a distance using the image sensor 14, A character string in a resolution image can be said to be a collection of multiple scattered figures. In such a case, if each - character is imaged by the image sensor 20 and the character is recognized, the processing time becomes long and it is necessary to further recognize the unit of the character string.

Therefore, in the second embodiment, an area of an appropriate size is determined to be an area of interest by determining the subjective outline of a set of scattered figures, rather than the outline of individual points, in the following manner.

FIG. 7 is an enlarged view of the vicinity between two points in FIG. 6, where each square represents one pixel, and the numerical value within the square represents luminance.

Blank cells indicate that the luminance is above a certain value. For example, using a 9x9 matrix interpolation filter F, place the center X of this interpolation filter F at a pixel whose luminance is above a certain value, and calculate the Find the average value of brightness, and center this average value at 1! i! Give as brightness of X. If this filtering process is performed on all pixels, interpolation will be performed between spots as shown in FIG. However, the luminance given in this way is not used during filter processing.

Through such processing, a contour (subjective contour) as a set of spots can be obtained, and the rest of the process is the same as in the first embodiment.

In the above, the interpolation area is defined as a luminance of a certain value or more, but depending on the image, the luminance of a certain value or less may be used as the complementary 170 area. Furthermore, the subjective contour can be obtained in the same manner even if the feature m is not the brightness but the area of a spot.

The * mark in FIG. 8 indicates omission of numerical description.

(3) Expansion Note that the present invention includes various other modifications.

For example, if the output of the image input circuit 26 is alternately supplied to the image memory 28 and the delayed image memory 32, there is no need to provide the image transfer circuit 30.

Further, depending on the application, the configuration may be such that only one of the spatial differentiation circuit 34 and the temporal differentiation circuit 36 is provided, or the spatial differentiation circuit 34 may be provided without the evaluation circuit 38.
Alternatively, the output of the temporal differentiation circuit 36 may be supplied to the region-of-interest determination circuit 40 so that both regions with large spatial changes and regions with large temporal changes are equally set as regions of interest.

Alternatively, an optical system may be arranged in which the image pickup elements +4 and 20 are fixed, and the field of view is moved between the nauf mirror IG and the variable magnification lens 1B. As such an optical system, it is possible to use, for example, the known optical system 6 used in the scanning optical system of a Raman spectrometer (Utility Model Publication No. 63-1999).
15815).

Furthermore, the low-resolution imaging device and the high-resolution imaging device may be separate TV cameras, and in this case, a plurality of high-resolution imaging devices may be provided. Furthermore, a high-resolution imaging device that images an area with large spatial changes and a high-resolution imaging device that images an area with large temporal changes may be separate devices.

Furthermore, the scope of use of the present invention is wide; for example, if it is used as a security camera installed in a wide area, a moving object tracking device, etc., the number of cameras installed and the image processing device can be reduced, and the area of interest can be focused more precisely. It becomes possible to take pictures with high resolution.

If used for astronomical observation or observation under a microscope, only the necessary data can be automatically obtained, making analysis easier.

[Effects of the Invention] As explained above, according to the image input device I according to the first or second invention, areas with large spatial changes are determined from images taken with a low-resolution imaging device, so it is noteworthy that The desired area can be automatically determined in a short time, and only this area is photographed using a high-resolution imaging device, making recognition, inspection, or observation easy, and the desired information can be obtained in a short time without waste. It has the excellent effect of performing various processing without human judgment, and greatly contributes to reducing equipment costs, saving labor, and making image input devices more intelligent.

Further, according to the third aspect of the present invention, since regions with large temporal changes are determined from images taken with a low-resolution imaging device, it is possible to automatically determine a region of interest in a short time.
The same excellent effects as the first or second invention described above are achieved.

[Brief explanation of the drawing]

FIGS. 1 to 5 show "First implementation Pi of the present invention". This person,
FIG. 1 is a partially cutaway perspective view showing the optical system of the image input device, FIG. 2 is a block diagram of the image input device, and FIGS. 3 to 5 are diagrams for explaining the operation of FIG. Fig. 3 shows the time and a low-resolution image at time (t+d), Fig. 4 shows a low-resolution image at time (t+2d), and Fig. 5 shows the area of interest C shown in Fig. 4. It is a figure showing a high-resolution image. 6 to 8 relate to the second embodiment of the present invention, in which FIG. 6 is a diagram showing an image having a region of interest consisting of a set of a plurality of scattered figures, and FIG. 7 is a diagram showing two points in FIG. 6. FIG. 8 is an explanatory diagram of interpolation filter processing in which the vicinity of the interpolation area is enlarged. FIG. 8 is a diagram corresponding to the seventh image after interpolation. 12 is the imaging lens I4. 20 is the image sensor 16 is a half mirror. I8 is the variable magnification lens 22a, and the x-y moving stage 22b is the x-y fixed stage. IO is the housing diagram at time t, (t+d). Low-resolution image Fig. 3 Low-resolution image at 1ll (t+2d) Fig. 4 Fig. 5 Fig. 6 Fig. 7 before interpolation processing magnifying the vicinity between two points in Fig. 6

Claims (1)

[Scope of Claims] 1) A low-resolution imaging device that images a first field of view, a high-resolution imaging device that images a second field of view narrower than the first field of view with a higher resolution than the low-resolution imaging device, and the second field of view. a field of view moving means for moving within the first field of view; and an image input device comprising: a change between adjacent pixel data of an image captured by the low-resolution imaging means or an image obtained by converting the image by a predetermined value. an attention area determining means for determining an area including the above portion as an attention area; and a control means for controlling the field of view moving means to make the second field of view correspond to the attention area. An image input device that captures images with high resolution. 2) A low-resolution imaging device that images a first field of view, a high-resolution imaging device that images a second field of view narrower than the first field of view with a higher resolution than the low-resolution imaging device, and a high-resolution imaging device that images the second field of view within the first field of view. and a field of view moving means for moving the image by an interpolation filter process to obtain a subjective contour of the image captured by the low-resolution imaging means or an image obtained by converting the image, and to calculate a subjective contour within the subjective contour. A region of interest determining means for determining the region of interest, and a control means for controlling the field of view moving means to make the second field of view correspond to the region of interest, and capturing an image of the region of interest with high resolution. An image input device characterized by: 3) A low-resolution imaging device that images a first field of view, a high-resolution imaging device that images a second field of view narrower than the first field of view with a higher resolution than the low-resolution imaging device, and a high-resolution imaging device that images the second field of view within the first field of view. and a field of view moving means that moves the image by the low-resolution imaging means, or an image obtained by converting the image, including a portion in which a change over time in each pixel data is greater than or equal to a predetermined value. A region of interest determining means for determining a region as a region of interest; and a control means for controlling the field of view moving means to change the second field of view to a field of view corresponding to the region of interest. An image input device characterized by: