JPH01162485A - Picture input device - Google Patents

Abstract

PURPOSE:To obtain the picture of a focused condition by selecting the mere picture element of a focused condition from the picture of plural distance image picking-up devices and synthesizing it. CONSTITUTION:Plural pictures obtained by plural distance image picking-up devices 21 are total differentiated respectively at a total differential circuit 24, and total differential results are stored to a total differential picture memory 25 respectively. The output of the memory 25 is inputted to an inter-picture difference circuit 26, a one story differential value is found by taking a one story difference and stored to a one story difference memory 27, moreover, a two story differential value is found by taking the on story difference and stored a two-story difference memory 28. A judging circuit 29 uses the output of memories 27 and 28 and selects a point, which the one story differential value is zero and the two story differential value becomes maximum for each picture element, extracts the picture at a picture synthesizing circuit 30, and writes and synthesizes at a corresponding position in a synthesizing picture memory 31.

Description

[Detailed description of the invention] [Table of contents] Overview Industrial field of application Conventional technology (Figures 12 and 13) Problems to be solved by the invention (Figure 14) Means for solving the problems (Fig. 1) Effect (Fig. 2 to Fig. 7) Example (Fig. 8 to Fig. 11) Effects of the invention [Summary] In-focus state can be maintained over the entire surface of a target object having multiple height levels. The purpose of this invention is to enable an image input device that obtains an image to obtain a completely focused image by selecting and combining only in-focus pixels from images from a multi-distance imaging device. , a multi-distance imaging means that images the object to be imaged on a plurality of original screens having different imaging distances, a total differentiation means that obtains a plurality of total differential screens from the multiple captured images, and a total differential screen that performs first-order differentiation of the total differential screen. a first-order differentiation means for obtaining a plurality of first-order differential screens; a second-order differentiation means for obtaining a plurality of second-order differential screens by second-order differentiation of all the differential screens; or an in-plane dispersion means for obtaining the dispersion of each image in the total differential screen. and the first differential value for each pixel is 0 and 2
a determination means for determining the distance in the optical axis direction at which the absolute value of the floor differential value is maximum or the distance in the optical axis direction at which the first floor differential value is 0 for each pixel and the variance value is the minimum for each image, and for each pixel. and an image synthesizing means for extracting and synthesizing pixels of the original screen corresponding to the distance determined.

[Industrial application field]

TECHNICAL FIELD The present invention relates to a human-powered image device that can obtain images that are in focus over the entire surface of a target object having multiple height levels.

In an imaging device, when multiple target objects are located at different distances from the imaging lens, or when each part of the surface of the same target object is located at a different distance from the imaging lens, the distance If the difference is not within the depth of focus determined by the lens, it is usually not possible to capture a clear image with all parts in focus.

However, in practice, it is necessary to obtain an image that is in focus over the entire surface even when the range in which the imaged object exists exceeds the depth of focus range of the imaging lens. There are cases.

[Conventional technology]

Conventionally, methods for capturing images of such a wide range of objects include methods of moving a sensor placed on the imaging plane with respect to a fixed imaging lens according to the imaging distance, or using a half mirror, etc. A method of dividing the optical path of an imaging lens and placing sensors at different focal positions to take images is currently being used.

FIG. 12 shows an example of a conventional multi-distance imaging device, in which the sensor is moved.

In FIG. 12, the imaging lens 11 is fixedly provided to the imaging device. At the imaging plane, the image is a CCD
It is detected by a sensor 12 such as an area sensor (video camera sensor), and the sensor 12 is mounted on a moving stage 13 and is configured to be movable according to the imaging distance.

Now, from the center of the imaging lens 11 to the object to be imaged 141°1
42, 14. The distance to , that is, the imaging distance is al.

When a2*a3, the distance from the lens center to the imaging plane, that is, the imaging distance is bl, b2. b3, if the moving stage 13 is controlled to move to the imaging distances b1 to b3 corresponding to the imaging distances a1 to a3, then all of the imaged objects 141 to 143 will be A focused image can be obtained.

FIG. 13 shows another example of a conventional multi-distance imaging device, illustrating a case where a plurality of sensors are used.

In FIG. 13, the optical path of the imaging lens 11 is divided by half mirrors 151, 152 and detected by sensors 121, 122, 123, respectively. , 12. , that is, the imaging distance b1.
b2° and b5 are selected to be the focusing distances when the distance to the object to be imaged 141.142.145, that is, the imaging distance is a1wa2+a3. Therefore the 13th
In the case of the figure as well, focused images can be obtained for all of the imaged objects 141 to 143.

[Problem that the invention seeks to solve]

The conventional multi-distance imaging device shown in FIGS. 12 and 13 is capable of imaging an object to be imaged at an imaging distance corresponding to a certain imaging distance in a focused state. However, objects to be imaged at other imaging distances are not within the focal depth range of the imaging lens, and therefore cannot be imaged in focus on the same screen.

FIG. 14 shows an example of an image taken by a conventional multi-distance imaging device, in which (a) shows the relationship between the imaging device and the object to be imaged. When the surface 16 is tilted from the vertical, the imaging distance differs for each part of the surface 16. Each part 16 of the current surface 16. , 162.1
65 is assumed to have a smaller 111st imaging distance,
The image on the sensor 12 when the focused imaging distance is set according to each imaging distance is (bl, (c
l, (as shown in dl. In each image, the position 161, 162, corresponding to the imaging distance at that time is
The portion corresponding to 163 is in focus, but the other portions are out of focus and a clear image cannot be obtained.

The present invention aims to solve these problems, and by selecting and compositing only in-focus pixels from images taken by a multi-distance imaging device, the imaging distance can be adjusted to the depth of focus of the imaging lens. The object of the present invention is to make it possible to obtain a completely focused image even for an object to be imaged that changes over a range.

[Means for solving problems]

FIG. 1 shows the basic configuration of the present invention, (
a) shows the invention of $1, which includes a multi-distance imaging means 1, a total differentiating means 2, a first-order differentiating means 5, a second-order differentiating means 4, a determining means 5, and an image synthesizing means 6. It has been shown that the Further, (b) shows a second invention, which includes a multi-distance imaging means 1, a total differentiating means 2, a first-order differentiating means 3, an in-plane dispersing means 7, a determining means 8, and an image synthesizing means 6. It is shown that it is equipped with.

The multi-distance imaging means 1 images the object to be imaged on a plurality of screens corresponding to different imaging distances.

The total differentiation means 2 performs total differentiation on each pixel of a plurality of captured images to obtain a plurality of total differential screens.

The first-order differentiating means 6 performs first-order differentiation between the plurality of total differential screens for each pixel to obtain a plurality of total differential screens.

The second-order differentiating means 4 performs second-order differentiation for each pixel between the plurality of total differential screens to obtain a plurality of second-order differential screens! That's what I'm looking for.

The determining means 5 determines, for each pixel, the distance in the optical axis direction at which the first differential value is 0 and the absolute value of the second differential value is maximum.

The image synthesis means 6 extracts and synthesizes pixels of the original screen corresponding to the distance determined for each pixel.

The in-plane dispersion means 7 obtains the dispersion of each image in a plane perpendicular to the optical axis in the entire differential screen.

The determining means 8 determines the distance in the optical axis direction at which the first-order differential value is 0 for each pixel in the entire differential screen and the dispersion value is minimum for each image.

[For production]

The same object to be imaged is imaged on a plurality of screens corresponding to different imaging distances depending on the focal depth range of an imaging lens. Then, each of the plurality of captured images is fully differentiated.

Figure 2 explains the total differential of the image, (al
indicates an object, and assumes that a black and white pattern as shown is to be imaged. (b) is the distance Z from the sensor to the imaging lens
When changes, each sensor position Z1 + Z 2 * Z
5, and is indicated by the image output e on the xx' line. In this case, Zl<Z2
<Z3. Here, the distance Z2 is closest to the in-focus position, so the image output e rises sharply, but the distance z,
, z3 indicates an out-of-focus state. (c
l is each distance Z 1 rz2. This shows the total differential data at z5. Here, the total differential g is defined with respect to the image output e by the following equation.

As shown in FIG. 2 (cl), the total differential output g exhibits a characteristic in which its value becomes the largest at the in-focus position, and in the present invention, this characteristic is utilized to determine focus.

FIG. 3 shows the total differential data of edges. f
At indicates an edge, and for example, as shown in the figure, an edge parallel to the Y direction is imaged, but the direction of the edge does not have to be in this direction. (bl shows that a certain curved surface can be obtained by drawing all the differential data on the X-Z plane without changing the distance 2 in the optical axis direction.

Figure 4 explains the properties of total differential data,
Changes in the image output e and the total differential output g at the edge of the object as shown in FIG. 2 are shown for a point A coinciding with the edge and a point B slightly away from the edge.

In Fig. 4, (al shows the in-focus state, (b) shows a slightly out-of-focus state, and (C) shows a markedly out-of-focus state. In each figure, at point A corresponding to the edge, the total differential output g matches the peak in both the in-focus and out-of-focus states.However, at point B, which is slightly off the edge, the total differential output g decreases in the in-focus state, but may increase in the out-of-focus state. It has been shown that there is.

FIG. 5 explains changes in total differential data in a total differential image. In the figure, (a) shows the change in the total differential output g in the optical axis Z direction at the edge, and (b) shows the change in the total differential output g in the Z direction at a point slightly away from the edge.

A shown by a broken line in FIG. 3 indicates a change in the total differential output g at the edge, and corresponds to K in FIG. 5(a). B also shows the change in the total differential output g at a point slightly away from the edge, which is explained in Figure 5(b), and (a) shows the change in the total differential output g at the edge indicated by (cl) indicates the change in the total differential output g at a point slightly away from the edge indicated by (bJ) in FIG.

As is clear from Figure 6 (aJ, (b)), point 2 on the edge is the in-focus position, and point 2 is slightly off the edge, as the second-order differential output is clear at this point. is the in-focus position.

This explains how to determine the in-focus position by determining the focal position. As shown in Figure 5 (al), for points on the line passing through the edge, the point with the total differential is the most in-focus point, but for points on the line slightly away from the edge, Since the decimal of the total differential output g is generated, an additional condition is required.In the second invention, the variance of the total differential data is used as such a condition.

In FIG. 7, (al indicates the dispersion of the total differential output g in the cross section C perpendicular to the optical axis passing through the peak point cY in the case of aJ in FIG. 5, and the dispersion σ
C is small. In addition, (bl is the dispersion of the total differential output g in the cross section perpendicular to the optical axis passing through one of the plurality of peak points d in the case of Fig. 5 (bl), and the dispersion σD in this case is It's thick.

The same applies to other peak points. Furthermore, Figure 5 (b
At points other than the peak in the case of l, for example at b, the dispersion is extremely thick.

The in-focus position can be determined by finding the position in the vicinity where the variance σ2=σX2+σy2 is the minimum.

In the present invention, when the focus position is determined for each pixel in this way, pixels corresponding to the focus position are extracted from data in a plurality of original image memories and combined into one screen in a composite image memory. By doing this, you can obtain an image that is in focus over the entire screen.

〔Example〕

FIG. 8 is a block diagram showing the overall configuration of an embodiment of the present invention. In the figure, reference numeral 21 denotes a multi-distance imaging device, which has the configuration shown in FIG. 12 or 13, etc., and includes an imaging lens and sensors such as a CCD area sensor, and has n n images corresponding to the imaging distance can be obtained. 22 is an image input circuit that digitizes the image signal obtained from the imaging device 21 and outputs it for each image. This Ringo is input into the original image memory 23 consisting of n sides and stored for each image.

In the embodiment of FIG. 8, in order to further perform image processing, the total differential circuit 24. Total differential image memory 25° Inter-image difference circuit 26. 1st floor difference memo IJ 27. 2nd floor difference memory 2
81 judgment circuit 299 image composition circuit 309 composition image memory 31, each part is connected to each other via bus 32, and is configured to operate according to a fixed sequence under the control of control circuit 33.

FIG. 9 is a diagram showing a configuration for image processing in the embodiment shown in FIG. 8, and the same parts as in FIG. 8 are designated by the same numbers.

Different imaging distances for imaging by the imaging device 21
In the original image memory 23, the images are stored in n-side image memories 231 to 23n for each image. On the other hand, the n images are fully differentiated in the n total differentiation units 241 to 24n in the total differentiation circuit 24, and the total differentiation results are stored in the n-side image memories 251 to 25n in the total differentiation image memory 25, respectively. .

The output of the total differential image memory 25 is inputted to the inter-image difference circuit 26, which calculates the first-order differential value by taking the first-order difference, and outputs (n-1) first-order difference memos IJ 271-2+2.
72-31...+27(n-1)-n is stored in the first-order difference memory 27. That is, the image memory 231
The difference between the output of the image memory 232 and the output of the image memory 232 is taken and stored in the first-order difference memory 27+-2, and the difference between the output of the image memory 232 and the output of the image memory 233 is taken and stored in the first-order difference memory 272-5. Similarly, the difference between the output of the image memory 23n- and the output of the image memory 23n is obtained and stored in the first-order difference memory 2sn-1)-0.

Furthermore, the output of the first-order difference memory 27 is sent to the inter-image difference circuit 26.
is inputted into the memory 281-2-5, the second-order differential value is determined by taking the first-order difference, and (n-2) second-order difference memories 281-2-5
, "°l 28(n-2)-(n-j)-n. That is, the output of the first-order difference memory 27,-2 and the output of the first-order difference memory 272-3. The difference is taken and stored in the second floor difference memory 281-2-5, and in the same manner, the first floor difference memory 27(n-2)-
(n-+) output and the first-order differential memory 27 (n-N-0
The difference between the output of
-(n-+)-n.

Next, the determination circuit 29 uses the output of the first-order difference memory 27 and the output of the second-order difference memory 28 to determine the distance Z for each pixel.
Select. The image synthesis circuit 60 stores the original image memory 2 corresponding to the distance selected by the determination circuit 29 for each pixel.
3 is extracted and written to the corresponding position in the composite image memory 31. When the control circuit 63 completes such processing for all pixels of the image in the original image memory 23 and writes all the pixels in the composite image memory 30, the control circuit 63 receives the composite image signal from the composite image memory 61. Output. This image is a desired image and is composed only of pixels in focus.

FIG. 10 shows another embodiment of the present invention, in which the same parts as in FIG. 8 are designated by the same numbers, and 64 is an in-plane distributed circuit.

In FIG. 10, n images corresponding to n imaging distances are obtained by the multi-distance imaging device 21, and the image input circuit 22
The storage of each image in the original image memory 23 consisting of n planes through the process is the same as in the embodiment shown in FIG.

The embodiment of FIG. 10 further includes various parts for image processing, but unlike the case of FIG. 8, it lacks a second-order differential memory and has an in-plane distribution circuit 64 instead. Similar to the case of FIG. 8, each part is connected to each other via a bus 32 and is configured to operate according to a fixed sequence under the control of a control circuit 33.

FIG. 11 is a diagram showing the configuration for image processing in the embodiment shown in FIG. 10, and the same parts as in FIG. 10 are indicated by the same numbers.

Different imaging distances for imaging by the imaging device 21
The image is stored in the n-side image memory 2 in the original image memory 26.
31 to 23n, respectively. Further, n images are obtained by n total differentiation units 241 to 2 in the total differentiation circuit 24.
4n, and the results are stored in the n-plane image memories 251 to 25 in the total differential image memory 25.
n respectively.

The output of the total differential image memory 25 is input to the inter-image difference circuit 26 to take the first-order difference, and the result is stored in (n-1) first-order difference memories 271-2 in the first-order difference memory 27.
+272-3 +...+ Stored in 27(n-1)-1.

That is, the difference between the output of the image memory 231 and the output of the image memory 232 is taken and stored in the first-order difference memory 27. -2, and stores the difference between the output of the image memory 262 and the output of the image memory 233 in the first difference memory 272-3,
Thereafter, in the same manner, the difference between the output of the image memory 23n-4 and the output of the image memory 23n is calculated to create a first-order difference memo IJ.
27(n-1)-0. The above processing is the same as in the embodiment shown in FIG.

The in-plane dispersion circuit 34 uses the image output of each image memory of the total differential image memory 25, and divides it into small f sections.

■ Find the variance σ = σχ + σy in the y plane.

The judgment circuit 29 is a first-order differential memory 271-2+ 272-
From the first-order differential value at 3 +...+ 27(n-1)-n, find the point where =0 for each pixel t. Among the points obtained in this way, if there is a point T that satisfies the condition t = 0 in multiple planes on the total differential image memory, the in-plane dispersion circuit 34 calculates the dispersion σ2=
The one with the minimum distance of σx2+σy2 is selected.

The image synthesis circuit 30 extracts the image of the surface of the entire image memory 23 corresponding to the distance selected by the determination circuit 29 for each pixel, and writes it into the corresponding position in the synthesized image memory 31.

The control circuit outputs a composite image signal from the composite image memory 31 when writing of all pixels in the original image memory 23 is completed. As in the case of the embodiment shown in FIG. 6, this image is composed only of pixels in a focused state.

〔Effect of the invention〕

As explained above, according to the present invention, from all differential screens of multiple images obtained by a multiple distance imaging device, the point where the first differential value is 0 and the second differential value is maximum for each pixel, or for each pixel, The point where the first differential value of Even for an object to be imaged whose depth of focus changes beyond the depth of focus range of the imaging lens, it is possible to obtain a fully focused image.

[Brief explanation of the drawing]

Figure 1 is a diagram showing the principle configuration of the present invention, Figure 2 is a diagram explaining the total differential of an image, Figure 3 is a diagram showing a total differential image of an edge, and Figure 4 is a diagram showing the properties of the total differential data. 5 is a diagram illustrating changes in total differential data in a total differential image. FIG. 8 is a block diagram showing the overall configuration of an embodiment of the present invention. FIG. 10 is a block diagram showing the overall configuration of another embodiment of the present invention, and FIG. 11 is a diagram showing the configuration for image processing in the embodiment shown in FIG. 12 is a diagram showing an example of a conventional multiple distance imaging device. FIG. 13 is a diagram showing another example of a conventional multiple distance imaging device. FIG. 14 is a diagram showing a conventional multiple distance imaging device. FIG. 3 is a diagram showing an example of an image taken by the device. 21...Multi-distance imaging device 22...Image input circuit 23...Original image memory 24...Full differential circuit 25...Full differential image memory 26...Inter-image difference circuit 27...1 Floor difference memory 28...Second floor difference memory 31...Synthetic image memory 52...Bus 33...Control circuit 64...In-plane distributed circuit

Claims (2)

[Claims] (1) A multi-distance imaging means (1) that images the object to be imaged on a plurality of original screens corresponding to different imaging distances, and a plurality of total differentials by fully differentiating the plurality of images for each pixel. a total differentiation means (2) for obtaining a screen; a first-order differentiation means (3) for obtaining a plurality of first-order differential screens by performing first-order differentiation on a pixel-by-pixel basis between the total differential screens; a second-order differentiation means (4) for obtaining a plurality of second-order differential screens by performing second-order differentiation on An image input device comprising: a determining means (5) for determining; and an image synthesizing means (6) for extracting and synthesizing pixels of an original screen corresponding to the determined distance for each pixel. (2) A multi-distance imaging means (1) that images the object to be imaged on a plurality of original screens corresponding to different imaging distances, and a plurality of total differentials by fully differentiating the plurality of images for each pixel. a total differentiation means (2) for calculating a screen; a first-order differentiation means (3) for performing first-order differentiation for each pixel between the total differential screens to obtain a plurality of first-order differential screens; In-plane dispersion means (7) for determining dispersion in a plane perpendicular to the optical axis; and determination means for determining the distance in the optical axis direction at which the first differential value is 0 for each pixel and the dispersion value is minimum for each image. An image input device characterized by comprising (8) and image synthesis means (6) for extracting and synthesizing pixels of the original screen corresponding to the determined distance for each pixel.