JPH0411468A - focus detection device - Google Patents

Abstract

PURPOSE:To attain highly stable detection at all times with high accuracy by providing a correction means detecting the movement of a picture and correcting the movement and detecting a focus based on a signal component whose movement is corrected by the movement detection means. CONSTITUTION:An X axis projection circuit 7 obtains an X axis component signal 8 from a moving vector obtained from a moving vector arithmetic circuit 6. an edge detection circuit 9 detects an edge from information such as a tilt with respect to a picture signal and selects an edge with a larger tilt with respect to the X axis as an edge for focus discrimination, for example. An edge calculation circuit 10 calculates the edge from the density difference of the edge and the tilt of the edge and outputs the result as an edge width signal 11. The signal 11 is subtracted from a signal 8 and the result is fed to a comparator 12, and the signal is a corrected edge width signal. The comparator 12 writes a smaller edge width signal to a memory 13 in two corrected edge width signals and gives a control signal to a lens control circuit 14. Thus, the deterioration in the accuracy such as an unsharpened edge is prevented.

Description

DETAILED DESCRIPTION OF THE INVENTION (Field of Industrial Application) The present invention relates to a focus detection device that detects a focus state using an imaging signal and is suitable for use in imaging devices such as television cameras and electronic cameras.

(Prior Art) In recent years, the development of video equipment such as video cameras, electronic cameras, etc. has been remarkable, and it has become essential to include an automatic focus adjustment device as a function of the equipment from the viewpoint of operability.

By the way, there are two types of focus detection devices: a passive device that obtains a focus signal by correlating images captured by a twin-lens optical system, and a passive device that obtains a focus signal by correlating images taken with a twin-lens optical system. Active automatic focus adjustment devices are used that determine whether the object is in focus.

On the other hand, recently, for imaging devices such as television cameras that can obtain video signals, devices that perform focus detection by image processing the video signals have been developed. Devices using this type of image processing obtain a signal for detecting the focus state from the image signal, so they can perform focus detection regardless of the subject distance, etc., and can also project infrared light. The development of this method is progressing rapidly because it eliminates the need for special elements and circuits that would otherwise be required for conventional processing, and has high precision.

Figure 1O is a diagram for explaining an example of a conventional focus detection method using image processing, and shows the intensity distribution of the edge portion of the subject image in an out-of-focus state and in an in-focus state. , Figure (a) shows an out-of-focus state, and Figure (b) shows an in-focus state.

In FIG. 5A, reference numeral 101 indicates the intensity distribution of the edge portion of the subject image when the subject image is out of focus, and the distribution is rounded due to blurring, and the width of the edge portion is wide.

Further, in FIG. 2B, reference numeral 102 indicates the intensity distribution of the edge portion of the subject image when in focus, and is shown for the same location as the intensity distribution 101 of the edge portion when out of focus. When in focus, it becomes a narrow, steep edge.

Therefore, the width of the edge portion of the subject image is detected, and in-focus or out-of-focus is determined based on the edge width. That is,
Focus is determined by utilizing the fact that the edge width is small when in focus.

The edge intensity distribution 101 when out of focus is the distribution of edge portions detected from the video signal, and the edge width I21 is expressed by the following equation.

ρ1=d,/(dl,(x)/dx)・・・・・・・・・
-(1) Here, dl represents the strength difference at the edge portion.

I, (x) is a function representing the intensity distribution 101 of the edge when out of focus, and therefore dI+(x)/dx represents the slope of the edge. The slope of this edge can be used by calculating the average value of the slope of focus from the point where the edge starts to rise until it becomes flat again.

Similarly, during focusing, the edge width ρ2 is calculated from the intensity distribution 102 during focusing using the following equation.

I22=d2/(dI2(x)/dx)・・・・・・
...(2) Here, d2 is the intensity difference l2(x) at the edge portion is a function representing the intensity distribution 102 of the edge at the time of focus, dI
2(xi/dx is the slope of the edge. d2 is approximately the same value as d, and dL(x)/dx is dI, (x)/d
It becomes thicker than x.

Therefore, β2 becomes a smaller value, and it can be seen that the edge width has become smaller, and it is determined that the focus has been achieved.

As described above, a method of calculating the edge width from the density difference and slope of the edge portion, and determining that the smaller the value is, the closer to in-focus is generally used as a position method for focus detection using image processing.

(Problems to be Solved by the Invention) However, according to the above-mentioned focus detection device, when the image has movement, that is, when the subject itself is moving, or due to camera shake, panning, etc. If the entire image is shifted, the image becomes blurred due to the movement of the image, and therefore the width of the edge also increases in accordance with the movement, making it impossible to accurately detect the focus.

(Means for Solving the Problems) The present invention has been made for the purpose of solving the above-mentioned problems, and is characterized by a motion detection means for detecting the movement of an image from the image pickup signal. a motion correction means for correcting the motion of the image based on the output of the motion detection means; and a signal component that changes depending on the focus state from among the imaging signal whose motion component of the image has been corrected by the motion correction means. and a focus detection means for extracting and performing focus detection.

(Function) As a result, image movement can be detected from the video signal, and focus detection can be performed using a signal that compensates for deterioration in resolution due to the image movement. A highly accurate focus detection device can be realized.

(Embodiments) Hereinafter, embodiments of the focus detection device according to the present invention will be described in detail with reference to the respective drawings.

FIG. 1 is a block diagram showing a first embodiment of a focus detection device according to the present invention.

In the figure, 1 is a subject, 2 is an imaging lens, and 3 is an imaging element such as a CCD or an imaging tube that photoelectrically converts the subject image formed on the imaging surface by the photographing lens 2 to generate an imaging signal. 4 is an amplifier that amplifies the image signal output from the image sensor 3 to a predetermined level, and 5 is an AD that converts the input analog image signal into a digital signal.
Converter, 6 is a motion vector calculation circuit for obtaining a motion vector of an image from the image signal converted into a digital signal by the AD converter 5, and 7 is a motion vector calculation circuit for projecting the motion vector onto the X axis, which is the horizontal direction of the screen. The X-axis projection circuit 8 is a component representing the X-axis component of the motion vector.

9 is an edge detection circuit that detects the width of the edge portion of the subject image; 10 is an edge width detection circuit that detects the width of the edge portion detected by the edge detection circuit 9; and 11 is an edge width signal.

12 is a comparator, and 13 is a memory. 14 is a lens control circuit, 15 is a lens control signal, and 16 is a lens drive device for moving the imaging lens 2 in the optical axis direction and focusing it.

On the other hand, 17 performs signal processing such as gamma correction and various filtering on the imaging signal output from the AD converter 5,
A video signal processing circuit that outputs a standard television signal in a television camera, etc.; 18 is a DA that converts the signal output from the video signal processing circuit 17 into an analog signal;
Converter 19 is a synchronizing signal adding circuit that adds a synchronizing signal to the image signal output from the DA converter 18, and 20 is a video output signal.

In the above configuration, the imaging lens 2 forms an image of the subject 1 on the imaging surface of the imaging element 3, and the imaging element 3 photoelectrically converts the subject image and outputs an image signal. The image formed on the image sensor 3 moves due to movement of the image sensor 2, the image sensor 3, or the subject 1.

The image signal output from the image sensor 3 is amplified by an amplifier 4 and converted into a digital signal by an AD converter 5.

A part of the digitized image signal is input to the motion vector calculation circuit 6. The motion vector calculation circuit 6 has a built-in frame memory in which the previous frame image is stored, and the motion vector of the image is calculated by comparing the current frame image with the previous frame image stored in the frame memory. Calculate. As a method of motion vector calculation, methods such as the so-called representative point matching method and gradient method can be used.

The X-axis projection circuit 7 takes the motion determined by the motion vector calculation circuit 6 and obtains the X-axis component signal 8.

The digitized image signal is sent to the edge detection circuit 9. The edge detection circuit 9 detects an edge of the image signal based on information such as the inclination, and selects, for example, the edge with the largest inclination with respect to the X-axis as the edge for focus determination. As described above, the edge width calculation circuit 10 calculates the edge width from the edge density difference and the edge slope, and outputs it as the edge width signal 11. The edge width signal 11 is sent to the comparator 12 after subtracting the X-axis component signal 8 of the motion vector. The signal sent to comparator 12 is an edge width signal corrected for image motion.

The comparator 12 then compares the input corrected edge width signal with the data in the memory 13.

In the memory 13, a corrected edge width signal in the previous field or frame is stored.

Comparator 12 writes the smaller value of the two corrected edge width signals into memory 13 and sends a control signal to the lens control circuit.

This control signal continues driving the current imaging lens 2 in the same direction when the newly input edge width signal is smaller than the previous one, and continues to drive the current imaging lens 2 in the same direction (if the newly input edge width signal becomes wider than the previous one, in the opposite direction). This is a signal that controls the amount of lens drive.Also, a signal may be sent that gradually changes the amount of lens drive depending on the amount of change in the corrected edge signal.Alternatively, if the amount of change is very small, When the amount of change changes from negative to positive, it may be determined that the in-focus position is reached and a control signal is sent to stop the amount of lens drive.To perform detailed control, the memory 13 stores several frames worth of data. Preferably, the corrected edge width signal is stored.

The lens control circuit 14 generates a lens drive signal 15 based on the control signal sent from the comparator 12, and the lens drive device 16 drives the imaging lens 2 based on the lens drive signal 15.

On the other hand, the digitized image signal is inputted to a video signal processing circuit 17, inputted to a DA converter 18, converted into an analog signal, synthesized with a synchronization signal by a synchronization signal adding circuit 19, and outputted as a video signal.

The principle of the present invention is shown below.

FIG. 2 is a diagram conceptually showing the movement of an image.

In the figure, 21 is an output screen.

For example, consider the screen of a monitor display. 22 is a previous image, and 23 is a current image. 24 is the previous image 2
2 is the motion vector when moving to the current image 23.

25 is a projection vector of the motion vector 24 onto the X axis.

On the output screen 21, the video signal 20 of FIG. 1 is displayed.

For example, if there is camera shake or panning, the displayed image will change from the previous image 22 to the current image 23 when moving from one time to another, even if the subject 2 is a stationary object.
Causes movement as in

Therefore, the motion vector calculation circuit 6 calculates a motion vector 24, and the X-axis projection circuit 7 calculates a projection vector 25, which is output as the X-axis signal component 8.

Here, since the current image 23 is accompanied by movement, the value of each pixel is integrated in the direction of the motion vector 24. Therefore, the current image 23 shown in FIG. 2 actually only shows the positions of the centers of gravity of the sides of each figure.

FIG. 3 is a front view showing the movement of the optical system.

In the figure, 31 indicates one cell when the image sensor is a solid-state image sensor such as a CCD.

For convenience of explanation, it is assumed that the direction in which the image moves matches the direction in which one of the cells of the image sensor 3 is arranged. Image sensor 3
photoelectrically converts the pattern of the optical image on the image sensor 3 during a certain predetermined exposure time. 32 is an optical image at the start of exposure, and 33 is an optical image at the end of exposure. As in the case of FIG. 2, the optical image 32 at the start of exposure has moved at the end of exposure due to movement of the image during exposure, and becomes the optical image 33 at the end of exposure. Motion vector 2 shown in Figure 3
4 is an optical image 32 at the start of exposure and an optical image 33 at the end of exposure.
It shows the movement between. This is almost the same as the one shown in FIG. 2, but strictly speaking it is slightly different.

That is, in FIG. 2, it is the motion vector of the image between approximately the middle of the exposure time of the two images. On the other hand, in FIG. 3, the motion vectors of the currently targeted image are shown from the start of exposure to the end of exposure.

Therefore, when the amount of motion of the image changes rapidly, the values of the two motion vectors may differ.

- For boat fishing, it is safe to assume that the two are almost the same, but since the motion vector calculation circuit 6 obtains the motion vector 24 shown in FIG. 2, it may be used after correction.

FIG. 4 is a characteristic diagram showing a point spread function due to image movement. 41 is a point spread function.

The length of the projection vector 25 is assumed to be C. At this time, subject 2
The line formed by one point moves during the exposure time and becomes roughly like a point spread function 41. If this is h (x), then h (x) =Rect(x/a) ・=・=
--(3). In other words, the image being captured is not only blurred due to defocus, but also blurred due to movement according to equation (3).

FIG. 5 is a characteristic diagram showing edge width correction.

51 is the video signal of the edge part, whose intensity distribution is I
Let it be (x). This corresponds to a signal directly obtained from the image sensor 3. Reference numeral 52 indicates the intensity distribution of the edge portion when there is no movement in the image, and let this intensity distribution be I'(x).

The shape of the 1° (x) edge is determined by the shape of the original subject and the amount of defocus.

The edge portion video signal 51 is a video signal of a portion selected by the edge portion detection circuit 9. In the edge width calculation circuit IO, the edge width ρ is determined from the edge density difference d and the slope dI(x)/dx.
, β=d/ (ar (x)/dx)...(4
). However, the flood also includes blur due to image movement.

There is a relationship between I (x) and I' (x) as shown in the following equation.

I (x)=I' (x) *h (x)...
(5) Here * represents a convolution operation.

h (x) is a function of the width C as shown in Figure 4,
Therefore, as shown in FIG. 5, I (x) becomes I' (x
), the slope of the edge widens by approximately c/2 to the left and right. Therefore, if the edge width for I'(x) is ρ, then the following equation approximately holds true.

℃＝ρ'＋Q・・・・・・(6)
The edge width l obtained directly from the video signal 51 of the edge portion includes the size C of the projection vector 25, has a larger value than originally, and changes depending on the movement of the image. In equation (6), it is clear that the flood can be found by subtracting C from °C, and if focus is determined based on 4°, it will not be affected by the movement of the image.

That is, as shown in FIG. 1, the X-axis signal component 8 having a value of C is subtracted from the edge width signal 11 having a value of ρ,
By determining β' and inputting it to a comparator and using it as a signal for determining focus, focus detection can be performed without being affected by image movement.

In the explanation here, it is assumed that the intensity distribution of the edge portion is obtained along the X-axis, and the motion vector is also projected onto the X-axis. Since the motion vector 24 is usually determined in the form of an X-axis component and a y-axis component, the X-axis projection process is not actually necessary at this time. However, if the intensity distribution of an edge portion is to be determined on an axis other than the horizontal axis, that is, the X axis, for example, in the steepest direction of the edge portion of interest, and the edge width is calculated, projection of the motion vector is required. The magnitude of a projection vector can be easily determined as the value of the inner product of a unit vector in that direction and a motion vector.

FIG. 6 is a block diagram showing another embodiment of the focus detection device according to the present invention.

In the figure, 61 is a frame memory, and 62 is a filter. 63 is a parameter setting circuit for providing parameters to the filter 62. 64 is a filtered video signal.

In this embodiment, a signal read out from the image sensor 3 is subjected to filtering processing based on a motion vector, and focus detection is performed from the signal.

Similar to the first embodiment, the AD converter 5 outputs a digitized video signal. The frame memory 61 temporarily stores this signal.

The digitized video signal is also sent to a motion vector calculation circuit 6, where a motion vector 24 is calculated. The frame memory 61 operates as a delay element for synchronizing the video signal when the calculation of the motion vector 24 is completed.

The signal read from the frame memory 61 is passed through the filter 62.
and filtering processing is performed. Filter 6
The frequency characteristics of No. 2 are set by the parameter setting circuit 63,
Parameters for the filter 62 are given from the calculation results of the motion vector 24 to correct image blur caused by motion.

The filtered video signal 64 is sent to the edge detection circuit 9, which selects the edge in the same manner as in the first embodiment, and from that point onwards, the focus determination operation is performed to drive the imaging lens 2. Let's do it.

On the other hand, the filtered video signal 64 is sent to the video signal processing circuit 17, and is finally output as the output video signal 20, similarly to the first embodiment.

The principle of this embodiment will be explained below.

When the magnitude of the motion vector 24 is a and the spatial coordinate along the direction of motion is Xo, the point spread function h' (x) generated by the motion vector 24 is h' (x) = Rec t (x'
/a)・・・・・・・・・(7) It becomes. This image deterioration is corrected by filtering.

FIG. 7 is a graph showing characteristics of a filter for filtering. 71 is a frequency characteristic indicating image deterioration due to movement.

If f is the frequency and the frequency characteristic 71 is expressed as H(f),
Since H(f) is the Fourier transform of the point spread function h'(x'), H(f) = (sin 1af) /if - (8)
It is.

72 is the frequency characteristic of the filter, and when expressed as P(f), p (f)=1/H(f)=πf/(sin xaf)---(9).

Such a filter is called an inverse filter. That is, H(f) ・P (F) = 1,
H(f) by filtering with P(f)
This is because the image deterioration caused by this is compensated for.

By filtering using this filter in the filter 62 of FIG. 6, the resolution of the output video signal 20 is increased and a good image can be obtained. However, since the inverse filter becomes infinite at the frequency where H(f) becomes O, it can only be realized approximately. Further, the frequency range may be a range where the frequency spectrum of the image signal exists.

As is clear from FIG. 7 and equations (8) and (9), the magnitude a of the motion vector 24 is included as a parameter in the degradation characteristic H(f) and the compensation filter characteristic P(f). Furthermore, the X° axis and the f axis are along the direction of the motion vector 24, and P (f) is the motion vector 2.
It can be seen that it depends on the size and direction of 4.

Therefore, it is desirable that the filter 62 adaptively change the filter according to the motion vector 24.

There are two methods for performing filtering processing in the filter 62. One is filtering on the frequency axis, where the image signal read from frame memorandum 1 is Fourier transformed using FFT (fast Fourier transform method), multiplied by an inverse filter P (f) and inverse Fourier transformed, and filtered. This is to obtain an image signal.

Another method is filtering on the time axis,
In this method, the inverse filter P (f) is subjected to inverse Fourier transform to obtain an impulse response, and the impulse response is convolved with the image signal from the frame memory 61 to obtain a filtering output.

When convoluting an inverse filter on the time axis, it is possible to start from the impulse response S (x) of the filter as shown below.

S(x') 2 δ°(X) * [Σδ(x-(2-1)a/2)sgn(x')
]・・・・・・・・・(10) Here, K is the proportionality constant δ(Xo) is the delta function, δ'(
x') is the derivative of the delta function.

* is a symbol representing convolution, and is also . Equation (10) is obtained by Fourier transforming Equation (9) using a delta function.

FIG. 8 is a graph showing the impulse response of the inverse filter.

81 is the impulse response of the inverse filter, (
10) Expression.

Since equation (10) continues indefinitely in the X° axis direction, it is necessary to terminate it midway. For this reason, it is desirable to multiply the signal by a window function such as a Hamming window before using it for filtering processing.

In an inverse filter, the characteristic is to have a large gain in the frequency region where most of the image information is lost due to image deterioration, that is, around the frequency where the value of the frequency characteristic 71 is O, and in the high frequency region where there is originally little image information, and often the SN The output image will have a poor ratio. For this reason, a Wiener filter or the like may be used as the filter 62.

The frequency characteristics of Wiener filter R(f) can be expressed as follows.

・・・・・・・・・・・・ (l 1 ) Here, Φ−
f), Φ, and (f) are noise and image signal power spectrum, respectively, and the subscript * indicates complex conjugate.

Φl1(f), Φ, (f) are difficult to obtain accurately, so Φ. (f) is constant assuming white noise, and Φ8(
f) is Gaussian. Or Φ, (f)/Φ word f)
It is determined in advance (.

In a Wiener filter, the value becomes almost the same as an inverse filter at a frequency where the signal component is sufficiently large compared to the noise component, and conversely, the value approaches 0 where the noise component is larger than the signal component.

FIG. 9 is a graph showing the frequency characteristics of the Wiener filter. 91 is the frequency characteristic of the Wiener filter. When compared with the frequency characteristic 72 of the impulse filter, it is clear that the gain is small in the frequency region where the SN ratio is poor. Even in the Wiener filter, the motion vector 2
Similarly to the case of the inverse filter, it is desirable to adaptively change its characteristics according to the filter.

Various other filters can be used in the filter 62. Since the frequency characteristic 71 has the characteristic of a low-pass filter, the filter characteristic of the filter 62 generally has the characteristic of a bypass filter or a band-pass filter.

In the filtered video signal 64, the edge portion of the edge signal 64 becomes a signal similar to that shown in FIG. That is, before filtering, the image signal 51 at the edge portion looked like the image signal 51, but after filtering, it becomes like the intensity distribution 52 when there is no movement in the image.

Therefore, if the edge signal width is calculated from the filtered video signal 64 and used for focus determination, stable focus detection can be performed with almost no influence from image movement. In this case as well, the value of the edge width determined is smaller than when the edge width is calculated without taking into account the movement of the image.

Another feature of this embodiment is that the output video signal 20 itself becomes a high-resolution, high-quality image due to filtering.

In cases where the entire screen is not moving but the subject is moving, it is sufficient to divide the areas and apply processing based on the same motion vector to the parts that have a common movement.

Although we have described here a method for image motion detection that calculates from video signals, if the imaging device is used in a manner where it is subjected to a lot of vibration and is equipped with an anti-vibration device, For example, motion vector signals may be obtained from the angle sensor, acceleration sensor, etc. of the anti-shake device.In this case, it is not easy to respond to the movement of the subject, but the movement of the entire image due to vibration is dominant. When the desired motion vector is obtained, good focus detection can be performed.

(Effects of the Invention) As described above, according to the focus detection device of the present invention, focus detection is performed by detecting a signal component that changes depending on the focus state from an image signal. A motion correction means for detecting and correcting the motion of the image is provided, and focus detection is performed based on the signal component whose motion has been corrected by the motion detection means. It is possible to prevent the deterioration of the accuracy of the focus detection means due to a decrease in the signal depending on the focus state, such as a decrease in the focus state, and the focus detection device is always highly accurate and stable without being affected by camera shake or subject movement. can be provided.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing the first embodiment of the motion correction device according to the present invention, FIG. 2 is a diagram showing the movement of an image, FIG. 3 is a diagram showing the movement of the optical system, and FIG. 4 is a diagram showing the movement of the image. Figure 5 is a diagram showing edge width correction; Figure 6 is a block diagram showing the second embodiment of the present invention; Figure 7 is a diagram showing filter characteristics of filtering. 8 is a diagram showing an impulse response of an inverse filter, FIG. 9 is a characteristic diagram showing frequency characteristics of a Wiener filter, and FIG. 10 is a schematic diagram showing a conventional focus detection method using image processing. 3I Figure 5 Z position

Claims (1)

[Scope of Claims] A focus detection device that detects a focus state based on an imaging signal output from an imaging device, comprising: a motion detection device that detects movement of an image from the imaging signal; a motion correction unit for correcting the movement of the image based on the output; and a movement correction unit that extracts a signal component that changes depending on the focus state from the image pickup signal whose movement component of the image has been corrected by the movement correction unit and adjusts the focus state. 1. A focus detection device comprising: focus detection means for performing detection.