JPH10136253A - Imaging device - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To improve the image quality of a high resolution image by pixel shift. A pixel shift mechanism shifts an image sensor in a horizontal direction, and an output image of the image sensor at each shift position is supplied to a DSP via an A / D converter and a memory control circuit. You. The focus detection circuit 32 is DS
The degree of focus is detected from the output of the P memory, and the exposure amount detection circuit 3
Reference numeral 4 denotes an exposure amount, and a white balance detection circuit 36
Detects a gain for white balance. The system control circuit 38 controls the photographing optical system 10 in accordance with the outputs of the circuits 32 and 34 so that the in-focus state and the appropriate exposure amount are obtained. The captured image at that time is adjusted in white balance by the DSP 30 and stored in the memory 28. Individual captured images resulting from the pixel shift are combined on the memory 28 and read out to the D / A converter 52 and / or the hard disk 54 as a high-resolution image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an imaging device,
More specifically, the present invention relates to an imaging apparatus that obtains a high-resolution image by synthesizing a plurality of captured images in which an image sensor is shifted in a lateral direction relative to an optical image of a subject.

[0002]

2. Description of the Related Art In an image pickup apparatus, a distance measurement, an exposure amount, a white balance, and the like are generally adjusted for each scene to be photographed. An ordinary imaging device has a fixed imaging element, and can only obtain an image limited by the resolution of the imaging element. On the other hand, a technique has been proposed in which a plurality of captured images in which an image sensor is relatively horizontally shifted (pixel-shifted) with respect to an optical image of a subject are synthesized to obtain an image having a resolution higher than the resolution of the image sensor. ing.

In an image pickup apparatus, distance measurement, exposure amount, white balance, etc. are generally adjusted for each scene to be photographed. However, even in an image pickup apparatus using pixel shift, the distance measurement, exposure amount, white balance, etc., are not adjusted. , Were adjusted individually when each individual image was taken before the composition.

[0004]

However, during photographing of a plurality of images necessary to obtain one composite image, external factors,
For example, when the illumination light changes, the exposure amount differs among a plurality of images to be synthesized, and the synthesized image becomes unnatural. Of course, all individual images need to be correctly focused, but this is not guaranteed in the conventional example. For example, if an out-of-focus image is included in a plurality of individual images due to vibration of an imaging device or the like, a high-resolution composite image cannot be obtained.

SUMMARY OF THE INVENTION It is an object of the present invention to solve these problems and to provide an image pickup apparatus which is easy to synthesize and can obtain a high quality synthesized image.

[0006]

According to the present invention, there is provided an image pickup apparatus comprising: a pixel shift unit for shifting a relative position between an image sensor and image pickup light in a plane perpendicular to an optical axis; And a synthesizing unit for synthesizing one image from a plurality of individual images obtained by the shift of the individual images. It is characterized in that one of them is executed.

As a result, the focus state and the like of the individual image serving as the basis of the composite image are appropriate, and the image quality of the high-definition composite image is improved. That is, even if the focus state or the light amount changes during shooting of a plurality of images that are the basis of the composite image, an individual image with appropriate shooting conditions can be obtained. As a result, the composite image also has appropriate shooting conditions. Become something.

[0008]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a schematic block diagram showing an embodiment of the present invention. Reference numeral 10 denotes a photographing optical system, which includes a focusing lens group 12, an aperture 14, another lens group 16, and an optical low-pass filter (LPF) 18. 20
Is a symbol C for converting an optical image by the photographing optical system 10 into an electric signal.
The CD-type image sensor 22 is a pixel shifting mechanism that shifts the image sensor 20 in the horizontal direction (the direction perpendicular to the optical axis).

Reference numeral 24 denotes an A / D converter for converting an analog output of the image pickup device 20 into a digital signal; 26, a memory control circuit; 28, a memory; 30, a digital signal processing circuit (DSP); A focus detection circuit for detecting a degree of focus, an exposure amount detection circuit for detecting an exposure amount from an output of the DSP 30, a white balance detection circuit for detecting a white balance from an output of the DSP 30, a system control circuit, , A focus control device; 42, a focus drive device for driving the focus lens group 12; 44, a lens position detection device for detecting the position of the lens group 12 whose focus is intoxicated; 46, an aperture control device; A driving device, 50 is an image sensor driving circuit for driving the image sensor 20, and 52 is a drive from the memory control circuit 26. D / A converter for converting the barrel video data into an analog signal, 54 a memory control circuit 26
Is a hard disk for storing digital video data from a computer.

The photographing optical system 10 forms an optical image of a subject on an imaging surface of an imaging device 20. The optical filter 18 of the photographing optical system 10 has a spatial frequency characteristic such that the response becomes zero at the Nyquist frequency (1/2 of the Nyquist frequency of the image sensor 20) of the synthesized image obtained by pixel shifting described later. , And serves to suppress aliasing components due to spatial sampling of the image sensor 20.

The image sensor 20 is movable in a plane perpendicular to the optical axis by a pixel shifting mechanism 22. On the imaging surface of the imaging device 20, complementary colors Cy (cyan) as shown in FIG.
A color filter array composed of Ye (yellow), Mg (magenta), and G (green) is formed. It is assumed that the color filter of the image sensor 20 is formed of square pixels having a pixel pitch p. Pixel shift mechanism 22
The image sensor 20 can be moved in the horizontal direction by the piezoelectric element. As shown in FIG. 2, after the image 1 is taken, the image 2 is shifted by p / 2 in the horizontal direction, and the image 2 is taken in the vertical direction. The image 3 is taken with a shift of p / 2, and furthermore,
The image 4 is photographed with a shift of p / 2 (in a direction opposite to the direction shifted to photograph the image 2). These images are stored in the memory 28 by the memory control circuit 26 in the arrangement shown in FIG.

From these four images, it is possible to obtain image data corresponding to raw image data (the pixel aperture is equal to the image pickup device 20 used) photographed at a sampling interval of p / 2. Are synthesized as shown in FIG. 3, a synthesized image having a higher resolution than the original resolution of the image sensor 20 can be obtained. Specifically, a composite image can be obtained by sequentially reading out each pixel data stored in the memory 28.

When each of the images 1 to 4 is photographed, the focus, exposure and white balance are adjusted. Specifically, the output signal of the image sensor 20 is converted into a digital signal by the A / D converter 24, and the digital signal is transmitted through the memory control circuit 26.
Transferred to P30. The DSP 30 generates luminance data in the distance measurement area and sends it to the focus detection circuit 32, sends the luminance data in the photometry area to the exposure amount detection circuit 34, generates a color signal, and sends it to the white balance detection circuit 36. send.

The focus detection circuit 32, as shown in FIG.
It comprises a band-pass filter (BPF) 60 and an integrator 62. The BPF 60 transmits a middle-high frequency component of the luminance signal, and the integrator 62 integrates the output of the BPF 60.
Since the middle and high frequency components of the luminance signal correspond to the contrast of the captured image, the integral value of the middle and high frequency components of the luminance signal can be used as the focus evaluation amount. It shows the characteristic of a convex shape as a peak. That is, the integral value of the middle and high frequency components of the luminance signal becomes maximum at the focal point and becomes smaller as the focal point is blurred. The system control circuit 38 calculates the focus evaluation amount from the focus detection circuit 32,
The focus drive circuit 42 is controlled via the focus control device 40 so as to move the focus lens group 12 in a direction in which the focus evaluation amount increases. Lens position detection circuit 4
4 detects the position of the focus / lens group 12 and supplies it to the focus control circuit 40. Thereby, the focus control circuit 4
0 indicates whether the focus lens group 12 has reached the movable ends. Focus detection circuit 3
When the output of No. 2 is maximized, it is the focal point, and the focusing adjustment is completed.

The exposure amount detection circuit 24 detects the current exposure amount from the luminance signal of the photometry area set in advance, and supplies the current exposure amount to the system control circuit 38. The system control circuit 38
The exposure time (electronic shutter speed) of the image pickup 20 is controlled by the CCD drive circuit 50 so that an appropriate exposure amount is obtained, and the aperture of the aperture 14 is controlled by the aperture control circuit 46 and the aperture drive circuit 48.

The WB detection circuit 36 detects a white point in the image based on the color signal from the DSP 30, and
The color signal gain is supplied to the system control circuit 38 so that the balance can be obtained. The system control circuit 38
The gain information from the detection circuit 36 is supplied to the DSP 30 via the memory control 26. The DSP 30 adjusts the white balance of the captured image based on the gain information.

In this way, the output image (image 1) of the image sensor 20 in the state where the focus and the exposure are adjusted is stored in a predetermined location of the memory 28, and the gain for white balance at that time is stored. Information is also stored in the memory 28.

Next, the pixel shift mechanism 22 is connected to the image pickup device 20.
Is moved in the horizontal direction by p / 2, the focus and the exposure are similarly adjusted, the gain of the white balance is determined, and the image 2 is photographed. The gain of the color signal and the image 2 are temporarily stored in the memory 28. Thereafter, each time the pixel is shifted as shown in FIG. 2, the focus and the exposure are adjusted, the gain of the white balance is determined, and images 2, 3, and 4 are photographed. The gain is temporarily stored in the memory 28.

Using the images 1, 2, 3 and 4 thus obtained and the gain data of the color signals corresponding to the images 1, 2, 3 and 4, the DSP 30 A high-resolution image having twice the number of pixels in both the vertical and horizontal directions is synthesized. This composite image is converted into an analog signal by the D / A converter 52 and output to the outside as a video signal. The composite image is also stored on the hard disk 54 as digital data.

As described above, each time a pixel is shifted, the focus and the exposure are adjusted and photographed, so that a plurality of images used for image composition can be photographed under appropriate photographing conditions even if the photographing conditions such as brightness change. . Therefore, it is possible to prevent only one of the plurality of images from becoming dark. In addition, gain information for white balance is determined at the time of capturing an image to be synthesized, and is used at the time of synthesis, so that an appropriate white balance can be obtained with the synthesized image.

In this embodiment, each time the pixel is shifted, the focus,
Although all of the exposure amount and the white balance are determined, it is also possible to adjust only one of the items whose conditions are changed during the pixel shift shooting depending on the shooting conditions. Further, the measurement may be performed every time two images are taken, instead of performing the measurement every time the image is taken.

[0023]

As can be easily understood from the above description, according to the present invention, in order to obtain a high-resolution image (composite image) by shifting pixels, it is necessary to perform measurement while capturing each individual image on which the high-resolution image is based. Since at least one of distance, light control, and white balance is performed, it is possible to cope with a change in shooting conditions such as illumination light due to an external factor. Since an appropriate exposure, white balance, and the like can be obtained for each individual image, the synthesized image also becomes natural.

Further, since it is possible to cope with a case where the in-focus state changes due to vibration of an image pickup device or the like, a part of the synthesized image is not out of focus, and a high-resolution image which is in focus over the entire surface is obtained. A composite image can be obtained.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram of an embodiment of the present invention.

FIG. 2 is a diagram illustrating a state of pixel shift and a positional relationship of an image obtained by the shift.

FIG. 3 is a diagram illustrating a high-resolution raw data image obtained by pixel shifting.

4 is a schematic configuration block diagram of a focus detection circuit 32. FIG.

FIG. 5 is a diagram illustrating a relationship between a position of a focusing lens group and a focus evaluation amount during a focusing operation.

[Explanation of symbols]

 10: Photographing optical system 12: Focusing lens group 14: Aperture 16: Other lens group 18: Optical low-pass filter (LPF) 20: Image sensor 22: Pixel shifting mechanism 24: A / D converter 26: Memory control circuit 28: Memory 30: Digital signal processing circuit (DSP) 32: Focus detection circuit 34: Exposure amount detection circuit 36: White balance detection circuit 38: System control circuit 40: Focus control device 42: Focus drive device 44: Lens position detection Device 46: Aperture control device 48: Aperture drive device 50: Image sensor drive circuit 52: D / A converter 54: Hard disk 60: Band pass filter (BPF) 62: Integrator

Claims (1)

[Claims] 1. A pixel shifting means for shifting a relative position between an image sensor and imaging light in a plane perpendicular to an optical axis, and a plurality of individual images obtained by shifting in a plurality of directions by the pixel shifting means. Claims 1. An imaging apparatus comprising: synthesizing means for synthesizing two images, wherein at least one of distance measurement, dimming, and white balance is executed at the time of capturing the individual image for the second and subsequent times. .