JPH0955892A - Solid-state imaging device - Google Patents

Abstract

(57) [Abstract] [Purpose] To provide a solid-state imaging device capable of obtaining high resolution with a single plate. SOLUTION: Image pickup data obtained without relatively displacing an image pickup device CCD3 with respect to a subject image formed by an optical system 1 and a displacement mechanism 9 such as a piezo element relatively move in horizontal and vertical directions. Image data obtained by displacement is combined to obtain one image data. The number of pieces of image data to be picked up differs depending on the shooting mode.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a solid-state image pickup device using a solid-state image pickup device such as a CCD, and more particularly to high resolution thereof.

[0002]

2. Description of the Related Art Conventionally, the following techniques have been used to increase the resolution of a solid-state image pickup device.

A. Multiple pixels of the image sensor itself b. Multi-plate (using multiple image sensors) c. The pixel shift a largely depends on the semiconductor technology and does not generally progress. b can be relatively easily realized by using an optical prism or the like, but there is a problem that the cost increases. Therefore, a method of adopting c is being adopted particularly in the single plate type.

[0004]

However, when the pixel shift in the single plate type is used, since the color filter is attached, a specific description of the shift is not found, and it is difficult to implement.

Further, since data handling for pixel shifting is not described, there is a problem that processing takes a long time.

The present invention has been made under such circumstances, and a first object thereof is to provide a solid-state image pickup device capable of obtaining a high-resolution image with a single plate, and according to the read / write speed of a recording medium. A second object is to provide a solid-state imaging device capable of high-speed imaging.

[0007]

According to the present invention, in order to achieve the first object, the solid-state imaging device has the following (1) to (1).
7) and the following (13) to (17) to achieve the second object.

(1) The image pickup data is obtained without relatively displacing the position of the image pickup element with respect to the subject image formed by the optical system, and the image pickup data is relatively displaced in the horizontal and vertical directions by a predetermined amount. A solid-state imaging device that obtains image data of one or more sheets and synthesizes them to create one image data,
A solid-state imaging device having a plurality of imaging modes in which the number of image data to be captured is different.

(2) The image pickup data is obtained without displacing the position of the image pickup device to which the color filter array is attached with respect to the subject image formed by the optical system, and the image pickup data is relatively moved in the horizontal and vertical directions by a predetermined amount. It is a solid-state imaging device that displaces image data to obtain one or more pieces of imaging data, synthesizes them to create one piece of image data, and has a plurality of imaging modes in which the number of pieces of image data to be captured is different. A characteristic solid-state imaging device.

(3) The solid-state image pickup device according to (2), wherein the image pickup mode has a mode in which images are picked up by three or more times at one or more half pixel positions with respect to the sampling pitch of the image pickup device, using different color filters. .

(4) The solid-state image pickup device according to (2), wherein the image pickup mode has a mode in which an image is picked up only once at one or more half pixel positions with respect to the sampling pitch of the image pickup element.

(5) In the image pickup mode, the image pickup is performed only once at one or more half pixel positions with respect to the sampling pitch of the image pickup element, a luminance signal is generated from the image data, and the image pickup is performed at the one pixel position. The solid-state imaging device according to (2) above, which has a mode of generating a color signal from image data.

(6) The solid-state image pickup according to the above (2), wherein the image pickup mode has a mode in which images are picked up by three or more times at one or more one-pixel positions with respect to the sampling pitch of the image pickup device, respectively with different color filters. apparatus.

(7) The solid-state image pickup device according to (2), wherein the image pickup mode has a mode in which image data is processed from only one piece of image pickup data.

(8) The solid-state image pickup apparatus according to (2), wherein the image pickup mode is composed of two pieces of image pickup data at a normal position and image pickup data shifted by 1.5 pixels in the horizontal direction and 0.5 pixels in the vertical direction. Imaging device.

(9) The solid-state image pickup device according to (2), wherein the color filter array is formed by repeating three colors in stripes in the vertical direction.

(10) The solid-state image pickup device according to (9), wherein the color filter array is a repetition of three colors of cyan, green and yellow.

(11) The solid-state image pickup device according to (2), which has a mode in which the image pickup mode is composed of two pieces of normal image pickup data and image pickup data at a position shifted by 1.5 pixel pitch in the horizontal direction.

(12) Optical LP that can be freely taken in and out of the optical system
The solid-state imaging device according to (2), further including F.

(13) Image pickup data is obtained without relatively displacing the position of the image pickup element with respect to the subject image formed by the optical system, and one piece is obtained by relatively displacing a predetermined amount in the horizontal and vertical directions. A solid-state imaging device that obtains the above-described imaging data and synthesizes them to create one image data,
A solid-state image pickup device characterized in that the imaged image data is compressed with different compression rates depending on the read / write speed of the recording medium.

(14) The read / write speed of the recording medium is measured by a self-check of a system controller at an appropriate time after the recording medium is installed in the device and after the photographing is performed, the solid-state image pickup device according to the above (13). .

(15) The solid-state image pickup device according to (13), wherein the compression rate is increased in the case of a recording medium having a low speed.

(16) The gain of each color is adjusted before compression so that the value of each color at the white point becomes equal.
The solid-state imaging device described.

(17) Before compression, the bit depth of the image data is set to 8-bit data with gamma applied.
3) The solid-state image pickup device described above.

[0025]

BEST MODE FOR CARRYING OUT THE INVENTION The present invention will be described in detail below with reference to an embodiment of a digital still camera.

[0026]

【Example】

(Embodiment 1) FIG. 1 is a block diagram showing a system configuration of a "digital still camera" which is Embodiment 1. The configuration and operation will be described below with reference to FIG.

First, a recording medium 8-for recording image pickup data
When 1 or 8-2 is connected to the I / F (interface) 7 of the digital still camera, the system controller 15 measures the time required for reading and writing the recording medium 8-1 or 8-2. Thereby, the compression rate of image data described later is adjusted. This operation will be described later in detail.

Next, before starting the main exposure, the imaging mode is set by the mode changeover switch 16. That is, the imaging modes are mode 1 high resolution 3 plate mode mode 2 high resolution single plate mode mode 3 normal resolution 3 plate mode mode 4 normal resolution single plate mode 1 mode 5 normal resolution single plate mode 2 and above. Details of each imaging mode will be described later.

When a shutter trigger (not shown) is turned on, main exposure starts.

A subject image (not shown) is passed through the optical system 1 and then the optical L
C through the PF set 1 and 2 (2-1 and 2-2 in the figure)
An image is formed on the CD3 surface. Each of the optical LPF sets 1 and 2 is composed of one or a plurality of crystal plates, and is put in or taken out from the optical path depending on the mode setting.
The optical image on the surface of the CCD 3 is output as an electric signal by driving the CCD 3, and the CDS (correlation double sampling circuit) 4, A / D circuit (analog-digital conversion circuit) 5
Then, the reset noise is reduced, digitized, and temporarily stored in the memory 10 via the memory controller 6.

In this embodiment, the CCD 3 is 1536 × 10.
The number of pixels is 24 (about 1.5 million pixels), and complementary color stripe color filter arrays are attached as shown in FIG. The A / D converter 5 has a 10-bit characteristic having a linear characteristic.

When a plurality of images are picked up in the image pick-up mode set as described above and the total number of picked-up images exceeds the capacity of the memory 10, the recording media 8-1, 8-2.
Data is transferred to. The following is a description of such a case.

Image data (single plate, CCD raw data) having a depth of 10 bits in the memory 10 is processed by a DSP (digital signa).
input to the data compression unit (compliant with JPEG) 12 via the (processor) 11. In the DSP 11, 10-bit depth data is converted into 8-bit data to which gamma (γ = 0.45) has been applied. In the data compression unit 12, the recording medium (8-
1, 8-2) is preset, and the input data is compressed accordingly and written on the recording medium 8-1 or 8-2.

Next, the second image of the plurality of images is taken. In that case, the optical L
After inserting / removing the PF sets 1 and 2, the CCD displacement mechanism (using a piezo element or the like) 9 is driven to drive the CCD 3
The image is taken after displacing the physical position of the horizontal or vertical direction. The data flow is the same as that of the first sheet. After repeating the above operation a predetermined number of times, the recording medium 8
In -1 or 8-2, compressed data with pixel shifting is recorded.

When the final image pickup is completed, the process of synthesizing the data is started. That is, the necessary data in the recording medium 8-1 or 8-2 is read out, decompressed via the data compression unit 12, converted into 10-bit linear data via the DSP 11, and then the order or the like is reconfigured according to the pixel shift position. , Color conversion, aperture, base clip, and other signal processing, and if necessary, the data is compressed again by the data compression unit 12 and then written on the recording medium 8-1 or 8-2. The compression rate at this time may be linked with the compression rate at the time of the above-mentioned processing or may be independent.

Further, since the data for processing becomes enormous when read all at once, the data is read in blocks, that is, by partially reading the data of the corresponding area from each image data.

Next, the details of each imaging mode will be described.

FIG. 2 is an explanatory diagram of the mode 1. That is,
C (cyan), G (green), Y as shown in FIG.
Vertical stripe type color filter array by (Yellow) is CC with 1536 x 1024 pixels (about 1.5 million pixels)
It is attached to D3 as an on-chip filter. FIG. 2 shows how to shift the image when the pixel shift photographing is performed using the CCD 3. Each color filter (C, G, Y)
Are circled for clarity.

First, the first image is picked up (1). Next, the second image is captured with a horizontal shift of one pixel pitch (same as the sampling pitch of the image sensor). Here, for example, 1
The points sampled by C on the second sheet are sampled by G on the second sheet (2). Similarly, the third sheet is imaged by further shifting by one pixel pitch in the horizontal direction. Here, the points previously sampled by G are sampled by Y (3). Next, the fourth image is captured at a position horizontally shifted by a half pixel pitch with respect to (1).
That is, sampling is performed at a position shifted by a half pitch from the sampling point in (1) (4). Then, the fifth image is captured at a position shifted by one pixel pitch from the position (4). That is, C on the 4th sheet
The points sampled at are sampled at G on the fifth sheet (5). Further, the sixth sheet is imaged at a position shifted by one pixel pitch with respect to the fifth sheet (6). The data of three colors of C, G, and Y at the sampling position of 1.5 million pixels and the interpolation position in the horizontal direction can be imaged by the data of these six sheets.

Next, the seventh and subsequent images are picked up in the state of being shifted by a half pixel pitch in the vertical direction. The vertical direction conforms to the shifting methods (1) to (6).

By the data of (7) to (12), it is possible to capture the data of three colors of C, G, and Y in the horizontal direction at a position shifted by a half pixel pitch in the vertical direction.

By using a total of 12 data items (1) to (6) and (7) to (12), a sampling position of 1.5 million pixels and a position shifted by a half pixel pitch in the horizontal and vertical directions, that is, This means that the imaging data for three colors has been obtained at the sampling position at 6 million pixels.
That is, it is a 6-megapixel (high resolution) three-plate mode.

FIG. 4 shows the flow of processing of those data.

Twelve 1536 × 1024 CCD raw data are reconstructed to obtain 3072 × 2048 pixel 3 (C,
It becomes the data of G, Y) (a). After that, it becomes R, G, B data through a matrix operation. The matrix operation is
It looks like this:

[0045]

[Equation 1]

After that, the base clip (B
C), aperture (APC), gamma (γ), etc., to generate R, G, B 8-bit data (b).

In the main imaging mode, neither of the optical LPF sets 1 and 2 is placed in the optical path.

Next, the mode 2 will be described. FIG. 5 is an explanatory diagram thereof.

First, the optical LPF set 1 is put in the optical path. Then, the first image is captured (1). After that, shift by 1.5 pixel pitch horizontally and half pixel pitch vertically
The second image is captured (2).

Considering (1) and (2) as the object image formed, the sampling is as shown in FIG. That is, the sampling is the same as that of the 6-megapixel single plate sensor. However, the sampling points are offset every horizontal line.

Next, the signal processing in mode 2 will be described. 7 and 8 are diagrams for explaining the luminance signal generation. That is, a luminance signal is generated by alternately switching between two adjacent horizontal lines in the horizontal direction. The odd line and the even line have different combinations of two lines (SW).

Then, the luminance signal is obtained by the BC, APC processing and γ circuit. The color signals are read out independently of the C, G, and Y signals before switching (SW) matrix 1
Input to, perform complementary color pure color conversion (similar to mode 1), and L
It is input to the matrix 2 through the PF and γ circuits.

Here, from R, G, B to R-Y, G-Y,
Conversion to BY is performed. Then, the luminance signal Y described above is added to the generated color difference signal to become R, G, B signals.

Next, the optical LPF set 1 will be described. FIG. 9 shows color and luminance carriers in the sampling shown in FIG. 6 on the frequency plane. On the other hand, in the optical LPF 1, as shown in FIG. 10, it is necessary to reduce chrominance carriers in the oblique directions by the traps 1 and 2, and reduce luminance carriers in the trap 3. Such an LPF 1 can be composed of three crystal plates.

Next, the mode 3 will be described. FIG. 11 is an explanatory diagram thereof.

Neither of the optical LPF sets 1 and 2 is set in a state where it can be put in the optical path. After that, the first image is captured (1). On the other hand, the second image is imaged with a horizontal shift of one pixel pitch (2). Further, the third sheet is imaged by shifting in the horizontal direction by one pixel pitch (3).

That is, each sampling point of 1,500,000 pixels has data for three colors (C, G, Y), and three-plate data for 1,500,000 pixels.

The three pieces of data are rearranged into C,
It becomes data of three sheets of G and Y (FIG. 12). Hereinafter, as for the processing, almost the same processing as in FIG. 4B is applied according to the mode 1.

Next, mode 4 will be described. FIG. 13 is an explanatory diagram thereof.

First, the optical LPF set 2 is put in the optical path. Then, the first image is captured (1). After that, the second image is imaged with a horizontal shift of 1.5 pixel pitch (2).

FIG. 14 shows the combined state (sampling position in mode 4). That is, it can be seen that double sampling is performed in the horizontal direction, and the color carrier points shift to the high range.

Next, the signal processing in mode 4 will be described. FIG. 15 is a diagram showing this. The two pieces of image pickup data are combined into a horizontally long data of 3072 × 1024 (a). Thereafter, signal processing is performed to generate R, G, B signals, or the signals are input to a scaling circuit for adjusting the aspect ratio (b). This circuit may be a simple LPF or may be a circuit that adds two adjacent horizontal pixels.

FIG. 16 shows a sampling carrier and an optical LP.
It is a figure which shows the trap by F set 2. It can be said that the mode is a mode in which the color moire is considerably reduced because it is trapped at a frequency much lower than the actual band (that is, the color Nyquist frequency).

Finally, mode 5 will be described. This is the normal imaging mode.

After the optical LPF set 2 is put in the optical path, one piece of data is imaged. The sampling position is shown in FIG.

It can be considered that the processing in this case is similar to the processing of FIG. FIG. 18 shows the sampling carrier and the trap in this case. The crystal LPF that realizes this trap can be realized with a single crystal that has a characteristic of shifting in the horizontal direction.

As described above, optimum processing is performed according to each image pickup mode. The following is a summary of each imaging mode, the type of optical LPF, and the number of captured images.

[0068]

[Table 1]

Next, the relationship between the recording medium and the compression rate will be described.

If the writing / reading speed of the recording medium is slow (flash memory etc.), the time required for writing / reading the image data becomes long. That means that the total shooting time for one image becomes long. Therefore, in the case of a slow recording medium, the slowness is discriminated by the first self-check, and the compression ratio is changed accordingly (that is, when the recording medium is slower, the compression is stronger), so that the total shooting time is changed even if the recording medium changes. Do not change. For example, for mode 1, HDD (hard disk
drive) 8-1 and the flash memory 8-2 perform the same shooting, assuming that HDD: read / write speed 3 MB / s flash M: read / write speed 1.5 MB / s, 12 data are required in mode 1. Assuming that the data compression rate in the HDD 8-1 is 1/2 and the data amount of one sheet is about 1.5 MB, the time required to write all the data is 1.5 MB × 12 sheets × 1/2/3 MB / S = 3s.

The same compression ratio is applied to the flash memory 8-
If it is performed in step 2, it becomes 1.5 MB × 12 sheets × 1/2 ÷ 1.5 MB / S = 6 s, which is about twice as long. Therefore, in this embodiment, in that case, it is known in advance that the speed is slow by self-check, and accordingly the compression rate is increased to ¼.

1.5 MB × 12 sheets × 1/4 ÷ 1.5 MB / S = 3 s, which is the same performance as that of the HDD 8-1.

Of course, there may be some image deterioration,
Reducing the time is effective. It is also possible to stop the increase in the compression rate in a recording medium that is too slow. However, in that case, there is a problem that the photographing time becomes long.

Next, the white balance in the data compression processing will be described. 10-bit CCD in the embodiment
By converting the raw data into a table in the DSP 11, γ =
An 8-bit signal of 0.45 is used, but white balance may be taken before that.

That is, the gain is adjusted so that the C, G, and Y values at the white points are equal based on the C, G, and Y gains at the white points on the screen. For example, when the values of white points in the screen are C = 200, G = 250, Y = 100, C '= 1.25 * C, Y' = 2.5 * Y, (G '= G ) Is used to convert all image data pixel values.

As a result, especially in an achromatic subject, folding back is reduced, and deterioration is less noticeable even when compression such as JPEG is performed. This may be performed at the time of data compression in the DSP 11.

(Second Embodiment) Next, a second embodiment will be described.

In the first embodiment, the optical LP is used in the mode 2.
Although the F set 1 was specially prepared to achieve high resolution by capturing two images, the cost for that may be increased.

Therefore, it is also possible to take two images for the luminance (without the optical LPF) and one image for the color signal (using the optical LPF set 2), that is, a total of three images. The two images for brightness are imaged in the same manner as in mode 2 of the first embodiment. However, the optical LPF is not set. Then, only the luminance signal is generated from the data. Next, an image is picked up in the same manner as in mode 5, and only the color signal of them is used and added with the luminance signal. However, in this case, the color signal requires interpolation enlargement processing. According to this, the number of images to be picked up for one image increases, but the cost can be reduced.

(Modification) In the first embodiment, the optical LP
It is also possible to omit the F set 2 itself. However, in that case, since the color carrier and the like cannot be suppressed, moire remains. However, it is possible to reduce the cost and the number of captured images at the same time.

The setting of the optical LPF is an example, and for the purpose of trapping the color carrier and the luminance carrier,
It goes without saying that it may have any configuration. Further, the optical plate is not limited to the crystal plate and may be another optical member.

When taking two images for brightness and one image for color,
It is also effective to take two brightness images first and one color image later. This is because a slight vibration occurs due to the attachment / detachment of the optical LPF. Therefore, by taking a picture for luminance which contributes to the resolution first, the vibration can be avoided even a little.

Further, at the edges of the peripheral portion, the resolution is not necessarily made high in all the areas by shifting, but if that portion is created by interpolation from the first sheet, data will not be lost and unnaturalness will occur. Can be eliminated.

Although the CCD itself is displaced by the piezo element or the like in the first embodiment, it may be displaced by using an optical member such as a wedge-shaped glass plate.

[0085]

As described above, according to the present invention,
It is possible to obtain a high-resolution image in which pixels are effectively shifted with a small number of captured images, and it is possible to perform high-speed imaging according to the read / write speed of the recording medium.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a first embodiment.

FIG. 2 is an explanatory diagram of mode 1

FIG. 3 is a diagram showing a pattern of a color filter array used in Example 1.

FIG. 4 is an explanatory diagram of signal processing in mode 1.

FIG. 5 is an explanatory diagram of mode 2

FIG. 6 is a diagram showing sampling positions in mode 2

FIG. 7 is an explanatory diagram of luminance signal generation in mode 2

FIG. 8 is an explanatory diagram of signal processing in mode 2.

FIG. 9 is a diagram showing color and luminance carriers by sampling shown in FIG. 6 plotted on a frequency plane.

FIG. 10 is a diagram showing setting of an optical filter.

FIG. 11 is an explanatory diagram of mode 3

FIG. 12 is a diagram showing a data structure of mode 3

FIG. 13 is an explanatory diagram of mode 4

FIG. 14 is a diagram showing sampling positions in mode 4

FIG. 15 is an explanatory diagram of signal processing in mode 4.

FIG. 16 is a diagram showing color and luminance carrier positions and trap positions in mode 4;

FIG. 17 is an explanatory diagram of mode 5

FIG. 18 is a diagram showing color and luminance carrier positions and trap positions in mode 5;

[Explanation of symbols]

 1 Optical system 2-1, 2-2 Optical LPF 3 CCD 15 System controller

Claims (17)

[Claims] 1. A single image obtained by obtaining image pickup data without displacing the position of an image pickup element relative to a subject image formed by an optical system, and displacing the image pickup data in a horizontal or vertical direction by a predetermined amount. Obtain the above imaging data, combine them, and
A solid-state imaging device for creating one piece of image data, comprising a plurality of imaging modes in which the number of pieces of image data to be picked up is different. 2. The image pickup data is obtained without relatively displacing the position of the image pickup device to which the color filter array is attached with respect to the subject image formed by the optical system, and the image pickup data is relatively moved in the horizontal and vertical directions by a predetermined amount. It is a solid-state imaging device that displaces at least one to obtain one or more pieces of imaging data, and synthesizes them to create one piece of image data, and has a plurality of imaging modes in which the number of pieces of image data to be captured is different. Solid-state imaging device. 3. The image pickup mode according to claim 2, wherein the image pickup mode has a mode in which images are picked up at three or more times at one or more half-pixel positions with respect to the sampling pitch of the image pickup element by different color filters. Solid-state imaging device. 4. The imaging mode is set to 1 at one or more half pixel positions with respect to the sampling pitch of the imaging device.
The solid-state imaging device according to claim 2, wherein the solid-state imaging device has a mode in which imaging is performed only once. 5. The image pickup mode is set to 1 at one or more half pixel positions with respect to the sampling pitch of the image pickup device.
Imaged only once to generate a luminance signal from the image data,
Further, it has a mode for generating a color signal from image data picked up at one pixel position.
The solid-state imaging device described. 6. The imaging mode is set to 3 at one or more one-pixel positions with respect to the sampling pitch of the imaging device.
The solid-state image pickup device according to claim 2, wherein the solid-state image pickup device has a mode in which images are picked up by different color filters more than once. 7. The solid-state imaging device according to claim 2, wherein the imaging mode has a mode in which image data is processed from only one piece of imaging data. 8. The image pickup mode comprises two pieces of image pickup data at a normal position and image pickup data shifted by 1.5 pixels in a horizontal direction and 0.5 pixels in a vertical direction. The solid-state imaging device described. 9. The solid-state image pickup device according to claim 2, wherein the color filter array is formed by repeating three colors in a stripe shape in the vertical direction. 10. The solid-state image pickup device according to claim 9, wherein the color filter array is a repetition of three colors of cyan, green and yellow. 11. The solid-state image pickup according to claim 2, wherein the image pickup mode has a mode including two pieces of normal image pickup data and image pickup data at a position shifted by 1.5 pixel pitch in the horizontal direction. apparatus. 12. The solid-state imaging device according to claim 2, further comprising an optical LPF that can be inserted into and removed from the optical system. 13. Image pickup data is obtained without displacing a position of an image pickup element relative to a subject image formed by an optical system, and one or more pieces are obtained by relatively displacing a predetermined amount in horizontal and vertical directions. Image data of the
A solid-state image pickup device for producing image data of one sheet, wherein the picked-up image data is compressed with different compression rates according to a read / write speed of a recording medium. 14. The read / write speed of the recording medium is measured by a self-check of a system controller at an appropriate time after the recording medium is installed in the apparatus and after the photographing is performed. Solid-state imaging device. 15. The solid-state imaging device according to claim 13, wherein the compression rate is increased in the case of a recording medium having a low speed. 16. The solid-state imaging device according to claim 13, wherein the gain of each color is adjusted before compression so that the value of each color at the white point becomes equal. 17. The solid-state imaging device according to claim 13, wherein the bit depth of the image data is gamma-added 8-bit data before compression.