JP3519002B2 - Imaging device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [0001] BACKGROUND OF THE INVENTION 1. Field of the Invention
Weighted addition of multiple images with multiple coefficients whose sum is 1
Imaging devices that perform dynamic range expansion processing
You. [0002] 2. Description of the Related Art Generally, an imaging device such as a television camera is used.
Means that the effective imaging brightness range is
It is uniquely determined by the photoelectric conversion characteristics of the image sensor. Toes
The lower limit of the output level of the solid-state image sensor is the noise level.
Is limited and the upper limit is limited to the saturation level
The working range is determined and the output level characteristics of the solid-state
Gender has a constant slope, resulting in a solid
The effective imaging luminance range of the imaging device is uniquely determined. Therefore, conventionally, for example, Japanese Patent Application Laid-Open No. 57-396
In Japanese Patent Publication No. 73, two kinds of imaging signals with different brightness are combined.
The dynamic range of the imaging signal of the solid-state imaging device.
An expanding imaging device is disclosed. [0004] However, in the future,
In the expansion of the dynamic range of the imaging device
Because only two types of imaging signals are added or subtracted,
On the luminance side, S / N deteriorates, and
The synthesized image signal is a linear combination and is a polygonal line.
Output from the solid-state image sensor at the connection point.
For example, the resulting color image
Does not produce a smooth color change, resulting in an uncomfortable image.
Problem. In Japanese Patent Application Laid-Open No. 6-141229,
Indicates the signal level of two or more images with different charge accumulation times
Weighted according to
Does not saturate dark to bright images
To obtain an image with less noise. [0006] This prior example is disclosed in, for example, FIG.
As shown in FIG.
Only near the boundary region where one of the images is saturated
The changed weights are combined by addition, and
Desired in / out by compressing by compression means according to several bands etc.
It is a composite image of force characteristics. However, in this prior example, for example, two images
When obtaining a composite image from an image, the first charge accumulation time is long.
On the side of the area lower than near the boundary area where the image is saturated,
This is a characteristic that a composite image is determined only by one image.
You. [0008] Compression processing is performed on the synthesized image at the subsequent stage.
Yes, but for one combined image that has already been added
The above characteristics cannot be changed basically.
No. Therefore, for example, in the case of an image having motion,
The outline on the degree side and the high luminance side are displaced or unnatural color change parts
The result is an image with some minutes. (Object of the Invention) The present invention has been made in consideration of the above points.
The purpose of the dynamic range
Enlarges and prevents S / N degradation on the low luminance side
In addition, you can obtain a smooth image without discomfort
It is to provide an imaging device. [0010] An imaging apparatus according to the present invention.
Is the subjectImaging means capable of imagingWhen,The imaging means is
Drive control of the first exposure time that saturates at the light amount level of 1
And at a second light level higher than the first light level
Drive control of the second exposure time when the imaging means is saturated.
Alternately taking pictures at the first exposure time from the imaging means.
An image is captured at the imaged first imaging signal and the second exposure time.
Drive means capable of alternately reading out the read second imaging signal
When,Before the imaging unit read by the driving unit
The first imaging signal has a monotonously decreasing curve characteristic.
Weight that can be weighted with a first weighting factor
Processing circuit,The sum with the first weighting coefficient is 1 and
A second weighting factor having a monotonically increasing curve characteristic
Second weighting coefficient generating means for generating;,The second weight
To the second weighting coefficient generated by the weighting coefficient generating means.
Therefore, the second imaging read out from the imaging unit
Signal while maintaining the sensitivity ratio with the first imaging signal.
A second weighting circuit for performing weighting processing;,The first
And the first image signal processed by the second weighting processing circuit
An addition circuit for adding the signal and the second image signal;,The addition
Based on the image signal output from the circuit, the display means
A video signal for generating a video signal capable of displaying an image of a subject
And processing means. [0011] BRIEF DESCRIPTION OF THE DRAWINGS FIG.
An embodiment will be described. (First Embodiment) FIGS. 1 to 7 show a first embodiment of the present invention.
FIG. 1 is a diagram illustrating an imaging apparatus according to a first embodiment.
FIG. 2 is an enlarged view of the D range for R in FIG.
FIG. 3A is a configuration diagram showing a configuration of a road, and FIG.
FIG. 3B is a diagram showing a memory map of the second LUT and FIG.
Weighting from first LUT and second LUT according to input signal
FIG. 4 (a) is an explanatory view showing a state in which the set coefficient is output.
4 (I) show the signals of the R range expansion circuit for R in FIG.
FIG. 5 is a timing chart showing the timing.
FIG. 6 is an explanatory diagram for explaining the operation of the R range expansion circuit for R, and FIG.
FIG. 7 is a configuration diagram showing a configuration of a modification of the imaging apparatus of FIG.
It is a block diagram of the D range expansion circuit for R in a form example. As shown in FIG. 1, an image pickup apparatus according to the present embodiment
The device 1 is a single-chip color imaging device C
CD2 and a synchronizing signal generating circuit (hereinafter referred to as a reference signal generating circuit)
3 and the reference signal from SSG3.
Timing generator for generating drive signals for CCD2
And a driving signal from the timing generator 4
And a CCD driver 5 for driving the CCD 2
You. Also, the image signal from the CCD 2 is amplified.
The preamplifier 6 and the sun from the timing generator 4
Correlated double sampling (hereinafter C
DSS) and the output of the CDS circuit 7
A / D converter 8 for A / D conversion of the CCD 2
After the output image signals are amplified by the preamplifier 6, C
The signal is dropped to the baseband band by the DS circuit 7 and A /
The signal is converted into a digital signal by the D converter 8.
ing. The A / D converter 8 is, for example, an 8-bit digital
To a video signal. Further, the image pickup apparatus 1 includes an A / D converter 8
A / D converted digital signal is converted into three color signals of RGB
Color separation circuit 9 for color separation and color separation by the color separation circuit 9
White balance adjustment for the digital signal
White balance circuit 10 and white balance circuit
Digital camera with white balance adjusted by road 10
Automatic gain control circuit (hereinafter, A
And a gain by the AGC circuit 11.
Knee processing and gamma compensation are performed on the adjusted digital signal.
Knee & γ circuit 12 that performs positive and knee & γ circuit 1
RGB data after knee processing and γ correction by 2
R for expanding the dynamic range of each signal
Range expansion circuit 15R for G, D range expansion circuit 15 for G
G and B D range expansion circuit 15B and R D range expansion circuit
Large circuit 15R, D range expansion circuit 15G for G and D for B
Dynamic range expansion by range expansion circuit 15B
Enhanced RGB digital signal
An enhancement circuit 16 for performing the processing, and an enhanced digital
D / A conversion of the power signal
And a D / A converter 18 for outputting to the
It is. The R range expansion circuit 15R for R is shown in FIG.
As shown in FIG.
Digital signal for one field of R signal that has been corrected
Field memory 21 for storing
21 and the current field of the R signal stored in the previous field.
Field R signal from the timing generator 4
The first select that is switched and output according to the field discrimination signal
A second selector 23 and a second selector 23.
Is connected to the first circuit through the inverter circuit 24.
One field before the R signal output by the collector 22 or
R signal of field different from R signal of current field
Is output. Further, the R range expansion circuit 15R for R
2 With respect to the R signal output from the selector 23 at a pixel rate
Outputs a predetermined function value for the output level of
First look-up table (hereinafter abbreviated as first LUT)
25 and the second LUT 26 and the pixels from the first selector 22
R signal output at the rate multiplied by the output of the first LUT 25
The first multiplier 27 and the second selector 23 calculate the pixel level.
Multiply the output of the second LUT 25 with the R signal output at the
A second multiplier 28, an output of the first multiplier 27 and a second
The output of the multiplier 28 is added to the output of the enhancer circuit 16.
And an adder 29 for outputting. The D range expansion circuits for G 15G and B
D range expansion circuit 15B for R is a D range expansion circuit 1 for R.
Since the circuit configuration is the same as that of 5R, the description of those configurations is as follows.
Omitted. The first LUT 25 and the second LUT 26 described above
Is a memory map as shown in FIG.
You. For example, the first LUT 25 and the second LUT 26
For example, 8-bit address 00-FF is weighted monotonically decreasing
Coefficient f (= cos^Two) And a monotonically increasing weighting coefficient g (=
sin^Two) And these weighting function data
The sum of f and g is set to 1. Here, the 8-bit addresses 00 to FF are
Corresponds to the luminance level of the input 8-bit digital signal
Is set. More specifically, imaging with CCD2
If the obtained image signal is 0, the digital signal is passed through the A / D converter 8.
Signal in this case, and in this case, the address of (00) h
The data is input to the first LUT 25 and the second LUT 26. As shown in FIG. 1, the output of the CCD 2 is actually
Since the signal has an AGC circuit 11 and the like in the middle, the imaging signal is
Amplified, gamma-corrected, and dynamics on the display surface of the monitor 19
It is adjusted to the level corresponding to the work range. That is, when the output signal of the CCD 2 is 0,
Indicates that the signal level input to the monitor 19 is also 0 and C
When CD2 is a saturated value, it is displayed on the monitor 19.
Image 1 also has a saturation value (Vuse in FIG. 5).YoSea urchin level
Adjusted. Further, an image pickup corresponding to the output signal of the CCD 2
In the case of a saturation value at which the signal is saturated (for example, Vuse in FIG. 5)
Through the A / D converter 8 etc. to the address of (FF) h
The data is input to the first LUT 25 and the second LUT 26. In addition,
For example, (00) h indicates 00 in hexadecimal notation. Input to the first LUT 25 and the second LUT 26
Is standardized to be a signal x in the range of 0 to 1.
(That is, the digital value (00) h is 0,
Value (FF) h is 1), the first LUT 25 is co
s^Two(Px), and the first LUT 25 outputs sin^Two(P
x) is output. Input to the first LUT 25 and the second LUT 26
The data corresponding to the address is read by the signal x to be read.
It is output and output. This state is shown in FIG. Here, p is a correction coefficient, as shown in FIG.
Signal when the image is captured with a short exposure time
If the brightness when saturates is ys2, the weight at that time
Finding coefficient g, ie sin^Two(Px) is 1,
For example, p ≒ (π · ys2 / 2) is set. Next, the present embodiment thus configured
The operation of the imaging device 1 will be described. From SSG3
Generated by the timing generator 4 based on the reference signal.
The CCD driver 5 drives the CCD 2 with the drive signal.
Then, the photoelectric conversion signal of the subject imaged by the CCD 2 is
After being amplified by the preamplifier 6, the CDS circuit 7
To the baseband, and digitally converted by the A / D converter 8.
Is converted into a signal. Here, the timing generator
The CCD driver 5 uses the drive signal generated by the
Shutter time different for each field, for example,
The subject is imaged with the shutter time of 1 set to 1/60 sec.
And at the same time, for example, the first shutter
1/4 of the data time (that is, 1/240 sec)
CCD2 to capture the image of the subject with the high-speed element shutter
Drive. That is, the subject image in the A field is
Imaged at the first shutter time of
The body image is captured during the second shutter time. In addition,
One field period (1/60 sec) for B field
The signal imaged in a period other than 1/240 sec in
Perform the process of sweeping away at high speed. As will be described later, the D range for R is expanded.
Large circuit 15R, D range expansion circuit 15G for G and D for B
The range enlargement circuit 15B uses these different shutter times.
Expand the dynamic range with the image signal of the subject
You. Thereafter, A / D conversion is performed by the color separation circuit 9.
Digital signal is separated into three color signals of RGB,
White balance circuit 10, AGC circuit 11, and kn
White balance adjustment and gain adjustment by e & γ circuit 12
After adjustment, knee processing and gamma correction, the D
15R, G range expansion circuit 15G for G and
The digital signal of RGB is output by the D range expansion circuit 15B for B.
The dynamic range expansion process of each issue is performed,
Enhancement processing by the enhancement circuit 16 and D / A conversion
D / A conversion by the device 18
Output to the monitor 19. Next, the R range expansion circuit 15R for R and the G range
D range expansion circuit 15G and B D range expansion circuit 15
B for details of the dynamic range expansion process
Referring to the timing chart of FIG.
The dilation circuit 15R will be described as an example. The timing signal from the timing generator 4
Field synchronized with vertical synchronization signal VD (FIG. 4A)
The discrimination signal (FIG. 4 (b)) is used as the R range expansion circuit 15R for R.
Through the first selector 22 and the inverter circuit 24
It is output to the 2 selector 23. The R output of the knee & γ circuit 12 is
Feed memory 21 of R range expansion circuit 15R for R (FIG.
4 (c)) and the first selector 22 and the second selector 23
Input to one input terminal and output from feed memory 21
(FIG. 4D) shows the first selector 22 and the second selector 2
3 is input to the other input terminal. Thus, based on the field discrimination signal,
And the first selector 22 outputs the first shutter time (1/1).
Digital of A-field image captured at 60 sec)
And the second selector 23 outputs the second signal
B-field imaged in 1/240 sec.
The digital signal of the image of the memory is output. The A-feed from the first selector 22
The digital signal of the field image is output to the first multiplier 27.
And the image of the B field from the second selector 23
Is converted to a second multiplier 28 and a first LUT 25 (see FIG.
4 (e)) and the second LUT 26 (FIG. 4 (f)).
It is. At this time, the first LUT 25 and the second LUT
Reference numeral 26 denotes a pixel level of a digital signal of a B-field image.
Cos for the signal x input at^Two(Px) and
sin^Two(Px) are output. In this case, the image in the A field is
Imaging with an exposure time (imaging time) four times as long as the
Therefore, as shown in FIG.
For example, an image pickup signal having characteristics such as x1 = y is obtained.
The image of the B field is imaged with characteristics such as x2 = y / 4
Signal. Then, the image pickup signal x of the image of the A field
1 is the saturation value Vuse at the brightness ys1, and the B field
The imaging signal x2 of the image of the brightness is ys2 (= 4ys1)
Becomes the saturation value Vuse. In this embodiment, the image of the A field is
The imaging signal x1 at a certain brightness ya in FIG.
In the case of x1a, the corresponding
The imaging signal is x2a. Then, as shown in FIG.
Thus, the imaging signal x2 corresponding to the image of the B field is
By being input to the first LUT 25 and the second LUT 26,
, And cos as weighting functions f and g^Two(Px) and
sin^Two(Px) is output. These cos^Two(Px) and sin^Two(P
x) goes to the first multiplier 27 and the second multiplier 28, respectively.
The first multiplier 27 outputs the A field at the pixel rate.
Image digital signal and cos^Two(Px) is multiplied
(FIG. 4 (g)), and the pixel rate in the second multiplier 28
And the digital signal of the B field image^Two(P
x) (FIG. 4 (h)). And the adder 29
The output of the first multiplier 27 and the output of the second multiplier 28 are added.
(FIG. 4 (i)). In FIG. 4, for simplicity, the An field
The output of each pixel in the Bn field is
Although the output of each pixel is represented by Bn,
In the process of expanding the dynamic range in the embodiment,
As described above, processing is performed at the pixel rate,
Is the pixel rate of the digital signal of the A-field image
Each output value at x, digital signal of B field image
Let u be the output value at the pixel rate of
Output M M = xcos^Two(Px) + usin^Two(Px) It will be obtained as follows. The D range expansion circuits for G 15G and B
D range expansion circuit 15B for R is a D range expansion circuit 1 for R.
Since it operates in the same manner as 5R, its description is omitted. Therefore, according to the present embodiment, FIG.
As described above, shooting is performed with the first shutter time (1/60 sec).
The digital signal x1 of the imaged A-field image and the
B imaged with a shutter time of 2 (1/240 sec)
For digital signal x2 (= u) of field image
Output from the R range expansion circuit 15R for R
The force M is cos^TwoWeighting by (px)
The first shutter time (1/60 sec)
Digital signal of the A-field image captured at
Conversely, sin^TwoWeighting by (px)
Mainly for the second shutter time (1/240 sec)
Digital signal of the B-field image captured at
Therefore, the S / N does not deteriorate in the low luminance range
The working range can be expanded. More specifically, in FIG.
Corresponding to a composite image obtained by weighting and adding two images.
The output M is, for example, brightness greater than 0 but brightness ys
In the region much lower than 1, the weight of the image in the A field
The weight f of the image in the B field is large.
Since the attachment g is small, that is, f> g> 0,
The composite image is mainly composed of the A field image and the B field image.
Creates a composite image that looks like a little mixed image
You. The synthesized image has, for example, a brightness of ys1.
In the larger area, the A field image is saturated, so
The weight f for the saturation value is small,
The weight g of the field image is large, that is, g>
Since f> 0, the composite image is a B-field image
To increase the brightness by mixing the saturation value of the image in the A field a little.
The characteristic that the brightness of the composite image increases smoothly with the addition
Image. That is, in the present embodiment, the first brightness y
From the first image saturated at s1 and the first brightness ys1
Saturated with high second brightness ys2 (four times ys1 in FIG. 5)
The second image to be summed is monotonically decreasing with respect to the first image;
For the second image, the monotonically increasing weighting factors f and g
Each weighted image is added by an adder to form one combined image.
I try to get a synthetic image. In this case, the sum of the weighting coefficients f and g is set to 1.
And the signal level of the composite image obtained by the addition
It becomes large when passing through the D range expansion circuit,
The image on the monitor 19 when displaying on the monitor 19 is saturated.
To prevent it from becoming bigger than
You. That is, if the sum is greater than 1, the D range expansion circuit
Although a compression circuit is required on the subsequent stage, in this embodiment
Is unnecessary. The output corresponding to the composite image shown in FIG.
The force Ma is taken at the brightness ya in the image of the A field.
The image signal x1a corresponds to the image signal x1a in the image of the B field.
The image signal x2a is weighted by weighting coefficients f and g, respectively.
Of the output Ma in this case.
The level is a weighting coefficient between the two signals x1a and x2a.
Divide by the division ratio proportional to the reciprocals 1 / f and 1 / g of f and g
Value. As can be seen from FIG. 5, the output M is bright.
If the function is a variable, its derivative is positive,
And it is monotonically decreasing. In other words, in areas with low brightness,
And continuously increases as the brightness increases
To reduce the gradient and increase the dynamic range.
You. The synthesized image obtained in this way is captured
By appropriately setting the coefficient p for the signal x,
Wide until the signal captured with the shutter time of 2 saturates
A function that increases continuously and smoothly in the range of brightness
It is possible to obtain a natural image without discomfort. One
That is, in the present embodiment, the composite image M is composed of two images (one
Weighted by all parts)
Calculation from the beginning.
Has a wide dynamock range that gives a composite image
And the brightness of the image increases as the brightness of the subject increases.
Natural and good image quality with smoothly changing characteristics
An image can be obtained. Also, if the subject is moving
In addition, the deviation of the contour between the low-luminance side and high-luminance side can be reduced.
You. In this embodiment, two images are weighted.
The sum of weighting coefficients f and g to 1
Setting, the image obtained by adding
Wide corresponding to the brightness of the image until the image is saturated
Circuit having dynamic range and D range expansion circuit 15
The signal level may be enlarged before and after passing through R etc.
Absent. On the other hand, Japanese Unexamined Patent Publication No.
In the previous example of the report, the boundary between two images was saturated with one image
Since the weights in the vicinity are changed and added,
In the area outside the vicinity, the signal level is
The compression circuit that compresses at the subsequent stage
You will need it. Further, a compression circuit is applied to the synthesized image.
Output level changes continuously with increasing brightness.
Characteristics, but the load on the compression circuit in this case is
growing. In this embodiment, the correction coefficient p is set to p ≒
(Π · ys2 / 2), but not limited to this, and the correction coefficient
p indicates that at least the output M is clear according to the characteristics of the CCD.
Choose to be a monotonically increasing function with increasing rust
It is possible. The output M is the above function, that is,
Chi xcos^Two(Px) + usin^Two(Px) Not only at low shutter speeds but at least in low brightness areas
Mainly for signals captured between
Any function may be used as long as it is mainly a signal captured in the data time. In FIG. 5, for simplicity, the signals x1,
x2 has been described in the case of changing linearly with brightness y
Can be similarly applied to the case where it is not nonlinear. In the present embodiment, the color separation circuit 9
R, G, and B signals are separated into RGB.
R range expansion circuit 15R for R and D Ren for G
And the D range expansion circuit 15B for B.
To increase the dynamic range.
The color separation circuit separates the signal into a luminance signal and a color difference signal.
Signal and color difference signal or only the luminance signal
The configuration may be such that the dynamic range is expanded. In this embodiment, after color separation,
It is configured to expand the dynamic range.
Not limited to this, as shown in FIG.
15R, G D range expansion circuit 15G and B D range
Instead of the enlargement circuit 15B, A / D converter 8
Immediately after the D conversion, a D range expansion circuit 40 is provided, and the A / D
Dynamic rendering is performed on the digital signal immediately after conversion.
May be configured to be larger, in this case other
Is the same as that of the present embodiment. In the first embodiment, the weighting factors f, g
Was determined by the B field image, but the A field
Field image and B field image.
You can also. In this case, the R range expansion circuit 15R for R
The configuration is shown in FIG. In FIG. 2, the output of the second selector 23 is
And the second LUTs 25 and 26.
However, in FIG. 7, the output of the first selector 22 is the first LUT 25
, And the output of the second selector 23 is
This is the point that is input to. Others are in Figure 2
Is the same as In FIG. 7, the first L
The weight coefficient f of the UT 25 is read, and the
The weighting coefficient g of the second LUT 26 is read from the image of
Is done. Also in this modification, the weighting coefficients f and g are set to f
= Cos^Two(Px), g = sin^Two(Px), p
x is the brightness when the second imaging signal x2 is saturated from 0
It is defined in the range of ys2. In the case of this modification, the A field and the B field
When the conditions for imaging in the field change (for example, A
The brightness of the subject changes between the field and the B field.
The first imaging signal in the A-field image
The weighting coefficient f read at the level of
Of the corresponding second imaging signal x2 in the field image.
The sum with the weighting coefficient g read by the bell deviates from 1.
Also appear. That is, in this case,
The numbers f and g are f + g ≒ 1. In contrast, the first implementation
, Etc., two weighting factors f,
g was determined, so set to satisfy f + g = 1
it can. (Second Embodiment) Second Embodiment
Are almost the same as in the first embodiment,
Only the points that are the same will be described.
Omitted. R range expansion circuit 15R for R of the present embodiment
Is transmitted to the knee & γ circuit 12 as shown in FIG.
One field of R signal after knee processing and gamma correction
Field memory 21 for storing digital signals for
And one field before stored in the field memory 21
Timing signal between the R signal of the current field and the R signal of the current field.
Switching by field discrimination signal from generator 4
And a selector 51 for outputting the field discrimination signal
Output level at pixel rate, input as address bits
A first LUT 5 for outputting a predetermined function value to be described later
2 and the field discrimination signal via the inverter circuit 53.
Input as upper address bits and output at pixel rate
A second L that outputs a predetermined function value to be described later for the level
UT 54, and the first multiplier 27 is a field memory.
The output at the pixel rate for the R signal
The output of the 1 LUT 52 is multiplied, and the second multiplier 28
Output at the pixel rate for the R signal from the e & γ circuit 12
The force is multiplied by the output of the second LUT 52. And adder
29 is an output of the first multiplier 27 and an output of the second multiplier 28
Is output to the enhancement circuit 16. The selector 51 is configured such that the field memory 21
A taken during the first shutter time (1/60 sec)
Knee & γ times when outputting field images
The image of the B field from the road 12 is selected, and the first LUT
In 52, a field discrimination signal is a higher address bit.
Is input as "0" and the second LUT 54 has a field
The discrimination signal is transmitted from the inverter circuit 43 to the upper address bit.
Is input as "1". The selector 51 has a field memory
21 is imaged in the second shutter time (1/240 sec)
When outputting the image of the B field
Select the image in the B field, and enter the image in the first LUT 42.
Input signal as upper address bit “1”
In the second LUT 54, a field discrimination signal is inverted.
Input as upper address bit "0" by the data circuit 53.
Is forced. The D range expansion circuits for G 15G and B
The D range expansion circuit 15B for the R
Since the circuit configuration is the same as that of the edge enlargement circuit 15R, the
Omitted. The first LUT 52 and the second LUT 5 described above
4 on a common memory map as shown in FIG.
The upper address bit is input as “0”.
When the output level at the pixel rate is x, co
s^Two(Px) and the upper address bit is “1”.
And the output level at the pixel rate is x
When sin^Two(Px) is output. So
The other configuration is the same as that of the first embodiment. Next, the operation of the present embodiment will be described with reference to FIGS.
This will be described with reference to the timing chart of (i). Thailand
Video signal VD from the mining generator 4 (FIG. 9A)
The field discrimination signal (FIG. 9 (b)) synchronized with
Selector 51 of range expansion circuit 15R, first LUT 52
And output to the second LUT 54 via the inverter circuit 24
Is done. The R output of the knee & γ circuit 12 is
Feed memory 21 of R range expansion circuit 15R for R (FIG.
9 (c)) and input to the selector 51 and the second multiplier 28.
And the output of the feed memory 21 (FIG. 9D) is selected.
And input to the first multiplier 27 and the first multiplier 27. The A field is stored in the feed memory 21.
Field when the digital signal of the image is stored
Circuit 41 from the selector 41 in response to the
The digital signal of the B-field image, which is the output of
Output to the first LUT 52 and the second LUT 54,
Is that the upper address bit is “0” in the first LUT 52.
And the output level at the pixel rate is x
cos^Two(Px) (FIG. 9 (e)), and the second LUT
In step 54, the upper address bit is input as "1" and
When the output level at the raw rate is x, sin^Two(P
x) is output (FIG. 9 (f)). Then, the pixel rate in the first multiplier 27 is
Digital signal and cos of A field image^Two(P
x) (FIG. 9 (g)), and the second multiplier 28
Digital signal of B field image at pixel rate
Issue and sin^Two(Px) is multiplied (FIG. 9 (h)). The B field is stored in the feed memory 21.
If the digital signal of the image of
Selector 41 from the feed memory 21 in response to the
The digital signal of the image of the B field passes through the first LUT
52 and the second LUT 54, in which case the first
In the LUT 52, the upper address bit is input as "1"
Where the output level at the pixel rate is x^Two
(Px) (FIG. 9 (e)), and the second LUT 54
Pixel rate when upper address bits are input as "0"
Cos when the output level at is x^TwoOutput (px)
(FIG. 9F). At this time, the first multiplier 27
And the digital signal of the B field image at the pixel rate
sin^Two(Px) (FIG. 9 (g)).
A-field image at pixel rate in squaring unit 28
Digital signal and cos^Two(Px) (FIG. 9)
(H)). Then, the adder 29 outputs the first multiplier 27
The output and the output of the second multiplier 28 are added (FIG. 9
(I)). Other configurations are the same as those of the first embodiment.
You. As described above, also in the present embodiment, the first
It is possible to obtain the same effect as that of the embodiment. As described above, the first and second embodiments
According to the image pickup apparatus of the embodiment and its modification, the image signal
The first step depends on the light amount level incident on the imaging means.
By changing the ratio of the imaging signal to the second imaging signal, the first imaging
A first image signal based on the image signal and a second image signal based on the second image signal;
Since the second image signal and the second image signal are synthesized,
In the expansion of the mic range, poor S / N on the low luminance side
And smooth input / output characteristics without discomfort
The effect is that natural images with varying
You. In the above-described embodiment, a long exposure
The first imaging signal corresponding to the first image captured between
Imaging corresponding to a second image captured with a short exposure time
Signal and a wide brightness range until the second imaging signal is saturated.
The first image signal and the second image signal are represented by two weighting coefficients f and g, respectively.
Weighted and added to the composite image
To obtain a combined imaging signal. On the other hand, when the first imaging signal is saturated,
In this case, the weighting coefficient G of the second imaging signal
For example, to generate an output M having the same characteristics as in FIG.
You may. The configuration in this case is, for example, the first embodiment.
The first LUT 25 and the second LUT 25 shown in FIG.
The data stored in the LUT 26 is different. For example, brightness y is 0
To ys1 are the same as in the first embodiment.
The output M1 corresponding to the composite image in the brightness range is     M1 = x1cos^Two(Px) + x2sin^Two(Px) (0 ≦ y <ys1) It is. When the brightness y is equal to or greater than ys1, the output M2
To M2 = x2 · G (x2 (y))(Y ≧ ys1) And In this case, the boundary condition of the output M2 is as follows:
Set. M2 (ys1) = M1 (ys1), dM1 / dy = dM2 / dy (where y = ys
1) (M1, M2 for brightness y at brightness ys1)
Equal). In this way, the outputs M1 and M2 are converted to the brightness y
Connect smoothly at s1. Also, M2 (ys2) = x2 (ys2) (= M (ys2)) And the output M2 is an example between the brightness ys1 and ys2.
For example, the connection characteristics are made smooth as shown in FIG. In this case, the coefficient of the first LUT 25 is 0
In the brightness range from to ys1, the image signal x2
Cos^Two (Px2) is output, and ys1
At the above brightness, the coefficient 0 is output. The second LUT 26 has a brightness ranging from 0 to y.
sin for the imaging signal x2 up to s1^Two (Px2)
Is output, and the weighting coefficient is output at a brightness of ys1 or more.
The attachment coefficient is set to a value that satisfies the above condition. The output corresponding to the composite image obtained in this case is
FIG. 10 shows the characteristics of the force M ′ with respect to the brightness y. This feature
The characteristics are basically the same as in FIG. However, the above configuration
As can be seen, the brightness is higher than the brightness at which the first imaging signal is saturated.
In the case of, the composite image is calculated using the weighting coefficient G of the second imaging signal.
Is determined. In the example shown in FIG. 10, the weighting coefficient G is bright.
The brightness y is monotonically decreasing, and G (y
s1)> 1, and brightness ys2 is 1, that is, G (ys2)
= 1. In the case of this modification, too, it is almost the same as the first embodiment.
It has a similar effect. In the above description, for example, FIG.
Is the D range expansion circuit 1 after the Knee & γ correction circuit 12
Since 5R, 15G, and 15B are arranged, the D range can be expanded.
Knee & γ correction by large circuit 15R, 15g, 15B
The function of the circuit 12 can also be used. Further, the output of the CCD 2 is
Of the image displayed on the monitor 19 at the value at which the output of the
The function of the amplifier that amplifies to the value is, for example, the D range in FIG.
It can be provided in the enlargement circuit 40. In other words, this place
If the sum of the weighting coefficients f and g is greater than 1,
It may be set to a fixed value. In this case, the D range expansion circuit
40 becomes the saturation value of the image displayed on the monitor 19.
Is set to a predetermined value. [Appendix] 1. A first imaging signal obtained by imaging an object at a first exposure time
And exposing the subject to a second exposure time shorter than the first exposure time.
In light time, the light amount level at which the first imaging signal is saturated is
And a second image signal that saturates at a higher value.
Imaging means for detecting the first and second imaging signals.
Monotonically decreasing with increasing light level until saturation
A small first weighting and a monotonically increasing second weighting
Under the condition that the sum of the first and second weights is 1,
Performing first and second weighting processing circuits;
The first and second imaging by a second weighting processing circuit
The first and second weightings are applied to the signal, respectively.
An addition circuit for adding the first and second image signals obtained
Output from the addition circuit,
Generates an image signal that can be displayed on the display
And an image processing device. 2. An imaging device according to claim 1, wherein said first and second
Variably set at least one of the exposure times
An exposure time setting unit is provided. 3. The first image of the subject taken at the first exposure time
1 image signal and the first image signal
The second shooting, which saturates at a value higher than the light level when
Imaging means for continuously outputting an image signal;
The amount of light until the second imaging signal is saturated
First weight of monotonically decreasing and monotonically increasing with increasing bell
The second weighting of the first and second weightings.
First and second weighting performed under the condition that the sum is 1
Processing circuit and the first and second weighting processing circuits.
For the first and second imaging signals, respectively.
First and second weighted first and second image signals
Image signal generating means comprising an addition circuit for adding
Display means for the image signal output from the adder circuit
Signal processing means for generating a video signal that can be displayed on the
Imaging device. 4. An imaging device according to claim 3, wherein said imaging means is
Imaging with a second exposure time shorter than the first exposure time.
As a result, the light amount level when the first imaging signal is saturated is
Outputting the second image signal that saturates at a value higher than the bell.
You. 5. An imaging device according to claim 3, wherein:
The image pickup means performs the second exposure time for generating the second image pickup signal.
In this case, the image is taken from the subject through the imaging window.
The amount of light incident on the
The light amount level when the first imaging signal is saturated.
Output the second image signal that saturates at a value higher than
You. 6. An imaging device according to claim 3, wherein the image signal synthesis is performed.
The means includes a light amount level when the first imaging signal is saturated.
In the light level region sufficiently lower than the
Is larger than the second weight, and the first imaging signal
In the light level range larger than the light level at which
The second weight is greater than the first weight.
Combine as follows. 7. The imaging device of claim 3, wherein the light level is
As x, the first weighting function F1 (x) is a coefficient
COS as p^Two (Px) and the second weighting
Function F2 (x) is SIN^Two (Px). 8. An imaging device according to claim 7, wherein
The coefficient p is the value of the imaging when the second imaging signal is saturated.
When the value of the light level incident on the window is s, p ≒ π
/ (2s) is satisfied. 9. An imaging device according to claim 3, wherein the image signal synthesis is performed.
Means for the first weighting of the first imaging signal
And the second weighting for the second imaging signal
It has weighting data storage means for storing each data. 10. The imaging device of claim 4, wherein the imaging means is
Capturing an image of the subject during the first exposure time;
And the second imaging signal are generated. 11. An imaging device according to claim 3, wherein the image signal
Forming means for calculating a differential coefficient of the synthesized image signal with respect to a light amount level;
The number of light levels until the second imaging signal is saturated.
Generating said composite image with characteristics that are continuous with increasing
You. 12. An imaging device according to claim 11, comprising:
The differential coefficient monotonically decreases with the light amount level. 13. A first imaging signal obtained by imaging an object at a first exposure time
And a second subject shorter than the first exposure time.
Light amount level at which the first imaging signal is saturated during the exposure time
Continuous output of the second image signal that saturates at a higher value
Imaging means, and for the first and second imaging signals
Monotonic with increasing light level until each saturates
The first weight of the decrease and the second weight of the monotone increase
The condition under which the sum of the first and second weights is approximately 1
Below, a first and a second weighting processing circuit,
The first and second weighting processing circuits
The first and second weights respectively for the imaging signals
Addition times for adding the injured first and second image signals
Image signal generating means comprising a road and
Video that can be displayed on the display means for the output image signal
An image pickup apparatus comprising: signal processing means for generating a signal. 14. The subject was imaged at the first exposure time
A first imaging signal and the first imaging signal representing the subject
The second, which saturates at a value higher than the light level when
An imaging means for continuously outputting an imaging signal;
And the amount of light until saturation for each of the second imaging signals
First weight of monotonic decrease with increasing level and monotonic
Increasing the second weighting by the first and second weighting
The first and second weights performed under the condition that the sum of
Weighting processing circuit and the first and second weighting processing circuits.
For the first and second imaging signals, respectively.
The first and second weighted first and second images
Image signal generating means comprising an adding circuit for adding the image signal.
And an image signal output from the addition circuit,
Signal processing means for generating a video signal displayable on a display means
And an imaging device comprising: 15. The subject was imaged at the first exposure time
A first imaging signal and the first imaging signal representing the subject
The second, which saturates at a value higher than the light level when
An imaging means for continuously outputting an imaging signal;
And the amount of light until saturation for each of the second imaging signals
First weight of monotonic decrease with increasing level and monotonic
Increasing the second weighting by the first and second weighting
Under the condition that the sum of is constant, at least the first imaging signal
For the range of light level until the signal is saturated,
And second to be performed on the second and third imaging signals, respectively.
Weighting processing circuit, and the first and second weighting processing
The first and second imaging signals by the
The first and second weighted first and second, respectively.
Image signal generator comprising an adder circuit for adding the image signal
And an image signal output from the addition circuit.
Signal processing for generating a video signal displayable on a display means
Means. 16. An imaging device according to claim 15,
A light amount level larger than a value at which the first image pickup signal is saturated
In the above, the first weighting processing circuit performs weighting of 0.
Output the coefficient. 17． An imaging device according to claim 15, wherein said first imaging device
At light levels higher than the value at which the image signal is saturated,
The second weighting circuit applies the second image signal to the second image signal.
The image signal is substantially determined by the weighting coefficient for
Is done. (18) Image a subject having the first brightness
Saturates the first image taken by the first exposure time control.
An image signal and a subject having a first brightness that is higher than the first brightness
The first exposure time shorter than the first exposure time is saturated when the body is imaged.
Continuous with the second imaging signal imaged by the exposure time control of 2.
Imaging means for selectively outputting light and light incident on the imaging means
The first imaging signal and the second imaging signal according to the volume level.
By changing the ratio of the image signal, the first image signal and the second image signal are changed.
Image signal combining means for combining the image signal of
Based on the image signal synthesized by the image signal synthesizing means,
Signal processing means for generating a video signal displayable on the display means;
An imaging device comprising: (19) Image a subject having the first brightness
Saturates the first image taken by the first exposure time control.
An image signal and a subject having a first brightness that is higher than the first brightness
The first exposure time shorter than the first exposure time is saturated when the body is imaged.
Continuous with the second imaging signal imaged by the exposure time control of 2.
Imaging means for selectively outputting light and light incident on the imaging means
According to the light amount level, when the light amount level is low,
Increase the ratio of the first imaging signal to the second imaging signal
When the light amount level is high, the first imaging signal
On the other hand, the ratio of the second imaging signal is increased,
Signal for synthesizing the second image signal and the second image signal
Synthesizing means, and an image synthesized by the image signal synthesizing means.
Generates a video signal that can be displayed on the display unit based on the image signal
Imaging device, comprising:
Place. (20) Image a subject having the first brightness
Saturates the first image taken by the first exposure time control.
An image signal and a subject having a first brightness that is higher than the first brightness
The first exposure time shorter than the first exposure time is saturated when the body is imaged.
Continuous with the second imaging signal imaged by the exposure time control of 2.
Imaging means for selectively outputting light and light incident on the imaging means
Depending on the light level, the higher the light level, the earlier
The first signal level for limiting the signal level of the first imaging signal.
Bell changing means, and light amount level incident on the imaging means
, The second light amount level becomes lower as the light amount level becomes lower.
Second signal level change for limiting the signal level of the imaging signal
Means and an output from the first signal level changing means.
The first image signal and the second signal level changing means
Image signal for synthesizing with the second imaging signal output from
Synthesizing means, and an image synthesized by the image signal synthesizing means.
Generates a video signal that can be displayed on the display unit based on the image signal
Imaging device, comprising:
Place. (21) The image signal synthesizing means includes:
The first image signal and the second image signal are combined and monotonically increased.
No additional feature 18, characterized in that an additional image signal is generated.
21. The imaging device according to item 20. (22) Image signals captured at two different exposure times
x, u (a ≦ x ≦ b: x is not saturated, x ≧ u)
about, f (t) + g (t) = 1, t = h (x) and d {f (t) + g (t)} / dt ≧ 0 M = xf (t) + u using functions f, g, and h that satisfy
A synthesis function that can be differentiated in an interval (a ≦ x ≦ b) defined as g (t)
It is characterized in that an image is synthesized at a pixel rate by a number M.
To expand the dynamic range of the image signal. (23) f (t), g (t) and h (t) are f (t) = cos^Two(T), g (t) = sin^Two(T),
t = px 22. The image pickup signal dump described in Appendix 22
How to expand the dynamic range. (24) Image a subject having the first brightness
Saturates the first image taken by the first exposure time control.
An image signal and a subject having a first brightness that is higher than the first brightness
The first exposure time shorter than the first exposure time is saturated when the body is imaged.
Continuous with the second imaging signal imaged by the exposure time control of 2.
Image signal in an image pickup apparatus having an image pickup means for outputting the image signal
The method of extending the dynamic range of the signal
The first imaging according to a light amount level incident on the means;
Changing the ratio of the signal to the second imaging signal,
First row for synthesizing the second imaging signal and the second imaging signal
And the image signal synthesized by the image signal synthesizing means.
Generating a video signal that can be displayed on display means based on
Dyna of imaging signal characterized by having two steps
How to expand the mic range. (25) Image a subject of first brightness
Saturates the first image taken by the first exposure time control.
An image signal and a subject having a first brightness that is higher than the first brightness
The first exposure time shorter than the first exposure time is saturated when the body is imaged.
Continuous with the second imaging signal imaged by the exposure time control of 2.
Image signal in an image pickup apparatus having an image pickup means for outputting the image signal
The method of extending the dynamic range of the signal
The light amount level according to the light amount level incident on the means;
Is lower than the first image pickup signal with respect to the second image pickup signal.
The ratio of the image signal is increased, and when the light amount level is high, the
The ratio of the second image signal to the first image signal is
High, the first imaging signal and the second imaging signal
A first step of synthesizing the image signal,
An image that can be displayed on the display means based on the synthesized image signal
And a second step of generating an image signal.
To expand the dynamic range of the image signal. (26) Image a subject having the first brightness
Saturates the first image taken by the first exposure time control.
An image signal and a subject having a first brightness that is higher than the first brightness
The first exposure time shorter than the first exposure time is saturated when the body is imaged.
Continuous with the second imaging signal imaged by the exposure time control of 2.
Image signal in an image pickup apparatus having an image pickup means for outputting the image signal
The method of extending the dynamic range of the signal
The light amount level according to the light amount level incident on the means;
The signal level of the first imaging signal is controlled as
A first step to limit the amount of light incident on the imaging means.
As the light level decreases according to the bell,
A second step of limiting the signal level of the imaging signal of No. 2 and
The first imaging signal from the first step and the second line
A third step of combining the second imaging signal with the second imaging signal from the step
And the image signal synthesized by the image signal synthesizing means.
Generating a video signal that can be displayed on the display means based on the fourth
Signal dynamics characterized by having the following steps:
How to extend the work range. (27) Image a subject with the first exposure time
Then, the first image captured by the first exposure time control is saturated.
A subject with an imaging signal and an exposure time shorter than the first exposure time
Image is captured by the second exposure time control, which is saturated when
An imaging unit that continuously outputs a second imaging signal;
According to the output level of each pixel of the imaging signal of
The ratio between the imaging signal of the second and the second imaging signal is changed,
An image for synthesizing the first image signal and the second image signal.
Image signal synthesizing means;
Video signal that can be displayed on the display means based on the extracted image signal
Signal processing means for generating
Imaging device. [0103] As described above, according to the present invention,Picture
To calculate the weighting factor for the luminance level change due to the element
Change in brightness level due to this pixel
Use the weighting factor obtained by the function based on
Prevents the S / N from lowering and provides a smooth ride.
A remarkable effect that a clear image can be obtained
Be done.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a configuration diagram showing a configuration of an imaging device according to a first embodiment of the present invention. FIG. 2 is a configuration diagram showing a configuration of an R D range expansion circuit of FIG. 1; FIG. 3A shows a first LUT and a second LUT of FIG. 2;
FIG. 3B is an explanatory diagram showing a manner in which weighted coefficients are output from a first LUT and a second LUT according to an input signal. 4 is a timing chart showing the timing of each signal of the R range expansion circuit for R in FIG. 2; FIG. 5 is an explanatory diagram for explaining the operation of the R D range expansion circuit of FIG. 2; FIG. 6 is a configuration diagram showing a configuration of a modified example of the imaging device of FIG. 1; FIG. 7 is a configuration diagram of an R D range expansion circuit according to a modification. FIG. 8 is a configuration diagram showing a configuration of an R D range expansion circuit according to a second embodiment of the present invention, and shows a first LUT and a second LUT;
The figure which shows the memory map of LUT. FIG. 9 is a timing chart showing the timing of each signal of the R range expansion circuit for R in FIG. 8A. FIG. 10 is a diagram showing characteristics relating to the brightness of an output corresponding to a synthesized image obtained in a modified example. [Description of Signs] 1 ... Imaging device 2 ... CCD 3 ... SSG 4 ... Timing generator 5 ... CCD driver 6 ... Preamplifier 7 ... CDS circuit 8 ... A / D converter 9 ... Color separation circuit 10 ... White balance circuit 11 ... AGC Circuit 12 ... Knee & γ circuit 15R ... R range expansion circuit for R 15G ... D range expansion circuit 15B ... D range expansion circuit 16 ... Enhancement circuit 17 ... 75Ω driver 18 ... D / A converter 19 ... Monitor 21 ... Field Memory 22 First selector 23 Second selector 24 Inverter circuit 25 First LUT 26 Second LUT 27 First multiplier 28 Second multiplier 29 Adder

──────────────────────────────────────────────────続 き Continued on front page (58) Field surveyed (Int.Cl. ⁷ , DB name) H04N 5/235 H04N 5/335

Claims (1)

(57) Claims: 1. An imaging means capable of imaging an object, and a first exposure in which the imaging means is saturated at a first light level.
Time drive control and a second light amount level higher than the first light amount level.
Second exposure time at which the imaging means is saturated at the light amount level of
And the first drive control is performed alternately, and the first
The first imaging signal imaged at the exposure time of
Alternately reading out the second imaging signal imaged in light time
Possible driving means, and the second from the imaging means read by the driving means.
For one imaging signal, a second signal having a monotonically decreasing curve characteristic
1st weighting that can be weighted by 1 weighting factor
A monotonically increasing song whose sum of the processing circuit and the first weighting coefficient is 1
A second weight for generating a second weighting factor having a line characteristic
And viewed with coefficient generating means, a second heavy generated by the second weighting coefficient generating means
Before being read out from the imaging means by the deposition coefficient
The sensitivity ratio of the second image signal to the first image signal is maintained.
Second weight processing circuit for performing weight processing while holding
And the first and second weighting circuits processed by the first and second weighting processing circuits.
An adding circuit for adding the image signal and the second image signal, and a display device based on the image signal output from the adding circuit.
Generating a video signal capable of displaying an image of the subject in a row
An image pickup apparatus comprising: a video signal processing unit .