JP2016005105A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To turn both a main subject and a background to a proper hue in image combination processing, the main subject and the background being irradiated by another light source different from a stroboscopic light during stroboscope emission.SOLUTION: An imaging apparatus is configured to image a subject and to process an obtained image signal. The imaging apparatus includes: means for calculating a first white balance correction value corresponding to a stroboscopic light from a main exposure image during stroboscope emission; means for calculating a second white balance correction value corresponding to an external light from an image during non-stroboscope-emission; means for dividing an image into plural blocks; means for calculating a stroboscope component and an external light component of each block from luminance values of the main exposure image and a non-stroboscope-emission image; means for calculating a combination ratio of blocks on the basis of the stroboscope components and the external light components of blocks; and means for combining image data which are obtained by performing development using the first white balance correction value and the second white balance correction value, on the basis of the combination ratio.

Description

  The present invention relates to an imaging apparatus, and more particularly to an imaging apparatus that includes a strobe capable of changing a color temperature and generates an image using white balance control.

An imaging apparatus using an imaging device such as a digital camera or a digital video camera has a white balance control function for adjusting the color tone of an image obtained by imaging.
For white balance control, a white object is imaged in advance to obtain the white balance coefficient, and the manual white balance control that applies the calculated white balance coefficient to the entire screen and the part that appears white from the captured image are automatically detected. Then, there is auto white balance control in which a white balance coefficient is obtained from an average value of each color component of the entire screen and the calculated white balance coefficient is applied to the entire screen.

Here, conventional auto white balance control will be described. The signal output from the image sensor is digitized by A / D conversion and divided into a plurality of blocks as shown in FIG. Each block is composed of R, G, and B color pixels, and color evaluation values (Cx [i], Cy [i]) are obtained for each block, for example, by the following calculation formula (1).
Cx [i] = (R [i] −B [i]) / Y [i] × 1024
Cy [i] = (R [i] + B [i] -2G [i]) / Y [i] × 1024 (1)
(Where i is a block number, R [i], G [i], and B [i] are average values of RGB pixels included in the block i, Y [i] = (R [i] + 2G [i] + B [I] / 4))
When the color evaluation value (Cx [i], Cy [i]) is included in the preset white detection range, it is determined that the block is white. Then, integral values SumR, SumG, and SumB of the color pixels included in the block are calculated, and white balance coefficients (WBCo_R, WBCo_G, and WBCo_B) are calculated as shown in the following equation (2).
WBCo_R = SumY × 1024 / sumR
WBCo_G = SumY × 1024 / sumG (2)
WBCo_B = SumY × 1024 / sumB
However, SumY = (sumR + 2 × sumG + SumB) / 4
However, such white balance control has the following problems. When there is a light source different from the flash light in the image in the flash lighting scene, the white balance coefficient calculated above is applied to the entire screen for white balance control. It was difficult to control. For example, if there is a low color temperature light source such as a light bulb color light source in the ambient light in a scene that is emitting a strobe light, the strobe light is a high color temperature light source. The white balance does not match the light source.

  On the other hand, when the white balance is matched with the low color temperature light source, the white balance is not matched with the strobe light. Even if white balance control is performed by adjusting the white balance in the middle of both light sources, the white balance does not match both light sources, the area illuminated by the strobe light is bluish, and the low color temperature light source is used. The irradiated area has a reddish color.

  In order to deal with such a problem, in Patent Document 1, a strobe light emission image and a strobe non-light emission image are compared for each arbitrary subject area to obtain a data ratio, and the contribution of strobe light is determined based on the ratio value. The white balance control is performed by selecting a white balance control value for each region of the video data when the flash is emitted and exposed according to the contribution. Patent Document 2 discloses a strobe light emission color temperature control method characterized in that the light emission color temperature is controlled by a signal from color temperature detection means for detecting the ambient color temperature.

Japanese Patent No. 3540485 JP-A-63-261331

  However, in the conventional technique disclosed in Patent Document 1 described above, when processing is applied to light sources having greatly different color temperatures, a tone tone jump occurs at the WB coefficient switching portion. Moreover, since the calculation of the strobe irradiation amount is complicated for a moving subject, it cannot be obtained accurately. In those cases, it is difficult to obtain a good image in the partial WB.

  Further, in the technique using the color temperature variable strobe of Patent Document 2, when the color temperature of the external light is outside the color temperature variable range of the strobe, the white balance is not optimal in the area where the strobe is irradiated and the area where it is not. There was a problem.

  In order to achieve the above object, according to the present invention, a strobe control unit capable of emitting light with a variable color temperature and control information corresponding to the color temperature emitted by the strobe control unit are used to determine a white balance control value for the strobe. A first white balance control value calculating means for calculating; a second white balance control value calculating means for calculating a white balance control value for external light by detecting an ambient color temperature; and the strobe. Strobe irradiation amount determining means for determining the amount of strobe irradiation by the control means for each image area, the first white balance control value calculating means, and the second white according to the strobe irradiation amount for each image area. From the result of the balance control value calculation means, there is a third white balance control value calculation means for calculating the third white balance control value for each image area. , Characterized in that the image generated using the third white balance control value.

  At that time, the white balance control value for external light and the white balance control value for strobe are compared, and if the difference of the white balance control value is within a predetermined range, white balance control is performed for each image area, When the difference is within a predetermined range, common white balance processing is performed in the screen. In this case, either the white balance control value for external light or the white balance control value for strobe, or from them White balance correction processing is performed using one obtained white balance control value.

  Further, the third white balance control value calculation means does not perform white balance control for each image area when the subject is a moving subject. It is possible to prevent a white balance correction error caused by an estimation error of a strobe irradiation amount generated in an area where the subject has moved, and to obtain a good image.

  In addition to the method of synthesizing the white balance control value itself, a first image generating means for generating an image by performing white balance control for strobe, and a second image generating a white balance coefficient at the surrounding color temperature. The first and second image generation units for each image region from the images generated by the first image generation unit and the second image generation unit according to the image generation unit and the strobe irradiation amount for each image region. You may combine the image which synthesize | combines an image.

  According to the present invention, it is possible to generate an image in which both the main subject and the background have appropriate colors in a scene that emits strobe light, and provide a preferable image to the user.

Flow chart showing image composition processing Diagram showing an example of block division The block diagram of the imaging device in the embodiment of the present invention Diagram showing white detection range Flow chart showing WB correction calculation processing Diagram showing time series in shooting The figure which shows the example of correction | amendment of the spectral difference by a drive mode Example of cutout calculation of subject position information Graph showing color-remaining processing A diagram clearly showing the relationship between the color temperature and the color evaluation value CxCy Block diagram showing switching of strobe color temperature and partial white balance Block diagram showing the data flow of image composition processing Block diagram showing the flow of data for white balance coefficient composition processing Figure showing an example of the flash table Diagram showing an example of the relationship between the flash output and color temperature Flow chart showing strobe control flow Flow chart for determining strobe color temperature and light emission Flowchart for performing subject motion detection processing based on image correlation

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

(Example 1)
FIG. 3 is a schematic diagram of this embodiment. In FIG. 3, reference numeral 301 denotes a solid-state imaging device made of a CCD, a CMOS, or the like, and its surface is covered with an RGB color filter such as a Bayer array so that color photography can be performed.

  The CPU 314 calculates the shutter speed and aperture value so that the entire image has a predetermined luminance, calculates the driving amount of the focus lens so as to focus on the subject in the focus area, and the exposure calculated by the CPU 314. The value (shutter speed, aperture value) and the driving amount of the focus lens are sent to the lens barrel control circuit 313, and the lens barrel device 315 is controlled based on each value. Similarly, when performing strobe light emission control, the strobe light emission amount is calculated by the CUP 314, and the strobe light emission device 316 emits light with a predetermined light amount.

  A white balance (WB) control unit 303 calculates a WB correction value based on information from the image signal stored in the memory 302, and uses the calculated WB correction value to store the image signal stored in the memory 302. WB correction is performed on. A detailed configuration of the WB control unit 303 and a method for calculating the WB correction value will be described later.

  Reference numeral 304 denotes a color matrix circuit that converts the color difference signals RY and BY by applying color gain so that the image signal corrected by the WB control unit 303 by WB correction is reproduced in an optimum color. Reference numeral 305 denotes a color difference signal R-. A low-pass filter (LPF) circuit that limits the Y and BY bands, and 306 is a CSUP (Chroma Suppress) circuit that suppresses a false color signal in a saturated portion of the image signal band-limited by the LPF circuit 305. On the other hand, the image signal WB corrected by the WB control unit 303 is also output to the luminance signal (Y) generation circuit 311 to generate the luminance signal Y, and the edge enhancement circuit 312 performs edge processing on the generated luminance signal Y. Emphasis processing is performed.

  The color difference signals RY and BY output from the CSUP circuit 306 and the luminance signal Y output from the edge enhancement circuit 312 are converted into RGB signals by the RGB conversion circuit 307, and the gamma correction circuit 308 converts the gradation signals to RGB signals. Tone correction is applied. Thereafter, it is converted into a YUV signal by the color luminance conversion circuit 309, further compressed by the compression circuit 310, and recorded as an image signal on an external recording medium or an internal recording medium.

  The strobe in this embodiment is, for example, an LED strobe, and the color temperature can be freely controlled by controlling each LED by arranging RGB filters so that the front part of each LED is irradiated. . Another method of controlling the color temperature is to control the color temperature of the light source by having each capacitor in the light source of high color temperature and low color temperature and controlling the charging capacity charged to that capacitor. However, it is also possible to control the color temperature by the distribution of current and voltage for each light source, or by the amount of time of the light emission time itself.

  In both methods, a light source using an LED can control the color temperature up to a range of color temperatures of a plurality of light sources, but cannot emit a light source having a color temperature outside that range.

Next, a WB correction value calculation method in the WB control unit 303 in FIG. 3 will be described in detail.
First, a first WB correction value calculation method when strobe light is emitted will be described with reference to FIG. The image signal stored in the memory 302 is read, and the screen is divided into arbitrary m blocks (step S101). Then, for each block (1 to m), the pixel value is added and averaged to each color to calculate a color average value (R [i], G [i], B [i],). The color evaluation values (Cx [i], Cy [i]) are calculated by using them (step S102).
Cx [i] = (R [i] −B [i]) / Y [i] × 1024
Cy [i] = (R [i] + B [i] -2G [i]) / Y [i] × 1024
However, Y [i] = (R [i] + 2G [i] + B [i]) / 4
Next, white detection is performed using a graph having coordinate axes as shown in FIG. 4 (step S103). The negative direction of the x coordinate (Cx) represents a color evaluation value when white of a high color temperature subject is photographed, and the positive direction represents a color evaluation value when white of a low color temperature subject is photographed. Further, the y coordinate (Cy) means the degree of the green component of the light source, and the green component increases as it goes in the negative direction, that is, it indicates a fluorescent lamp.

  The graph and table shown in FIG. 10 are blackbody radiation axes (hereinafter referred to as white axes), and show the relationship between color temperature and CxCy value. The correspondence between the color temperature and the CxCy value can be clarified by acquiring image data with a camera with a known color temperature light source on the blackbody radiation axis in advance and calculating CxCy. Of course, not only the color temperature direction but also the light source in the grain direction and magenta direction of the above-described white fluorescent lamp can similarly clarify the correspondence relationship with the CxCy value. Further, by acquiring the CxCy value corresponding to the above-described flash emission color temperature, if the flash emission color temperature is known, the WB correction value for the flash light source can be calculated.

  In step S103, it is determined whether the color evaluation value (Cx [i], Cy [i]) of the i-th block calculated in step S102 is included in the preset white detection range 401 shown in FIG. Step S103). The white detection range 401 limits the detection range because the strobe light is a known light source. If the calculated color evaluation values (Cx [i], Cy [i]) are included in the white detection range 401 (YES in step S103), it is determined that the block is white, and the color average of the block is determined. The values (R [i], G [i], B [i]) are accumulated (step S104), and if not included, the process proceeds to step S105 without adding. The processing of step S103 and step S104 can be expressed by equation (3).

  Here, in Expression (3), when the color evaluation value (Cx [i], Cy [i]) is included in the white detection range (401 in FIG. 4), Sw [i] is not included in 1. In this case, by setting Sw [i] to 0, the process of adding or not adding color evaluation values (R [i], G [i], B [i]) according to the determination in step S103 is substantially performed. Is going to. In step S105, it is determined whether or not the above process has been performed for all blocks. If there is an unprocessed block, the process returns to step S102 to repeat the above process, and if all the blocks have been processed, the process proceeds to step S106. .

In step S106, the first WB correction values (WBCol_R1, WBCol_G1, WBCol_B1) are calculated from the integrated values (sumR1, SumG1, SumB1) of the obtained color evaluation values using the following equation (4).
WBCol_R1 = sumY1 × 1024 / sumR1
WBCol_G1 = sumY1 × 1024 / sumG1 (4)
WBCol_B1 = sumY1 × 1024 / sumB1
However, sumY1 = (sumR1 + 2 × sumG1 + sumB1) / 4
Alternatively, the strobe light WB correction value may be set in advance as a known WB correction value.

  Next, a method for calculating the second WB correction value when the flash is not emitted will be described. The second WB correction value is calculated from the non-flash image data. FIG. 6 is a diagram in which shooting control is arranged in time series. A live view image is periodically shot before SW1, and AF lock and AE lock are performed at SW1. Test light emission and main exposure are performed after SW2, and image data exposed before “test light” captured before the test light emission is image data when no strobe light is emitted. Image data that has been exposed after the main exposure may be used as image data when the flash is not emitted.

  The calculation process is performed in the same manner as the first WB correction value calculation described above. The difference from the first WB correction value calculation processing is that processing is performed in a white detection range for external light as in the white detection range 402 in FIG. This is because strobe light is a known light source, whereas external light is not a known light source, and thus cannot be limited to the white detection range 401 during strobe light emission. The white detection range 402 is obtained by photographing white under different light sources in advance and plotting the calculated color evaluation values.

  This white detection range can be set separately depending on the shooting mode. The second WB correction value can be applied even when the imaging drive mode is different. For example, an EVF (Electronic View Finder) WB correction value calculated in the past can be used. However, when a spectral difference occurs depending on the drive mode, ΔCx and ΔCy are corrected as shown in FIG. 7 to calculate a WB correction value.

  Here, when the background is a low color temperature light source such as a light bulb color light source, the second WB correction value may be calculated so as to leave the color in consideration of the atmosphere of the scene. For example, by using a process as shown in FIG. 9 and controlling the color tone to remain when the color temperature of the second WB correction value is low, an image that retains the red color of the light bulb color light source is generated. Can do.

  Next, the image composition process will be described with reference to the flowchart of FIG. As shown in FIG. 1, image synthesis is performed when the difference between the first WB correction value and the second WB correction value is small, when there is movement of the subject, or when the distance between the imaging device and the subject is large. Normal white balance control is performed without processing. A normal white balance control method will be described later. In cases other than these conditions, image composition processing is performed. In the image composition process, first, in step S204 in FIG. 1, the developed image Yuv1 is developed from the strobe light emission image at the time of the main exposure using the first WB correction value.

  Next, in step S205, the developed image Yuv2 is developed using the second WB correction value from the strobe light emission image at the time of the main exposure. In step S206, the strobe light emission image, the strobe non-light emission image, the developed Yuv1 and the developed Yuv2 at the time of main exposure are each divided into n blocks. In step S207, the pixel values are added and averaged for each color for each block (1 to n) from the non-flash image to calculate a color average value (R2 [i], G2 [i], B2 [i],). The luminance value a [i] of each block is calculated. The formula for calculating the luminance value a [i] of each block is as shown in equation (6).

... (6)
The calculated luminance value a [i] of each block is used as the external light component of each block.
In step S208, as in the process of calculating the luminance value a [i] of each block from the non-flash image, blocks (1 to n) are obtained for each block (1 to n) from the flash image during the main exposure. For each pixel, the pixel value is added and averaged to each color to calculate the color average value (R1 [i], G1 [i], B1 [i]), and the luminance value b [i] of each block is calculated. By subtracting the luminance value a [i] of each block of the corresponding non-flash image from the calculated luminance value b [i] of each block, the strobe component c [i] for each block is calculated.
c [i] = b [i]-a [i]
In step S209, based on the ratio of the strobe component c [i] and the external light component a [i] for each corresponding block, the Mix rate α [i] for each block when combining Yuv1 and Yuv2 is calculated. .

  In step S210, Yuv1 and Yuv2 are synthesized using the mix rate α [i] for each block to generate Yuv3. The color evaluation values (Y3 [i], u3 [i], v3 [i]) in Yuv3 are the color evaluation values (Y1 [i], u1 [i], v1 [i]) in Yuv2, and the color evaluation values (Y2 in Yuv2) [I], u2 [i], v2 [i]) are used to calculate as in equation (7).

        ... (7)

  In order to alleviate the color shift at the block boundary part, the pixel interpolation process may be performed in step S209 to calculate the mix rate α ′ [j] for each pixel from the mix rate α [i] for each block. For example, bilinear interpolation is used as the pixel interpolation process, and the Mix rate α ′ [j] for each pixel is calculated from the Mix rate α [i] for each block. At this time, in step S210, Yuv1 and Yuv2 are synthesized using the mix rate α ′ [j] for each pixel to generate Yuv3.

  The color evaluation values (Y3 [j], u3 [j], v3 [j]) in Yuv3 are the color evaluation values in Yuv1 (Y1 [j], u1 [j], v1 [j]). The color evaluation values in Yuv2 (Y2 [J], u2 [j], v2 [j]) are used to calculate as in equation (8).

        ... (8)

  Further, in step S210, the processing may be made variable by determining whether or not to perform image composition processing for each block according to the block luminance value in the strobe light emission image at the time of main exposure. When the block luminance value of the block is low or high, image synthesis processing is not performed on the block, and image development processing is performed using a WB correction value calculated by normal white balance control described later. When the block luminance value of the block is other than that, the same processing as the image composition processing described above is performed.

  Next, a normal white balance control method will be described in detail. First, the first and second WB correction values are calculated in the same manner as the processing for calculating the WB correction value described above. Subsequently, Mix (weighted addition) processing of the first and second WB correction values is performed. In the Mix (weighted addition) process, weighted addition is performed according to the irradiation ratio of external light and strobe light applied to the subject.

  The subject area cutout for calculating the irradiation ratio will be described. Image data before test light emission is acquired, and a luminance block a composed of this pre-light emission image data or an m × n matrix shown in FIG. 8B is calculated. Next, flash test light emission is performed under the same conditions as before the test light emission, data at the time of test light emission is acquired, and similarly, a luminance block b configured by an m × n matrix is acquired.

  The luminance blocks a and b are temporarily stored in a RAM or the like provided in the imaging device. The background image of the subject image is assumed to be substantially unchanged between the pre-test light emission image data and the test light emission data. Therefore, the difference data between the two luminance blocks a and b becomes the reflected light of the subject area during the flash test light emission, and the subject position information c can be obtained from the difference data.

  The subject position block is acquired from the subject position information thus calculated, and the main exposure brightness (Y1) and strobe non-flash brightness (Y2) of the acquired block are calculated. If the exposure conditions at the time of main exposure and non-light emission are different, Y1 and Y2 are calculated by aligning the exposure conditions. Mix processing (weighted addition) based on the light amount ratio is performed using the first and second WB correction values with the ratio of Y1 and Y2 calculated in this way as the light amount ratio irradiated to the subject. The WB correction value obtained in this way is determined as the WB correction value to be used for the WB processing by the WB control unit 303, and the image is developed by the image developing unit to obtain final image data. FIG. 12 is a block diagram showing the flow of the synthesis process in this embodiment. When a shooting instruction is given, the EVF image 102 and the main exposure image 100 immediately before the SW2 is pressed are also saved. The stored image at the time of the main exposure is resized to a size equivalent to the EVF image in the resizing processing block 101, and then the block integration is calculated. Similarly, the block integral value is calculated for the EVF image. At that time, the integration area of the block is set to be the same with respect to the angle of view in the EVF and the main exposure image.

  At 103, an exposure amount difference between main exposure and EVF shooting is calculated from the relationship between sensitivity, aperture value, and shutter speed during main exposure and EVF, and an EVF image is obtained so that the block integration value becomes the same exposure amount. The conversion processing of the block integral value is performed. At 104, the strobe irradiation amount is calculated for each image area. In 105, the image composition ratio is determined in accordance with the strobe irradiation amount. In 106, the composition ratio for each block is enlarged to the same size as the main exposure image.

  In the external light WB coefficient calculation processing 107, the external light WB coefficient is calculated from the EVF image, the WB multiplication is performed in the WB processing block 109 on the main exposure image, and the development processing is performed in the development processing block 113. And a YUV image for external light is output.

  Also, in the strobe WB coefficient calculation process 111, a strobe WB coefficient is calculated from the strobe emission amount information 110. In the WB calculation process 109, a strobe WB process is performed on the main exposure image, a development process is performed in the development processing block 113, and a strobe YUV image is output.

  The strobe YUV image and the external light YUV image are subjected to image combining processing in the image combining processing 114 according to the image combining ratio.

  FIG. 13 is almost the same as FIG. 12, but differs from FIG. 12 in that the image is synthesized after the development processing, whereas FIG. 13 differs in that the external light and the WB coefficient of the strobe are synthesized. Instead, a WB coefficient synthesis process 201 is performed for synthesizing the external light WB coefficient and strobe WB coefficient for each image area. The WB coefficient is used for WB multiplication in the WB process 119 and developed in the development process 113. Processing is done.

  In FIG. 12, it is possible to generate a good image including color reproduction in which the optimum color processing parameter can be set for each light source. On the other hand, in FIG. 13, since the development process does not need to be performed twice, it is not necessary to secure a calculation time and a memory for storing images, and this is a technique that can save system resources.

  FIG. 11 is a flowchart showing the flow of image composition processing in this embodiment. When the photographing instruction SW2 is pressed, in S301, the color temperature is calculated from the white balance result CxCy at the time of the final live view. When the CxCy value is not located on the white axis, it can be simply calculated based on the relationship between the Cx value and the color temperature. In addition, a perpendicular may be drawn on the white axis to calculate the color temperature from the value of the CxCy value at the intersection of the white axes.

  In S302, the exposure value, the flash emission amount, and the emission color temperature are determined. Details will be described later. In S303, a photographing operation is performed. In S304, the strobe emission color temperature determined in S303 is compared with the color temperature of the external light to determine whether it is included in a predetermined range. If it falls within the predetermined range, a normal WB process is performed in S306.

  If it is not included in the predetermined range in S304, the image composition processing in S304 is performed. The image composition processing here is the image composition processing according to the strobe irradiation amount in the subject described above. In step S307, the generated image is saved and the process ends.

  FIG. 16 is a block diagram illustrating in detail a method for determining a shooting exposure condition in the present embodiment. In S400, the Tv value (AE_Tv) is determined by the AE operation. Specifically, during EVF operation, imaging is performed under predetermined imaging conditions, and CCD RAW data is stored in a memory. The image data is divided into a plurality of blocks, and an average luminance value is calculated for each block.

Thereafter, a weighted average value (Y_Ave) is calculated for each block. In that case, you may perform center-weighted metering to calculate the screen average value with weighting of the center weight, or evaluation metering according to the subject, such as weighting the subject area according to the result of face detection. Alternatively, average photometry for simply calculating the screen average value may be performed, or spot photometry for performing AE only on a part of the spot area on the screen may be performed. The difference between the screen average value and the AE target value (AETarget) is converted into the number of stages (DeltaAE), and the Bv value is determined according to the following equation.
Bv = Av + Tv-Sv + DeltaAE (9)
Here, DeltaAE = Log (Y_Ave / AETarget) / Log
Next, from the Bv value, the aperture, shutter speed, and photographing sensitivity are determined according to the program diagram and the user setting value.

  In S401, it is determined whether or not to emit the strobe from the Bv value and the result of the backlight determination. For example, a strobe may be emitted when the shutter speed becomes 1/40 or less, and it is determined that a strobe is emitted when it is determined that the scene is backlit, such as when the face area is dark as a result of screen analysis. Also good. If the flash is not emitted, the EF process ends.

  If it is determined that the strobe light is emitted, an exposure condition for strobe light emission is determined in S402. When the shutter speed determined by AE is outside the flash synchronization speed, the shutter speed may be limited within the flash synchronization speed, or the exposure time determined by AE may be used. When the exposure is changed, the program shift is performed by varying the Av value and the Sv value so that the expression (9) is satisfied.

  In S403, the pre-flash exposure condition is determined from the Av / Tv · Sv value determined in S402. In the case of a general CCD, an image for pre-emission may be driven with a high pixel addition and thinning rate in order to perform sensor driving with a high readout speed. For this reason, there may be a case where the sensitivity that can follow the main exposure may change. Therefore, it is necessary to change the exposure by the sensitivity change.

  The pre-flash emission may be determined by using a predetermined flash amount determined in advance for each Bv value, or by estimating the subject distance from the face size or AF result, and depending on the subject distance. The pre-emission amount may be determined. Of course, it is needless to say that it must be determined within the range of the pre-flash exposure time. In the case of a strobe with variable color temperature, the color temperature of the strobe is set to a color temperature close to the color temperature of the external light according to the color temperature of the external light.

  In S404, image data is acquired in a non-flashing state under the shooting conditions determined in S403. In S405, pre-flash is actually performed to acquire an image. In S406, the block average value of each of the image that has been subjected to the strobe light emission in S404 and the image that has not been subjected to the S405 strobe light emission is calculated.

The return value reached by the strobe is calculated for each block from the difference between the two images indicating whether or not the strobe emits light, and the luminance value (PreY) obtained by weighting and adding the weight corresponding to the photometric method in the screen is calculated. Similarly, the luminance value obtained by adding the weighted average is similarly calculated even when the image is added to the non-flash image (NoFlash_Y). In step S407, the amount of light emitted from the strobe (Delta EF) is calculated. More specifically, with respect to the luminance value for each block of the non-light-emitting image of the strobe in S406,
AETarget = α × PreY × ExpCrtGain_PreFlash + NoFlash_Y × ExpCrtGain_NoFlash (10)
Α value satisfying the above is calculated.

However,
ExpCrtGain_PreFlash: Correction value for correcting the difference between the aperture and sensitivity at the time of main exposure and pre-flash ExpCrtGain_NoFlash: The correction value for correcting the difference between the aperture, sensitivity and shutter speed at the time of main exposure and non-flash photography The number of stages is converted from α to satisfy, and the value is calculated as DeltaEF. FIG. 14 is a graph and a table showing how to calculate the flash emission time with respect to the value of DeltaEF. The vertical axis of the graph is the DeltaEF value with respect to the amount of light at the time of pre-emission, and the horizontal axis is the flash emission time (unit: μSec).

  In the present invention, when it is determined that a desired image cannot be obtained as a result of performing pre-flash of the strobe, a mode for permitting the change of the exposure condition is provided, and the setting is referred to as “Safety EF”.

  If SafetyEF is invalid in S408, shooting is performed under the exposure conditions determined in S407. If SafetyEF is ON in S408, the flash emission time (Flash Duration) is calculated in S409.

  A typical strobe uses a xenon tube. The light emission characteristic of the xenon tube controls the light emission time by controlling the current, but afterglow occurs even when the current is stopped. Therefore, in S409, the strobe light emission time is included including the strobe current control time and the expected afterglow time. For example, if the strobe light emission time is 1200 μSec and the afterglow time is 200 μSec, the strobe light emission time is 1400 μSec. In S409, the longest Tv (FlashMaxTv) at which the strobe can be tuned is calculated from the strobe light emission time calculated in S409.

  Of course, individual variations are also caused in the light emission / end time of the strobe, so that portion may be included as a margin. In S411, the strobe light emission time is compared with the exposure time determined in S403. If the shutter speed for the main exposure is lower, the EF shooting condition process is terminated as it is. If the shutter speed for the main exposure is faster, the exposure time is set to FlashMaxTv or less in S412 and the photographing conditions are recalculated so as to satisfy Equations 1 and 2.

  In this case, since the exposure time is shortened and the aperture value is decreased, the aperture value is reset in S413. In step S414, the flash emission amount is reset according to equation (10) by the amount changed. In this case, since the aperture is shifted to the open side, the amount of light emitted from the strobe is controlled to be small, so that the strobe light emission time does not become longer than the Tv value set in S412. After that, the process returns to S409 to reset the exposure conditions.

  As described above, at the time of shooting, the subject brightness before pre-flash and the subject brightness at pre-flash are calculated, the degree of brightening due to strobe flashing is calculated from the difference, and the subject brightness is set to an appropriate level based on these values. The difference in the number of stages is calculated as follows. The strobe light emission amount can be calculated by referring to the light emission table from the pre-light emission amount for the difference in the number of stages.

  In S403, it is desirable that the color temperature at the time of pre-emission is close to the color temperature of outside light. The reason why the color temperature at the time of main emission and the color temperature at the time of pre-emission are the same is the main exposure. This is because the amount of light emission at the time can be calculated more accurately. When performing the calculation, the light emission table shown in FIG. 14 is calculated for each color temperature, so that the light emission amount at the time of actual photographing can be calculated even if the color temperature changes. Of course, if the light emission table for each color temperature is different from the color temperature at the time of shooting, calculation may be performed by linear interpolation between the tables including the color temperature at the time of shooting, or A light emitting table may be substituted.

  Hereinafter, the details of the method for determining the exposure value of S302, the light emission amount of the strobe light, and the light emission color temperature will be described. Here, the color temperature and light emission amount of the strobe are determined from the relationship between the light emission amount of the strobe and the color temperature of the strobe.

  FIG. 17 is a block diagram showing processing blocks for determining the flash emission amount and the color temperature. In S501, shooting conditions such as the above-described exposure conditions and flash emission determination are determined. In S502, DeltaEF is compared with MaxDeltaEF_Temp which is the maximum DeltaEF value in the light emission table at a predetermined color temperature.

  When the difference between DeltaEF and MaxDeltaEF_Temp is equal to or smaller than a predetermined threshold, light emission is performed at the set color temperature and Duration value. On the other hand, if the difference between DeltaEF and MaxDeltaEF_Temp is greater than or equal to a predetermined threshold and the value of DeltaEF is large, the color temperature variable processing in S503 is performed.

  FIG. 15 is a graph showing an example of a step between the emission color temperature and the emission amount. If the pre-emission color temperature differs from the main emission color temperature, the emission amount used for pre-emission is calculated in advance for each color temperature, and the difference between the emission amounts for each color temperature and the emission table described above are calculated. The amount of light emission can be determined by adding the difference.

Similarly, in MaxDeltaEF_Temp, the difference in the maximum light emission amount may be tabulated for each color temperature. However, the difference in light emission amount for each color temperature is a difference in brightness caused by the difference in spectral transmittance of the color filter applied to the front part of the LED, so the number of light emission stages at the maximum light emission amount and the color temperature in the normal light emission region The difference is the same.
However, this is not the case when the number of LEDs and the power value are different for each color filter, and it is necessary to calculate the maximum light emission amount for each color temperature as described above.

  By using the maximum light emission table for each color temperature, it is possible to calculate a change in brightness caused by changing the color temperature. The difference between DeltaEF and MaxDeltaEF_Temp is included in a predetermined threshold, and is as close to the color temperature of the external light as possible. The strobe color temperature is selected. If the color temperature does not fall within the predetermined range even when the color temperature is changed, the color temperature that provides the maximum light emission amount is selected.

  In S504, it is determined whether or not the strobe light emission time has changed when the color temperature is changed in S503. If the flash light emission time is extended, the exposure is re-determined in S505 by performing recalculation of S409-S414 in S501.

  In general, in the case of an LED light source that superimposes the three primary colors, the low color temperature light source mainly emits light from the red filter part LED, and the LED light source of a daylight white color temperature light source uses red / The color temperature is controlled by emitting light from all green and blue LED light sources. For this reason, the number of light sources emitted from the low color temperature light source is reduced, and the amount of light is reduced. Of course, it is possible to increase the amount of light by applying the corresponding current or voltage, but basically, the maximum light emission is the light emission at a medium color temperature at which all three LEDs can emit light uniformly.

  That is, the maximum amount of light that can be emitted differs depending on the color temperature of the strobe, and therefore the relationship between the emission color temperature that can be photographed and the desired amount of light emission differs. In such a case, the shortage of the light emission amount is compensated by changing the light emission color temperature according to the unachieved light emission amount.

  Further, it may depend on the time during which the strobe can emit light, that is, the shutter speed. As can be seen from FIG. 14, when the shutter speed is sufficiently long, it is possible to extend the light emission time in order to increase the light emission amount of the flash, but when the shutter speed is short, the flash irradiation time is also limited. Therefore, a sufficient amount of light emission cannot be obtained. In such a case, a desired light amount can be obtained by setting a light emission time of medium color temperature that can obtain a large amount of light within a short time, but it may be different from the color temperature of external light. is there.

  Next, the subject tracking method will be described. Many methods for detecting the correlation and motion vector of an object have been proposed, and the simplest method will be described here.

  FIG. 18 is a flowchart for calculating the image correlation. In S201, image data from which correlation is obtained is acquired. In step S202, a target frame image for obtaining image correlation is acquired. In step S203, the position of the image area of the target frame is shifted in the vertical, horizontal, and diagonal directions. Here, the position shift amount is moved to the vector movement amount detection range. Of course, in order to see a wide range, it takes a long time for detection because the number of comparison targets increases, and it goes without saying that the detection capability decreases when the target region is narrowed.

  In S204, an image difference between the pixel-shifted image and the reference frame is calculated. In S205, the integral value of the absolute value of the image difference in S204 is acquired. Needless to say, the smaller the absolute value of the integral value of the image difference, the higher the image correlation. In S206, when the correlation value is high, the correlation value and the amount of pixel shift are stored. In S208, it is determined whether or not the pixel shift has been completed in all directions, and if not completed, the process returns to S203. If all directions have been calculated, the direction and correlation value having the highest correlation are calculated in S209.

It is possible to detect the motion of the subject by calculating a motion vector by such a method, performing these for each of a plurality of image areas, and calculating an integral value of the motion vector amount.
There are various methods for calculating the motion vector, and this method is not limited. For example, a person or a main subject may be detected, and the motion may be determined based on the difference between the detected position coordinates. By performing these determinations immediately before SW2 shooting and determining the amount of movement, it is possible to estimate the amount of movement during shooting.

  A method other than motion vector detection in motion discrimination will be described. In the motion discrimination method using EVF, even if the EVF image is stationary, subject blurring such as shaking of the person being exposed or camera shake caused by the photographer may occur. Due to the subtle difference between the strobe illumination area and the WB control area, a white balance control error occurs at the boundary between the subject and the background area, resulting in a color fringing.

As a technique for avoiding these problems, a method for determining the motion between the main exposure and the EVF image will be described in detail.
1. A color evaluation value CxCy value is acquired for each block from the EVF image immediately before SW2.
2. The exposure amount is converted from the strobe light emission image and the strobe non-light emission image, and the strobe irradiation amount is calculated for each block.
3. The degree of color change is calculated according to the light amount ratio from the difference between the white balance coefficient when the flash is not emitted and the white balance coefficient when the flash is emitted.
By calculating the light intensity ratio between the strobe and the outside light for each area according to the strobe irradiation amount of 4.2, and adding it to the color evaluation value of the strobe non-emission image from the color change degree calculated in 3. A color estimation evaluation value of the luminescent image is calculated.
The color evaluation value of the main exposure image is acquired.
If the comparison result of 6.4 and 5 is equal to or smaller than a predetermined threshold value, it is determined that the area is not moving, and if it is equal to or higher than the predetermined threshold value, the area is determined to be moving. To do.
From the result of 7.6, the number of blocks in the area that has been moved is counted, and when the number of blocks is greater than or equal to a predetermined number, it is determined as a moving scene.

  In the above example, the motion is determined based on the number of motion blocks. It may be determined by a combination. By doing so, it is possible to perform motion discrimination between the strobe light emission image and the strobe non-light emission image.

(Other embodiments)
The present invention can be applied to an apparatus (for example, a digital still camera, a digital video camera, etc.) composed of a single device even if a system composed of a plurality of devices (for example, a host computer, an interface device, a camera head, etc.) is applied. May be.

  The object of the present invention can also be achieved as follows. First, a storage medium (or recording medium) that records a program code of software that implements the functions of the above-described embodiments is supplied to a system or apparatus. Then, the computer (or CPU or MPU) of the system or apparatus reads and executes the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also the following can be achieved. That is, when the operating system (OS) running on the computer performs part or all of the actual processing based on the instruction of the read program code, the functions of the above-described embodiments are realized by the processing. It is.

  As the recording medium for storing the program code, for example, a flexible disk, hard disk, ROM, RAM, magnetic tape, nonvolatile memory card, CD-ROM, CD-R, DVD, optical disk, magneto-optical disk, MO, etc. Can be considered. Also, a computer network such as a LAN (Local Area Network) or a WAN (Wide Area Network) can be used to supply the program code.

100 exposure images, 101 resize processing, 102 EVF images,
103 exposure amount calculation processing, 104 strobe emission amount calculation processing for each area,
105 image composition ratio determination processing, 106 resizing processing,
107 external light WB coefficient calculation processing, 109 WB processing, 110 emission amount information,
111 Strobe WB coefficient calculation processing, 113 development processing, 114 image composition processing,
201 WB coefficient composition processing, 301 CCD image sensor, 302 memory,
303 WB circuit, 304 color conversion MTX circuit, 305 LPF circuit,
306 CSUP circuit, 307 RGB conversion circuit, 308 gamma correction circuit,
309 color luminance conversion circuit, 310 compression circuit, 311 Y generation circuit,
312 edge enhancement circuit, 313 lens barrel control circuit, 314 CPU,
315 Lens barrel device, 316 Strobe light emitting device

Claims (10)

An imaging device,
Strobe control means that can emit light with variable color temperature;
First white balance control value calculating means for calculating a white balance control value for a strobe from control information corresponding to a color temperature emitted by the strobe control means;
Second white balance control value calculation means for calculating a white balance control value for external light by detecting the ambient color temperature;
Image dividing means for dividing the image data into a plurality of blocks; strobe irradiation amount determining means for determining the amount of strobe irradiation by the strobe control means for each area of the image;
A third white balance control value is calculated for each image area from the results of the first white balance control value calculation means and the second white balance control value calculation means in accordance with the strobe irradiation amount for each image area. Third white balance control value calculating means for performing
Have
An image pickup apparatus that generates an image using the third white balance control value.   When the white balance control value for external light and the white balance control value for strobe are compared, and the difference of the white balance control value is within a predetermined range, white balance control is performed for each image area, and the difference is The image pickup apparatus according to claim 1, wherein a common white balance process is performed in a screen when the image is within a predetermined range.   When common white balance processing is performed in the screen, white balance correction processing is performed using either a white balance control value for external light or a white balance control value for strobe, or one white balance control value obtained from the white balance control value. The imaging apparatus according to claim 2, wherein:   The difference in the white balance control value is within a predetermined range, wherein the white balance control value is determined for each color evaluation region of a color evaluation value obtained for each image region. Imaging device. An imaging device,
Strobe control means capable of emitting light with variable color temperature; first white balance control value calculating means for performing white balance control for strobe light from control information corresponding to the color temperature emitted by the strobe control means;
The first image generation means for generating an image using the first white balance control value and the surrounding color temperature are estimated to perform white control for external light and calculate a second white balance control value to obtain an image. Second image generating means for generating;
A strobe amount determining means for determining a strobe amount by the strobe control means for each area of the image;
An image that combines the first and second images for each image region from the images generated by the first image generation unit and the second image generation unit in accordance with the strobe irradiation amount for each image region. Combining means;
An imaging device comprising: Compare the white balance control value for external light and the white balance control value for strobe, and if the difference of the white balance control value is not within the specified range, determine the composition ratio for each image area Image composition,
When the difference is within a predetermined range, the image is based on a combination ratio calculated using one of the control values of the white balance control for external light or the control value of the white balance for strobe, or both. The imaging apparatus according to claim 5, wherein the image generation is performed by combining the images.   The difference in the white balance control value is within a predetermined range, and the white balance control value is determined for each color evaluation region of a color evaluation value obtained for each image region. Imaging device.   The third white balance control value calculating means determines the white balance control and the composition ratio of the composition processing for each image area according to the determination result of the motion determining means for determining whether the subject is a moving subject. The imaging apparatus according to any one of claims 1 to 7, wherein the imaging apparatus is characterized in that   The imaging apparatus according to claim 8, wherein the movement determination unit performs a movement determination process using a plurality of images immediately before a still image shooting instruction.   The motion determination means determines a strobe irradiation amount for each image region using an image immediately before the still image shooting instruction and a still image shooting image, and determines a color of the strobe light emission image based on the determination result of the strobe irradiation amount. In addition to estimating the taste, it is determined as a still area when it is within a predetermined range from the estimated value compared to a still image, and is determined as a moving area when it is outside the predetermined range. The image pickup apparatus according to claim 8, wherein a motion determination is performed based on the motion determination result of the image.