JP2009200743A - Image processor, image processing method, image processing program and imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To optimize the brightness and gradation of a specific target region (in an image) according to various shooting conditions without preparing a plurality of sets of feature histograms and tone curves in advance.
SOLUTION: A specific target region detecting unit 27 for detecting a specific target region from the entire image data, and a histogram relating to luminance of the specific target region image data detected by the specific target region detecting unit 27 is calculated as a specific target region luminance histogram. The specific target region luminance histogram calculation means 28, the tone curve generation means 23 for generating a tone curve from a predetermined luminance histogram, and the image data corrected by the tone curve generated by the tone curve generation means 23 are used as output image data. And an image correction means 24.
[Selection] Figure 3

Description

  The present invention relates to an image processing device, an image processing method, an image processing program, and an imaging device.

  An imaging apparatus such as a digital camera generally incorporates an automatic exposure control mechanism for optimizing exposure. As the exposure control method, the screen is divided into a plurality of areas and metered, and the average metering system that controls the exposure according to the weighted average value of the light quantity in those areas, by focusing on the central part of the screen A center-weighted metering method for controlling the exposure and a spot metering method for controlling the exposure by metering a local point on the screen are generally employed.

  Even in an imaging apparatus equipped with such an automatic exposure control mechanism, it is often difficult to perform proper exposure adjustment, for example, in shooting with backlight.

  For example, in a state where the sun is directly behind the subject, the luminance difference between the background and the subject is large, so that the subject is blacked out in the average metering method or the center-weighted metering method. Also, the background part such as the sky often flickers white. According to the spot metering method, the exposure can be controlled so that the subject is not blacked out even with true backlight, but overexposure tends to occur in the background such as the sky. Even in the spot photometry method, the position and size of the photometry frame in the screen are constant, so that it is not always possible to achieve proper exposure of the subject under various shooting conditions.

  In view of this, various measures have been proposed in order to shoot with appropriate exposure even in such a scene where it is difficult to achieve appropriate exposure.

  For example, in Patent Document 1, it is determined whether the backlight state is true backlight or other backlight by evaluating the degree of polarization of the luminance histogram, and dynamic range correction and tone curve correction suitable for each are performed. A method has been proposed. In this method, a backlight scene is considered, and there is a problem that proper correction is not performed in other scenes.

In Patent Document 2, a feature histogram corresponding to a plurality of typical captured scenes and a tone curve corresponding to the feature histogram are prepared in advance, and the degree of approximation with the histogram of the captured image is obtained, and the feature histogram having the highest degree of approximation is supported. A method has been proposed in which the tone curve is corrected based on the degree of approximation, and the tone correction of the captured image is performed using the tone curve. According to this method, it is possible to perform gradation correction of captured images in many scenes, but there is a problem that a plurality of sets of feature histograms and tone curves must be prepared in advance.
JP 2002-92607 A JP 2007-0697907

  The present invention optimizes the brightness and gradation of a specific target area (for example, a person's face) in an image according to various shooting conditions without preparing a plurality of sets of feature histograms and tone curves in advance. An object of the present invention is to provide an image processing apparatus, an image processing method, an image processing program, and an imaging apparatus that can be configured as follows.

In order to achieve the above object, the invention according to claim 1 is a specific target area detecting means for detecting a specific target area from the entire image data having luminance information,
Specific target area luminance histogram calculating means for calculating a histogram relating to the luminance of the specific target area image data detected by the specific target area detection means among the entire image data having luminance information;
A tone curve generating means for generating a tone curve;
Image correction means for correcting the image data with the tone curve generated by the tone curve generation means to obtain output image data,
The tone curve generation means corresponds to the tone curve when the average luminance point of the specific target area image data calculated from the specific target area luminance histogram is used as an input value when an initial tone curve is given in advance. The image processing apparatus is characterized in that a new tone curve is generated by correcting the initial tone curve so that the output value becomes the median value of the output value range width of the initial tone curve.

According to a second aspect of the present invention, in the image processing apparatus according to the first aspect,
Furthermore, it has an overall luminance histogram calculation means for calculating a histogram relating to the luminance of the entire image data having luminance information as an overall luminance histogram,
The tone curve generation means corrects the initial tone curve, and then divides the entire luminance histogram into two luminance regions, a high luminance side and a low luminance side, with an average luminance point of the specific target region image data as a boundary. In addition, the corrected tone curve is divided into two tone curves corresponding to the two luminance regions with the average luminance point of the specific target region image data and the median value of the output value range width as a boundary. ,
On the tone curve corresponding to each luminance area when the average luminance point of the whole luminance histogram in each divided luminance area is calculated and the average luminance point of the whole luminance histogram in each luminance area is used as the input value Each tone curve is corrected to generate a new tone curve so that the corresponding output value becomes the median value of the output value range of each tone curve.

According to a third aspect of the present invention, in the image processing apparatus according to the second aspect,
The tone curve generating means includes
The entire luminance histogram in each luminance region is further divided into two luminance regions, a high luminance side and a low luminance side, with the average luminance point calculated from the entire luminance histogram in each luminance region as a boundary, and each luminance region Each tone curve is further divided on the basis of the average luminance point calculated from the overall luminance histogram and the median value of the output value range width of each tone curve,
On the tone curve corresponding to each luminance area when the average luminance point of the whole luminance histogram in each divided luminance area is calculated and the average luminance point of the whole luminance histogram in each luminance area is used as the input value Each tone curve is corrected to generate a new tone curve so that the corresponding output value becomes the median value of the output value range of each tone curve.

According to a fourth aspect of the present invention, in the image processing apparatus according to the third aspect,
The tone curve generating means includes
The luminance area and tone curve dividing process and the tone curve correcting process are repeated a predetermined number of times to generate a new tone curve.

The invention according to claim 5 is the image processing apparatus according to claim 3,
The tone curve generating means includes
The luminance area and tone curve dividing process and the tone curve correcting process are repeated until the luminance area cannot be divided by an integer value to generate a new tone curve.

According to a sixth aspect of the present invention, in the image processing apparatus according to the third aspect,
The tone curve generating means includes
When a certain luminance area cannot be divided by an integer value, only the luminance area that can be divided by an integer value is subjected to the dividing process of the luminance area and the tone curve, and the correction process of the tone curve, and a new It is characterized by generating a tone curve.

  The invention according to claim 7 is the image processing apparatus according to any one of claims 1 to 6, wherein the specific target area detecting unit detects a human face area as the specific target area. The specific target area luminance histogram calculating means calculates the luminance histogram of the image data of the human face area as the specific target area image data as the specific target area luminance histogram.

  The invention according to claim 8 is the image processing apparatus according to claim 7, wherein when there are a plurality of human face areas detected by the specific target area detecting means, the human face area is calculated from the specific target area luminance histogram. The average luminance point of the specific target area image data is a luminance center of the detected plurality of face areas.

  The invention according to claim 9 is the image processing apparatus according to claim 7, wherein when there are a plurality of human face areas detected by the specific target area detecting means, the detected face areas A priority order is set, and the average luminance point of the specific target area image data calculated from the specific target area luminance histogram is the luminance center of the face area having the highest priority.

  According to a tenth aspect of the present invention, in the image processing apparatus according to the ninth aspect, the priority is set based on a size of a face area or a distance to the face.

The invention according to claim 11 is a specific target region detecting step of detecting a specific target region from the entire image data having luminance information,
A specific target region luminance histogram calculating step for calculating a histogram relating to the luminance of the specific target region image data detected in the specific target region detection step among the entire image data having luminance information;
A tone curve generation process for generating a tone curve;
An image correction step of correcting the image data with the tone curve generated by the tone curve generation step to obtain output image data,
In the tone curve generation step, when an initial tone curve is given in advance, the corresponding output on the tone curve when the average luminance point of the specific target area image data calculated from the specific target area luminance histogram is used as an input value The image processing method is characterized in that a new tone curve is generated by correcting the initial tone curve so that the value becomes the median value of the output value range width of the initial tone curve.

The invention according to claim 12 is the image processing method according to claim 12,
Furthermore, it has an overall luminance histogram calculation step for calculating a histogram relating to the luminance of the entire image data having luminance information as an overall luminance histogram,
In the tone curve generation step, after correcting the initial tone curve, the entire luminance histogram is divided into two luminance regions, a high luminance side and a low luminance side, with the average luminance point of the specific target region image data as a boundary. In addition, the corrected tone curve is divided into two tone curves corresponding to the two luminance regions with the average luminance point of the specific target region image data and the median value of the output value range width as a boundary. ,
On the tone curve corresponding to each luminance area when the average luminance point of the whole luminance histogram in each divided luminance area is calculated and the average luminance point of the whole luminance histogram in each luminance area is used as the input value Each tone curve is corrected to generate a new tone curve so that the corresponding output value becomes the median value of the output value range of each tone curve.

The invention according to claim 13 is the image processing method according to claim 12,
In the tone curve generation step,
The entire luminance histogram in each luminance region is further divided into two luminance regions, a high luminance side and a low luminance side, with the average luminance point calculated from the entire luminance histogram in each luminance region as a boundary, and each luminance region Each tone curve is further divided on the basis of the average luminance point calculated from the overall luminance histogram and the median value of the output value range width of each tone curve,
On the tone curve corresponding to each luminance area when the average luminance point of the whole luminance histogram in each divided luminance area is calculated and the average luminance point of the whole luminance histogram in each luminance area is used as the input value Each tone curve is corrected to generate a new tone curve so that the corresponding output value becomes the median value of the output value range of each tone curve.

The invention according to claim 14 is the image processing method according to claim 13,
In the tone curve generation step,
The luminance area and tone curve dividing process and the tone curve correcting process are repeated a predetermined number of times to generate a new tone curve.

The invention described in claim 15 is the image processing method described in claim 13,
In the tone curve generation step,
The luminance area and tone curve dividing process, and the tone curve correcting process,
The process is repeated until it becomes impossible to divide the luminance area with integer values, and a new tone curve is generated.

The invention described in claim 16 is the image processing method described in claim 13,
In the tone curve generation step,
When a certain luminance area cannot be divided by an integer value, only the luminance area that can be divided by an integer value is subjected to the dividing process of the luminance area and the tone curve, and the correction process of the tone curve, and a new It is characterized by generating a tone curve.

  The invention according to claim 17 is an image processing program for causing a computer to execute the processing in the image processing method according to any one of claims 11 to 16.

  The invention according to claim 18 is an image pickup apparatus using the image processing apparatus according to any one of claims 1 to 10.

According to the present invention, the specific target area detecting means for detecting the specific target area from the entire image data having luminance information,
Specific target area luminance histogram calculating means for calculating a histogram relating to the luminance of the specific target area image data detected by the specific target area detection means among the entire image data having luminance information;
A tone curve generating means for generating a tone curve;
Image correction means for correcting the image data with the tone curve generated by the tone curve generation means to obtain output image data,
The tone curve generation means corresponds to the tone curve when the average luminance point of the specific target area image data calculated from the specific target area luminance histogram is used as an input value when an initial tone curve is given in advance. The initial tone curve is corrected so that the output value becomes the median value of the output value range of the initial tone curve, and a new tone curve is generated. Even if it is not prepared in advance, the brightness and gradation of a specific target area (for example, a human face) in an image can be optimized according to various shooting situations.

Furthermore, according to the present invention, there is further provided an overall luminance histogram calculating means for calculating a histogram relating to the luminance of the entire image data having luminance information as an overall luminance histogram,
The tone curve generation means corrects the initial tone curve, and then divides the entire luminance histogram into two luminance regions, a high luminance side and a low luminance side, with an average luminance point of the specific target region image data as a boundary. In addition, the corrected tone curve is divided into two tone curves corresponding to the two luminance regions with the average luminance point of the specific target region image data and the median value of the output value range width as a boundary. ,
On the tone curve corresponding to each luminance area when the average luminance point of the whole luminance histogram in each divided luminance area is calculated and the average luminance point of the whole luminance histogram in each luminance area is used as the input value Each tone curve is corrected so that the corresponding output value becomes the median of the output value range width of each tone curve, and a new tone curve is generated. Even without preparing a curve in advance, the brightness and gradation of a specific target area (for example, a human face) in an image can be optimized according to various shooting conditions, and image data A tone curve suitable for the overall brightness and gradation can be created (corrected).

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings. The present invention relates to image processing applied to an imaging apparatus such as a digital camera having a tone correction function for image data. In the following description, for convenience of explanation, the present invention is described as being applied to a digital camera (digital still camera). That is, description will be made assuming that the imaging device is a digital camera (digital still camera).

  According to the present invention, by controlling one tone curve, the brightness and gradation of a specific target region (for example, a human face) in an image are optimized according to various shooting situations. An image processing apparatus, an image processing method, an image processing program, and an imaging apparatus are provided.

  In this case, in the present invention, one tone curve is partially corrected according to a specific target region (for example, a human face) in the image and the entire image and used for gradation correction. Even if the feature histogram and tone curve are not prepared in advance, the brightness and gradation of a specific target area (for example, a human face) in an image can be optimized according to various shooting conditions. it can.

  Therefore, the present invention is particularly suitable for application to an imaging apparatus such as a digital camera having a function of correcting the gradation of image data. In the following description, for convenience of explanation, the present invention is described as being applied to a digital camera (digital still camera). That is, description will be made assuming that the imaging device is a digital camera (digital still camera).

  FIG. 1 is an external view of a digital still camera as an imaging apparatus to which the image processing of the present invention is applied, and FIG. 2 is a diagram illustrating a hardware configuration example of the digital still camera.

  First, an outline of the operation of the digital still camera will be described.

  1 is set to the shooting mode and the power button is pressed, the camera is activated in the recording mode. The mode dial and the power button are included in the operation unit in FIG. 2. When the CPU detects that the mode dial is in the shooting mode and the power button is turned on, the CPU controls the motor driver, Move the lens barrel unit to a position where it can be photographed. Furthermore, power is turned on to each part such as an image sensor and an LCD display to start operation.

  When the power of each part is turned on, the finder mode operation starts.

  In the finder mode, light incident on the image sensor (CCD, CMOS) through the lens is converted into electricity and sent to the A / D converter. The A / D converter converts the input analog signal into 12-bit digital data (digital signal). Each signal converted into a digital signal is taken into the sensor I / F unit in the digital signal processing IC, converted into a YUV signal that can be displayed by the YUV conversion unit, and written to the frame memory by the memory controller. It is. The YUV signal is read by the memory controller and sent to the TV or LCD monitor via the display output control unit for display. This process is performed at 1/30 second intervals, and a finder mode display updated every 1/30 seconds is obtained.

  This image sensor is an 8 million pixel sensor that can output all pixels every 1/30 seconds during still image shooting. In the viewfinder mode, the surrounding pixels are combined with decimation and thinning out for 1/30 seconds. For example, the number of pixels is reduced to 640 × 480 pixels for output.

  The sensor I / F of the digital signal processing IC detects an AF evaluation value indicating the degree of focusing on the screen, an AE evaluation value that detects the subject brightness, and a subject color from the digital RGB signal captured in the block. The calculated AWB evaluation value is calculated. These data are read out as feature data to the CPU and used for each of AE, AF, and AWB processes.

  The AF evaluation value is created by, for example, an output integrated value of a high frequency component extraction filter or an integrated value of a luminance difference between adjacent pixels. When in the in-focus state, the edge portion of the subject is clear, so the high frequency component is the highest. By utilizing this, at the time of the focus detection operation by AF, the AF evaluation value at each focus lens position is acquired, and AF is executed with the point where the maximum is obtained as the focus position.

  The AE evaluation value and the AWB evaluation value are created from the respective RGB integration values. For example, the screen is equally divided into 256 areas (16 horizontal divisions and 16 vertical divisions), and the RGB integration of each area is calculated. The CPU reads the RGB integration values, and in AE, the brightness of each area is calculated, and an appropriate exposure time is determined from the brightness distribution. In AWB, an AWB control value that matches the color of the light source is determined from the RGB distribution. The AE and AWB processes are continuously performed during the finder mode.

  When the release shutter button of FIG. 1 is operated, an AF operation that is a focus position detection and a still image recording process are performed.

  That is, when the release shutter button is pressed, a still image shooting start signal is taken into the CPU from the camera operation unit of FIG. 2, and the CPU drives the lens via the motor driver in synchronization with the frame rate, thereby climbing the mountain. Execute AF. When the AF target range is the entire region from infinity to close, the focus lens moves to each focus position from close to infinity, or from infinity to close, and each frame (= each) created by the digital signal processing IC The CPU reads the AF evaluation value at the (focus position). The focus lens is moved to the in-focus position with the point at which the AF evaluation value at each focus position is maximized as the in-focus position.

  After the AF is completed, the image is converted into a digital RGB signal by the image sensor and output, and stored in the frame memory via the digital signal processing IC. The digital RGB signal is read again into the digital signal processing IC, converted into YUV data by the YUV converter, and written back to the frame memory.

  At the time of capturing a still image, YUV converted image data is sent to an image compression / decompression circuit in the digital signal processing IC. The YUV data sent to the image compression / decompression circuit is compressed and written back to the frame memory. The compressed data in the frame memory is read through the digital signal processing IC and stored in the data storage memory.

FIG. 3 is a diagram showing a configuration example of the image processing apparatus of the present invention. Referring to FIG. 3, the image processing apparatus of the present invention includes an image size conversion means 21 that converts the size of input image data when the image data is input,
An overall brightness histogram calculating means 22 for calculating a histogram relating to the brightness of the entire image data whose size has been converted by the image size converting means 21 as an overall brightness histogram;
A specific target region detecting unit 27 for detecting a specific target region from the entire image data whose size has been converted by the image size converting unit 21;
A specific target area luminance histogram for calculating a histogram related to the luminance of the specific target area image data detected by the specific target area detection means from among the entire image data whose size has been converted by the image size conversion means 21. Calculating means 28;
Tone curve generating means 23 for generating a tone curve from a predetermined luminance histogram;
An image correction unit (tone map processing unit) 24 that outputs image data by correcting the image data with the tone curve generated by the tone curve generation unit 23 is provided.

  Here, the tone curve is an input / output conversion characteristic (function) when the horizontal axis is input image information and the vertical axis is output image information. That is, the tone curve is defined as a function for converting the luminance value of each input image into the luminance value of the corresponding output image. Thus, obtaining the output image for the input image by applying the tone curve (or performing a predetermined correction) is generally called tone mapping (tone map processing).

  Further, when the input image data has a sufficiently small size, the image size conversion means 21 does not function the input image data, and converts the input image data into the specific target area detection means 27, the specific target area luminance histogram. It can also be directly given to the calculation means 28 and the overall luminance histogram calculation means 22. That is, in this case, the image size conversion means 21 is not necessary, the specific target area detection means 27 detects the specific target area directly from the input image data, and the specific target area luminance histogram calculation means 28 directly The specific target area luminance histogram related to the specific target area luminance of the input image data is calculated. Further, the overall luminance histogram calculation means 22 can directly calculate the total luminance histogram related to the luminance of the entire input image data.

  However, in actuality, the input image data (captured image data) is of a large size, so that the amount of calculation when obtaining a luminance histogram and the like is reduced, and the overall processing is further accelerated. In general, instead of the luminance data of the input image itself, the size of the input image data is reduced by the image size conversion means 21, and a luminance histogram of a reduced image such as a display device monitoring (through) image is used. Even if the reduced image is used, the tendency of the input image such as the luminance histogram is the same, so there is no significant difference in the required tone curve (corrected according to the present invention), and tone mapping (image conversion / correction using the tone curve). In the case of implementation), the original size (large size) image is input to the corrected tone curve and converted into an output image, so there is no deterioration in image quality, and the effect of speeding up is greater.

  Further, the specific target area detecting means 27 detects, for example, a human face area as the specific target area.

  As a method of detecting a person's face area from a subject image, the technique shown in the following document is known, and the specific target area detecting means 27 uses any of the methods shown in the following document. Thus, the face area of the person can be detected.

  That is, the color as shown in the document “Proposal of a modified HSV color system effective for face area extraction” in the TV Society Journal Vol. 49, No. 6, pp. 787-797 (1995). A person's face area can be detected using a method of extracting an image from a mosaic image and extracting the face area by focusing on the skin color area.

  Further, it is described in the document “Method for extracting a face region from a still gray scene image” in the Journal of the Institute of Electronics, Information and Communication Engineers Vol. 74-D-II, No. 11, pp. 1625-1627 (1991). Using the method of extracting the head area of the front person using the geometric shape features of each part constituting the head of the front person image such as hair, eyes and mouth, Can be detected.

  In the case of a moving image as shown in the document “Image Face Detection for Videophone and its Effect” in Image Lab 1991-11 (1991), a human image generated by a subtle movement of a person between frames is used. The face area of a person can be detected by using a method of extracting a front person image using a contour edge.

  By the way, in the present invention, the tone curve generating means 23 generates an optimum tone curve using a luminance histogram for each image data. Specifically, the tone curve generating means 23 generates an optimum tone curve using an average value of luminance obtained from the luminance histogram. That is, the tone curve generating means 23 sets the tone value before correction so that the corresponding output value on the tone curve when the average value of luminance obtained from a predetermined luminance histogram is input is set to the optimum gradation output range. The curve is corrected to generate a new tone curve, and the optimal tone for each image is obtained by performing image (gradation) correction using the new tone curve generated in this way. Correction can be performed.

  Hereinafter, the tone curve generating means 23 of the present invention will be described in detail. In the following description, it is assumed that the specific target area detection unit 27 detects a human face area as the specific target area. That is, it is assumed that the specific target area is a human face area.

  In the first embodiment of the present invention, the tone curve generating means 23 is configured to specify a specific target region luminance histogram (a luminance histogram of image data of a human face region (a face region luminance histogram) when an initial tone curve is given in advance. )), The corresponding output value on the tone curve when the average luminance point of the specific target area image data (image data of the human face area) calculated from (2) is used as the input value is the output value range width of the initial tone curve. The initial tone curve is corrected to a median value, and a new tone curve is generated.

  4, 5, 6 and 7 are diagrams for explaining the processing operation of the tone curve generating means 23 in the first embodiment of the present invention.

  FIG. 4 is a diagram illustrating an example of image data. That is, FIG. 4 is an example of an image in which a person is shown, and a rectangle F in the face portion indicates that a face is detected in that range. The brightness histogram of the face area (face area brightness histogram) is created for the rectangular area F. 5 is a diagram illustrating an example of the specific target region luminance histogram (luminance histogram of the image data of the human face region F (face region luminance histogram)) of the image data of FIG. 4, and FIG. 6 is a diagram of the image data of FIG. FIG. 6 is a diagram for explaining a new tone curve generation process in a case where the specific target region luminance histogram is FIG.

  It is assumed that the initial tone curve is given in advance as indicated by reference numeral 31 in FIG. Further, assuming that the average luminance point of the image data of the face area F calculated from the specific target area luminance histogram of FIG. 5 is indicated by the symbol A0 in FIG. 6, the tone curve generating means 23 is shown in FIG. Thus, the corresponding output value on the tone curve when the average luminance point A0 of the image data of the face area F calculated from the specific target area luminance histogram of FIG. The initial tone curve 31 is corrected so as to be the median value H0 of the band width (output luminance value range width) y0, and a new tone curve 32 is generated.

  That is, when the brightness histogram (face area brightness histogram) of the image data of the face area F is as in the example shown in FIG. 5, the image data of the face area F is an overall dark image, and the face area brightness histogram is It can be seen that it is close to the dark part (left).

  In the example of FIG. 6, the tone curve 31 when the average luminance point A0 of the image data of the face region F obtained from the luminance histogram (face region luminance histogram) of the image data of the face region F is set as the input position on the tone curve 31. A new tone consisting of two broken lines (tone curves) 33 and 34 is obtained by bringing the corresponding point above to the median value H0 of the output value range width (output brightness value range width) y0 of the tone curve 31. A curve 32 is generated. Here, the initial tone curve 31 is a basic tone curve with linear input / output characteristics, and the correction arrangement point when the luminance average of the image data of the face region F is set as the input point is P1, so that two A new tone curve 32 composed of broken lines (tone curves) 33 and 34 is generated.

  When the image is corrected by applying the new tone curve 32 generated as described above, the specific target region luminance histogram (luminance histogram of the image data of the human face region F (face region luminance histogram)) is shown in FIG. It becomes like this. Compared with FIG. 5 which is the specific target region luminance histogram (luminance histogram of the image data of the human face region F (face region luminance histogram)) before correction, the shape of the entire specific target region luminance histogram (face region luminance histogram) is You can see that it is moving in the bright direction. Therefore, the dark face area image is brightly corrected so as to be closer to the center of the gradation expression area.

  As described above, in the first embodiment of the present invention, the brightness and gradation of the image data of the face region F can be changed according to various shooting situations without preparing a plurality of sets of feature histograms and tone curves in advance. It can be optimized (by controlling the tone curve around the person's face, the person's face is set to the optimal brightness, and the gradation of the person's face is optimized. Is possible).

In the second embodiment of the present invention, the tone curve generating unit 23 further includes an overall luminance histogram calculating unit 22 that calculates a histogram related to the luminance of the entire image data having luminance information as an overall luminance histogram,
The tone curve generation unit 23 corrects the initial tone curve, and then converts the entire luminance histogram into two luminance regions, a high luminance side and a low luminance side, with the average luminance point of the specific target region image data as a boundary. And dividing the corrected tone curve into two tone curves corresponding to the two luminance regions with the average luminance point of the specific target region image data and the median value of the output value range width as a boundary. And
On the tone curve corresponding to each luminance area when the average luminance point of the whole luminance histogram in each divided luminance area is calculated and the average luminance point of the whole luminance histogram in each luminance area is used as the input value Each tone curve is corrected to generate a new tone curve so that the corresponding output value becomes the median value of the output value range of each tone curve.

  8, 9, and 10 are diagrams for explaining the processing operation of the tone curve generating means 23 in the second embodiment of the present invention. FIG. 8 is a diagram showing an example of a luminance histogram (overall luminance histogram) of the entire image in FIG. FIG. 9 shows the new tone curve 32 of FIG. 6, and FIG. 10 explains the process of further correcting the new tone curve 32 of FIG. 6 (FIG. 9) to generate a new tone curve 35. FIG.

Referring to FIGS. 9 and 10, the entire luminance histogram of FIG. 8 is set to the high luminance with the average luminance point A0 of the image data of the face area F calculated from the specific target area luminance histogram (face area luminance histogram) of FIG. Is divided into two luminance regions (input luminance regions) x1 and x2 on the low-luminance side and the low-luminance side, and the correction is performed with the median value H0 of the average luminance point A0 and the output value range width y0 as a boundary The tone curve 32 is divided into two tone curves 33 and 34 corresponding to the two luminance regions (input luminance regions) x1 and x2.
The average luminance points A1 and A2 of the overall luminance histogram in each of the divided luminance areas (input luminance areas) x1 and x2 are calculated, and the average luminance points of the entire luminance histogram in the respective luminance areas (input luminance areas) x1 and x2 are calculated. When A1 and A2 are input values, the corresponding output values on the tone curves 33 and 34 corresponding to the luminance regions (respective input luminance regions) x1 and x2 are the output values of the tone curves 33 and 34, respectively. The tone curves 33 and 34 are corrected so that the median values H1 and H2 of the domain widths y1 and y2 are obtained, and a new tone curve 35 is generated.

  That is, after the processing described in FIG. 6 is performed, the average brightness A0 of the image data of the face area F calculated from the specific target area brightness histogram (face area brightness histogram) in FIG. In this way, the image data is divided into a dark area x1 and a bright area x2. Further, in each divided area (each input brightness area) x1, x2, average brightness points A1, A2 of the overall brightness histogram for each area (input brightness area) x1, x2 are calculated. The obtained average luminance points A1 and A2 for the regions (input luminance regions) x1 and x2 are used as input points for the tone curves 33 and 34 for the regions (input luminance regions) x1 and x2, respectively. The tone curve 33 for each region (input luminance region) x1 and x2 is arranged so that the corresponding output point above is arranged at the median values H1 and H2 of the output value range width (output luminance value range width) y1 and y2. , 34 are corrected to tone curves 35 (36, 37, 38, 39) as shown in FIG.

  In the second mode of the present invention, by performing such processing, the image data of the face region F can be obtained in accordance with various shooting situations without preparing a plurality of sets of feature histograms and tone curves in advance. The brightness and gradation of the person's face can be optimized (by controlling the tone curve around the person's face, the person's face can have the optimum brightness, and the person's face In addition, it is possible to optimize (tone) and to create (correct) a tone curve suitable for the brightness and gradation of the entire image data.

In the third embodiment of the present invention, the tone curve generating means 23 further includes the luminance regions (where each luminance region (each input luminance region) is calculated with the average luminance point calculated from the entire luminance histogram as a boundary. The whole luminance histogram in each input luminance region) is further divided into two luminance regions (input luminance regions) on the high luminance side and low luminance side, and the whole luminance histogram in each luminance region (each input luminance region). Each tone curve is further divided with the average luminance point calculated from the above and the median of the output value range width of each tone curve as a boundary.
When the average luminance point of the whole luminance histogram in each divided luminance area (input luminance area) is calculated, and the average luminance point of the whole luminance histogram in each luminance area (input luminance area) is used as the input value, Correct each tone curve so that the corresponding output value on each tone curve corresponding to the brightness area (each input brightness area) becomes the median value of the output value range width of each tone curve, It is characterized by generating a simple tone curve.

  FIG. 11 is a diagram for explaining the above processing operation of the tone curve generating means 23 in the third mode of the present invention. FIG. 11 is a diagram for explaining a process of generating a new tone curve 40 by further correcting the new tone curve 35 (36, 37, 38, 39) of FIG.

Referring to FIG. 11, the tone curve generating means 23 uses the luminance regions (the above-mentioned luminance regions (the input luminance regions) x1 and x2) from the average luminance points A1 and A2 calculated from the overall luminance histogram. The entire luminance histogram in each input luminance region) x1, x2 is further divided into two luminance regions (input luminance regions) x3, x4, x5, x6 on the high luminance side and low luminance side, Each tone with the average luminance points A1 and A2 calculated from the overall luminance histogram in each input luminance region x1 and x2 and the median values H1 and H2 of the output value range widths y1 and y2 of the tone curves 33 and 34 as boundaries. The curve 35 is further divided (36, 37, 38, 39),
Average luminance points A3, A4, A5, and A6 of the overall luminance histogram in each of the divided luminance areas (input luminance areas) x3, x4, x5, and x6 are calculated, and the respective luminance areas (input luminance areas) x3, x4 are calculated. , X5, x6, each tone curve corresponding to each luminance region (each input luminance region) x3, x4, x5, x6 when the average luminance points A3, A4, A5, A6 of the whole luminance histogram are input values. The corresponding output values on 36, 37, 38, 39 are the median values H3, H4, H5, H6 of the output value range widths y3, y4, y5, y6 of the respective tone curves 36, 37, 38, 39. Similarly, each tone curve 36, 37, 38, 39 is corrected to generate a new tone curve 40.

  By repeating the above processing until the luminance output values of the tone curves corresponding to the integer input set points on all luminance axes are rearranged, the image data of the face area F is optimal and the entire image data is also considered. A tone curve for optimizing the brightness and gradation of the image can be finally generated.

  FIG. 12 is a flowchart showing a specific example of the processing of the tone curve generating means 23 in the first, second and third embodiments of the present invention.

  Referring to FIG. 12, first, face detection processing is performed on an image captured by the imaging optical system (an image as illustrated in FIG. 4) to obtain information on the face area F, and then the image is illustrated in FIG. 5. A brightness histogram (face area brightness histogram) of the face area F is calculated (S101).

  Next, the average value of the brightness histogram (face area brightness histogram) of the face area shown in FIG. 5 is calculated, and the input is the average value of the brightness histogram of the face area (face area brightness histogram) in the tone curve shown in FIG. If there is, the tone curve is corrected so that the output value becomes the center value (for example, 128 if the output is 8 bits) (S102). Then, the whole luminance histogram shown in FIG. 8 is divided into a bright part (x2) and a dark part (x1) with the point set in S101 and S102 as a boundary (S103).

  Next, for the divided overall luminance histogram, the average values of the whole luminance histograms on the low luminance side and the high luminance side are calculated in the divided area (S104). When the input is the average value of the histogram calculated in S105, the tone curve is corrected so that the output value becomes the center value in the region divided in S103 (S105). Then, the whole luminance histogram (divided histogram) is further divided into a bright part and a dark part with the average value of the whole luminance histograms on the low luminance side and high luminance side calculated in S104 as a boundary (S106).

  After that, the same processing is sequentially repeated for each of the divided overall luminance histogram regions on the low luminance side and the high luminance side (S104 to S106 are repeated), and the division for the resolution of the input value is completed. At the time (S107), the tone curve generation process ends.

  By performing tone map conversion of the image using the tone curve calculated by the above processing, the human face portion detected in the image by the first tone curve correction (S102) becomes appropriate brightness. Further, in a portion other than the face region, a tone curve in which many gradations are assigned to a main portion in an image having a large frequency of the entire luminance histogram by the second and subsequent tone curve corrections (S105). Therefore, correction is performed so that the gradation in the image is optimized.

  In this manner, after performing face detection processing on the image captured by the imaging optical system to obtain face area information, processing for generating a tone curve for optimizing the brightness and gradation of the image is performed. Can be done.

  The tone curve generation processing described above is ideally performed until the luminance output values of the tone curves corresponding to the integer input set points on all luminance axes are rearranged, but the processing load and processing In consideration of time or the like, the processing may be interrupted at a predetermined number of times or a predetermined number of divisions as appropriate, and image correction using the tone curve calculated at that time may be performed.

That is, the tone curve generation means 23
The luminance area (input luminance area) and tone curve dividing process and the tone curve correcting process are repeated a predetermined number of times to generate a new tone curve.

Alternatively, the tone curve generating means 23
The brightness area (input brightness area) and tone curve dividing process, and the tone curve correction process are repeated until the brightness area (input brightness area) cannot be divided by an integer value, and a new tone curve is obtained. Can be generated.

Alternatively, the tone curve generating means 23
When a certain luminance area (input luminance area) cannot be divided by an integer value, the luminance area (input luminance area) and tone curve are divided only for the luminance area (input luminance area) that can be divided by an integer value. A new tone curve can be generated by performing the processing and the tone curve correction processing.

  Specifically, for example, in the process shown in FIG. 11, if the average luminance points A1 and A0 of the luminance histogram are adjacent to each other, for example, and the region (input luminance region) x4 can no longer be divided by integer values, At this stage, tone curve generation processing is terminated, or tone curve generation processing (correction processing) is continued only for regions (input luminance regions) x3, x5, and x6 other than region (input luminance region) x4. be able to.

  Further, in the above-described example, the face area of the person detected by the specific target area detection unit 27 is one, but when there are a plurality of human face areas detected by the specific target area detection unit 27, The average luminance point of the specific target area image data calculated from the specific target area luminance histogram (face area luminance histogram) can be set as the luminance center of the plurality of detected face areas.

  Alternatively, when there are a plurality of face areas of the person detected by the specific target area detection means 27, priority is set for the detected plurality of face areas, and calculation is performed from the specific target area luminance histogram (face area luminance histogram). The average luminance point of the specific target area image data to be processed can be set as the luminance center of the face area having the highest priority. Here, the priority order is set based on the size of the face area or the distance to the face, for example. As described above, when using the luminance histogram information of the face of the most important person among the detected faces, make the face of the most important person have an optimal brightness and the face portion of the main person. Can be optimized.

  In addition, the present invention can be realized without mounting special hardware as will be described later, and the amount of calculation is small compared with a method of obtaining a tone curve for each region and correcting each to synthesize the corrected image. Fast enough.

  As described above, the image processing apparatus of the present invention is realized by the CPU in the example of FIG.

  The processing in the image processing apparatus and the image processing method described in the above embodiments for carrying out the present invention is provided in the form of a program (image processing program) that is realized by the CPU in the example of FIG. Can do.

  The program for causing the CPU to execute the processing in the image processing apparatus described in the above-described best mode for carrying out the present invention is, for example, a hard disk, floppy (registered trademark) disk, CD-ROM, MO, DVD, etc. Are recorded on a computer-readable recording medium and executed by being read from the recording medium by the computer. Further, this program can be distributed via a network such as the Internet through the recording medium.

In the above description, the imaging apparatus to which the image processing of the present invention is applied is described as a digital camera. However, the imaging apparatus to which the image processing of the present invention is applied is not necessarily limited to a digital camera.

1 is an external view of a digital still camera as an imaging apparatus to which image processing of the present invention is applied. It is a figure which shows the hardware structural example of a digital still camera. It is a figure which shows the structural example of the image processing apparatus of this invention. It is a figure which shows an example of image data. FIG. 5 is a diagram illustrating an example of a specific target region luminance histogram (luminance histogram of image data of a human face region F (face region luminance histogram)) of the image data in FIG. 4; It is a figure for demonstrating the processing operation of the tone curve production | generation means in the 1st form of this invention. It is a figure which shows a brightness | luminance histogram when an image is correct | amended by applying the new tone curve of FIG. 6 produced | generated by the tone curve production | generation means of this invention. It is a figure which shows the example of the brightness | luminance histogram (overall brightness histogram) of the whole image of FIG. It is a figure for demonstrating the processing operation of the tone curve production | generation means in the 2nd form of this invention. It is a figure for demonstrating the processing operation of the tone curve production | generation means in the 2nd form of this invention. It is a figure for demonstrating the processing operation of the tone curve production | generation means in the 3rd form of this invention. It is a flowchart which shows the specific example of the tone curve production | generation process of this invention.

Explanation of symbols

21 Image size conversion means 22 Luminance histogram calculation means 23 Tone curve generation means 24 Image correction means (tone map processing means)
27 Specific target area detection means 28 Specific target area luminance histogram calculation means

Claims (18)

Specific target area detecting means for detecting a specific target area from the entire image data having luminance information;
Specific target area luminance histogram calculating means for calculating a histogram relating to the luminance of the specific target area image data detected by the specific target area detection means among the entire image data having luminance information;
A tone curve generating means for generating a tone curve;
Image correction means for correcting the image data with the tone curve generated by the tone curve generation means to obtain output image data,
The tone curve generation means corresponds to the tone curve when the average luminance point of the specific target area image data calculated from the specific target area luminance histogram is used as an input value when an initial tone curve is given in advance. An image processing apparatus for generating a new tone curve by correcting the initial tone curve so that an output value becomes a median of an output value range width of the initial tone curve. The image processing apparatus according to claim 1.
Furthermore, it has an overall luminance histogram calculation means for calculating a histogram relating to the luminance of the entire image data having luminance information as an overall luminance histogram,
The tone curve generation means corrects the initial tone curve, and then divides the entire luminance histogram into two luminance regions, a high luminance side and a low luminance side, with an average luminance point of the specific target region image data as a boundary. In addition, the corrected tone curve is divided into two tone curves corresponding to the two luminance regions with the average luminance point of the specific target region image data and the median value of the output value range width as a boundary. ,
On the tone curve corresponding to each luminance area when the average luminance point of the whole luminance histogram in each divided luminance area is calculated and the average luminance point of the whole luminance histogram in each luminance area is used as the input value A new tone curve is generated by correcting each tone curve so that the corresponding output value becomes the median value of the output value range width of each tone curve. The image processing apparatus according to claim 2.
The tone curve generating means includes
The entire luminance histogram in each luminance region is further divided into two luminance regions, a high luminance side and a low luminance side, with the average luminance point calculated from the entire luminance histogram in each luminance region as a boundary, and each luminance region Each tone curve is further divided on the basis of the average luminance point calculated from the overall luminance histogram and the median value of the output value range width of each tone curve,
On the tone curve corresponding to each luminance area when the average luminance point of the whole luminance histogram in each divided luminance area is calculated and the average luminance point of the whole luminance histogram in each luminance area is used as the input value A new tone curve is generated by correcting each tone curve so that the corresponding output value becomes the median value of the output value range width of each tone curve. The image processing apparatus according to claim 3.
The tone curve generating means includes
An image processing apparatus, wherein the luminance region and tone curve dividing process and the tone curve correcting process are repeated a predetermined number of times to generate a new tone curve. The image processing apparatus according to claim 3.
The tone curve generating means includes
Image processing characterized by generating a new tone curve by repeatedly performing the luminance area and tone curve dividing process and the tone curve correcting process until the luminance area cannot be divided by an integer value. apparatus. The image processing apparatus according to claim 3.
The tone curve generating means includes
When a certain luminance area cannot be divided by an integer value, only the luminance area that can be divided by an integer value is subjected to the dividing process of the luminance area and the tone curve, and the correction process of the tone curve, and a new An image processing apparatus that generates a tone curve. 7. The image processing apparatus according to claim 1, wherein the specific target area detecting unit detects a human face area as the specific target area. The luminance histogram calculating means calculates a luminance histogram of image data of a human face area as the specific target area image data as the specific target area luminance histogram. 8. The image processing apparatus according to claim 7, wherein the average luminance of the specific target area image data calculated from the specific target area luminance histogram when there are a plurality of human face areas detected by the specific target area detecting means. The image processing apparatus according to claim 1, wherein the point is a luminance center of the plurality of detected face areas. 8. The image processing apparatus according to claim 7, wherein when there are a plurality of human face areas detected by the specific target area detecting means, priority is set for the detected plurality of face areas, and the specific target area is set. The image processing apparatus according to claim 1, wherein the average luminance point of the specific target area image data calculated from the luminance histogram is a luminance center of the face area having the highest priority. The image processing apparatus according to claim 9, wherein the priority order is set based on a size of a face area or a distance to a face. A specific target area detecting step of detecting a specific target area from the entire image data having luminance information;
A specific target region luminance histogram calculating step for calculating a histogram relating to the luminance of the specific target region image data detected in the specific target region detection step among the entire image data having luminance information;
A tone curve generation process for generating a tone curve;
An image correction step of correcting the image data with the tone curve generated by the tone curve generation step to obtain output image data,
In the tone curve generation step, when an initial tone curve is given in advance, the corresponding output on the tone curve when the average luminance point of the specific target area image data calculated from the specific target area luminance histogram is used as an input value An image processing method, wherein a new tone curve is generated by correcting the initial tone curve so that a value becomes a median value of an output value range width of the initial tone curve. The image processing method according to claim 12.
Furthermore, it has an overall luminance histogram calculation step for calculating a histogram relating to the luminance of the entire image data having luminance information as an overall luminance histogram,
In the tone curve generation step, after correcting the initial tone curve, the entire luminance histogram is divided into two luminance regions, a high luminance side and a low luminance side, with the average luminance point of the specific target region image data as a boundary. In addition, the corrected tone curve is divided into two tone curves corresponding to the two luminance regions with the average luminance point of the specific target region image data and the median value of the output value range width as a boundary. ,
On the tone curve corresponding to each luminance area when the average luminance point of the whole luminance histogram in each divided luminance area is calculated and the average luminance point of the whole luminance histogram in each luminance area is used as the input value A new tone curve is generated by correcting each tone curve so that the corresponding output value becomes the median value of the output value range width of each tone curve. The image processing method according to claim 12.
In the tone curve generation step,
The entire luminance histogram in each luminance region is further divided into two luminance regions, a high luminance side and a low luminance side, with the average luminance point calculated from the entire luminance histogram in each luminance region as a boundary, and each luminance region Each tone curve is further divided on the basis of the average luminance point calculated from the overall luminance histogram and the median value of the output value range width of each tone curve,
On the tone curve corresponding to each luminance area when the average luminance point of the whole luminance histogram in each divided luminance area is calculated and the average luminance point of the whole luminance histogram in each luminance area is used as the input value A new tone curve is generated by correcting each tone curve so that the corresponding output value becomes the median value of the output value range width of each tone curve. The image processing method according to claim 13.
In the tone curve generation step,
An image processing method characterized in that the luminance region and tone curve dividing process and the tone curve correcting process are repeated a predetermined number of times to generate a new tone curve. The image processing method according to claim 13.
In the tone curve generation step,
The luminance area and tone curve dividing process, and the tone curve correcting process,
An image processing method, characterized in that a new tone curve is generated by repeatedly performing division of luminance areas with integer values. The image processing method according to claim 13.
In the tone curve generation step,
When a certain luminance area cannot be divided by an integer value, only the luminance area that can be divided by an integer value is subjected to the dividing process of the luminance area and the tone curve, and the correction process of the tone curve, and a new An image processing method characterized by generating a tone curve. An image processing program for causing a computer to realize processing in the image processing method according to any one of claims 11 to 16. An image pickup apparatus using the image processing apparatus according to any one of claims 1 to 10.

Images