EP0739571A1 - Camera couleur a gamme dynamique large utilisant un dispositif a transfert de charge et un filtre mosaique - Google Patents

Camera couleur a gamme dynamique large utilisant un dispositif a transfert de charge et un filtre mosaique

Info

Publication number
EP0739571A1
EP0739571A1 EP94907434A EP94907434A EP0739571A1 EP 0739571 A1 EP0739571 A1 EP 0739571A1 EP 94907434 A EP94907434 A EP 94907434A EP 94907434 A EP94907434 A EP 94907434A EP 0739571 A1 EP0739571 A1 EP 0739571A1
Authority
EP
European Patent Office
Prior art keywords
color
dynamic range
components
wide dynamic
imaging apparatus
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP94907434A
Other languages
German (de)
English (en)
Inventor
Ran Ginosar
Tamar Genossar
Ofra Zinaty
Noam Sorek
Daniel J. Kligler
Yehoshua Y. Zeevi
Arkadi Neyshtadt
Dov Avni
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
i Sight Inc
Original Assignee
i Sight Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by i Sight Inc filed Critical i Sight Inc
Publication of EP0739571A1 publication Critical patent/EP0739571A1/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
    • H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
    • H04N25/135—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on four or more different wavelength filter elements
    • H04N25/136—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on four or more different wavelength filter elements using complementary colours
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2209/00—Details of colour television systems
    • H04N2209/04—Picture signal generators
    • H04N2209/041—Picture signal generators using solid-state devices
    • H04N2209/042—Picture signal generators using solid-state devices having a single pick-up sensor
    • H04N2209/045—Picture signal generators using solid-state devices having a single pick-up sensor using mosaic colour filter
    • H04N2209/046—Colour interpolation to calculate the missing colour values

Definitions

  • This invention pertains to video imagery and more particularly to apparatuses and techniques for providing enhancement of video color images.
  • the present invention uses a four-color mosaic filter with a single chip CCD in conjunction with color wide dynamic range algorithms. It is also applicable, however, to other types of mosaic filters known in the art. Description of the Prior Art
  • video imaging apparatus including means for providing a plurality of video color images of a scene at different exposure levels using a single CCD chip, each color image being separated into several (e.g., four in the preferred embodiment) different components prior to sensing by the CCD chip by way of a multiple color mosaic filter in front of the CCD chip.
  • the pixel outputs are then decoded — subjected to specific mathematical operations by the processing electronics following the CCD output — to generate the video luminance and chrominance signals.
  • the present invention integrates the digital processing of the mosaic color CCD data with ADAPTIVE SENSITIVITYTM dynamic range enhancement. This integration provides for a substantial savings in total system processing hardware chip count and cost. It also permits better control of the color and detail production of the camera's video output.
  • the mosaic storage format also provides for a unique video image compression technique.
  • Figure 1 is a general block diagram of the present invention.
  • Figure 2 is a representative illustration of the data image elements, with the size of the data image elements exaggerated.
  • Figure 3 is a general block diagram of the long and short processing of the present invention.
  • Figure 4 is a block diagram of the color path of the present invention.
  • Figure 5 is a block diagram of the intensity path of the present invention.
  • Figure 6 is a block diagram of the look-up table processing of the present invention.
  • Figure 7 is a block diagram of the joint operations of the present invention.
  • Figure 8 is a block diagram of the differential color, intensity result block of the present invention.
  • Figure 9 is a block diagram of the color suppression factor block of the present invention.
  • Figure 10 is a block diagram of the color conversion block of the present invention.
  • FIG. 11 is a block diagram of the mosaic generation block of the present invention.
  • FIG. 1 is a block diagram of the apparatus 10 of the present invention.
  • Apparatus 10 includes a mosaic filter 12 which is bonded to the front of CCD 14 (preferably a single chip) , generally as part of the CCD integrated circuit manufacturing process.
  • CCD 14 charge output When the CCD 14 charge output is read out, the photoelectric charges from vertically adjacent sensor elements of CCD 14 are combined in the analog shift register (not shown) .
  • the on-chip addition gives rise to a, ⁇ , y and ⁇ elements as described below and as described in the Sony CCD 1992 Data Book (Sony part number ICX038AK) , as well as earlier editions.
  • the mosaic complementary additive color image comprises alternating first and second rows of image data elements 18 A — 18 D wherein first rows include alternating ⁇ and ⁇ data elements (18 A and 18 B , respectively) , and wherein the second rows include alternating ⁇ and S data elements (18 c and 18 D , respectively) .
  • Each mosaic element of filter 12 covers the sum of two adjacent pixel sensors of CCD 14 so that each pixel output of CCD 14 is representative of one of the above color combinations given for the various image data elements 18.
  • apparatus 10 includes four major functions as summarized below:
  • Each of the long/short exposure length processing functions outputs its point intensity information, obtained from the Y path processing, to four look-up tables (LUTs) . These tables determine the point intensity result of the two exposures, the normalized color weighting or color selection function and the saturation color suppression factor. This information serves the joint operation processing stage.
  • the four LUTs are programmable, thus enabling operation with different functions when necessary. In an alternative embodi ⁇ ment, these LUTs may be replaced by a programmable, piecewise linear (PWL) or other digital function generator.
  • PWL piecewise linear
  • Joint operations processing joins results produced by the long and short processing blocks, and results obtained from the functions implemented in the table processing LUTs, and evaluates the final output of the algorithm.
  • the processing is divided into: a. Color components and Y result calculation — evaluates the final result of the color components and the intensity of each pixel. b. Color suppression factor calculation — evaluates the color suppression factor for each pixel, based on both edges and saturation information. c. Color conversion processing — converts mosaic differential color space to RGB color space and produces RGB and Y/Cr/Cb outputs for each pixel.
  • Generate Mosaic processing converts RGB color space back to mosaic color space for each pixel.
  • the mosaic information generated enables economical hardware storage of processed images. This information can be retrieved and replayed through the algorithm — in Replay Mode — to produce RGB or Y/Cr/Cb output of the stored result.
  • apparatus 10 includes long/short processing as implemented by mosaic long exposure field block 20 and mosaic short exposure field block 22 which obtain, respectively, a long and a short exposure from CCD 14 in order to allow subse ⁇ quent processing by long exposure processing block 24 and short exposure processing block 26.
  • the terms "long” and “short” exposures are used here generally to denote two image inputs to apparatus 10.
  • long is used to mean an input with a higher exposure level
  • short a lower exposure level.
  • the higher exposure may be generated in several ways, including longer integration time, typically obtained by controlling the "electronic shutter” of the CCD chip; higher gain in the analog amplifiers preceding digitization; or a larger mechanical iris opening or other external gating means.
  • These two image inputs are usually generated by a single CCD chip, but may also be generated simultaneously by two separate, boresighted CCD chips, as disclosed in the aforementioned earlier applications.
  • the two inputs may be generated either sequentially (as in the case of the first method above — integration time control) or concurrently (by using two input channels with different gain levels) .
  • field memories are required at the input to apparatus 10 (in blocks 20 and 22) to synchronize the data coming from the two sequential fields or frames. These memories are not needed in concurrent modes, except for purposes of "freezing" the image for electronic, digital storage.
  • Switching logic incorporated in blocks 20 and 22 controls the data flow into and out of these field memories, depending on which mode (sequential or concurrent) is used. Of course, this implementation could be expanded to more than two exposure levels. Blocks 24 and 26 may typically be provided on separate processing chips or incorporated together in a single chip. The processing for each exposure is divided into two paths: l. Color path processing— handles color information for each pixel (see color path block 28 in Figure 3 and, in more detail, in Figure 4) ; and 2. Intensity (Y) path processing —handles intensity information for each pixel (see Y path block 30 in Figure 3 and, in more detail, in Figure 5) .
  • long/short exposure processing blocks 24, 26 include mosaic white balance block 32.
  • Mosaic white balance block 32 receives the following field of information from long/short exposure field blocks 20, 22:
  • the mosaic white balance block 32 After processing, the mosaic white balance block 32 outputs color-corrected data values:
  • Mosaic white balance block 32 contains mosaic color balance functions. These functions may typically be implemented as eight mosaic white balance LUTs (look-up tables) . That is, for each exposure there is a set of four LUTs, one for each mosaic data type: ⁇ , ⁇ , y, and ⁇ . Independent calculation of white balance correction factors is performed for each exposure. This enables white balancing scenes where the observable parts of the two exposures are at different color temperatures.
  • the LUTs may contain multiplicative correction factors which are evaluated as follows:
  • Y denotes a selective average over Y in the given white image and , ⁇ , y and ⁇ are the respective average values of ⁇ , ⁇ , y and ⁇ . Saturated or cutoff pixels are excluded from this average. Since by definition,
  • these LUTs are replaced by digital multipliers.
  • the LUTs may also be loaded with correction functions other than simple linear multiplicative factors.
  • the mosaic balance correction factors can be computed based on four average signals, namely, ⁇ , ⁇ , y and ⁇ , instead of merely two of them as above. This alternative yields improved uniformity of color balance under difficult conditions.
  • the white balance function may be done on the RGB color components in the color conversion block 78 (described below) .
  • color path block 28 is shown in more detail.
  • the input to color path block 28 is the image data ⁇ wb , jS wb , 7 wb , ⁇ S wb after processing by mosaic white balance block 32.
  • color difference evaluation block 34 receives data ⁇ wb , ⁇ wb , y wh , ⁇ wb from mosaic white balance block 32 and calculates color difference components dr, db for each pixel in the array:
  • Color difference components dr, db are thereafter received by low-pass color component block 36 which calculates a low-pass color component dr- p or db ⁇ for each pixel:
  • Block 36 performs horizontal low-pass filtering on dr ⁇ and d ⁇ calculated in block 34. This reduces color artifacts caused by interpolation.
  • the low-pass filter width is five pixels and its coefficients are ! «, , , -**, V4. The equations follow: For pixels (i,j) in even lines i:
  • Delay buffer 38 receives the output from low-pass color component block 36 and directs - 1), db
  • Vertical interpolation block 40 receives the low-pass color components as described above and generates interpolated low-pass color components dr',,, in the odd numbered lines and db in the even numbered lines:
  • the interpolated low-pass color components dr;, db; are multiplexed with the original low-pass components dr*, db, p to give the color path output values dr and dp for each pixel.
  • This function is performed by multiplexer 42, which separates the output received from delay buffer block 38 and vertical interpolator block 40 into a first path including db ⁇ ' (i ⁇ ) and db ⁇ i,*,) and a second path including dr, p (i CTen ) and r ⁇ i ⁇ ) .
  • FIG. 5 discloses in more detail the intensity (Y) processing block 30 shown in Figure 3, one sees that the input to intensity (Y) processing block 30 from mosaic white balance block 32 ( Figure 3) is received by intensity evaluation block 44 which outputs computed intensity Y for each pixel. Since only one of the four data types ( ⁇ , ⁇ , y, ⁇ ) is present at any given pixel, the intensity evaluation block 44 calculation is performed as follows (based on the prior definition of Y) :
  • the output from intensity evaluation block 44 is received by delay buffer 46, generate output intensity block 48 and limit block 50.
  • Delay buffer 46 is a delay line of two horizontal lines, required for the 3x3 and 1x3 matrix transformations in Y path block 30. Together with the color path delay buffer 38 and with Y path delay buffer 54, it may be implemented in a preferred embodiment in mosaic data space, operating on the input ⁇ , ⁇ , y, ⁇ data before the intensity (Y) evaluation block 44 and color difference evaluation block 34. It is shown here schematically for clarity.
  • Vertical low-pass filter 52 receives intensity (Y) signals from the intensity evaluation block 44 as delayed by delay buffer 46.
  • Block 52 generates the vertical low- pass intensity Y vtp defined as:
  • ⁇ ( i,j) Y(i-l,J) + 2Y(i,j) + Y(i-H,j)
  • the unfiltered intensity (Y) input will sometimes exhibit horizontal stripes, one pixel high in each field, in areas of transition to saturation. These stripes stem from the different color spectra of the a , ⁇ , y, and ⁇ pixels, as a result of which the ⁇ +7 value of Y(i(even),j) may, for instance, reach saturation at a lower level of optical intensity than the ⁇ + ⁇ value of the vertically adjacent Y(i+l(odd) ,j) . Y v * averages these values to obtain a function that is smooth over the transition area.
  • Generate output intensity block 48 receives intensity (Y) information from intensity evaluation block 44 and vertical low-pass intensity (Y v *) information from vertical low-pass filter 52.
  • the output of block 48 is output intensity (Y ⁇ ) to point processing LUT block 62 (see Figure
  • Block 48 replaces the original luminance Y, computed by the intensity evaluation block 44, with Y v * when Y * approaches saturation, in order to prevent the appearance of horizontal stripes as explained above.
  • Block 48 implements the function: Y ***-f ⁇ vl ⁇ ⁇ thrahold
  • Limit block 50 receives intensity (Y) signals from intensity evaluation block 44 and generates limited luminance Y ⁇ . Limit block 50 cuts off the upper range of intensity (Y) values that are to be input to edge detection block 56, in order to prevent detection of false edges or horizontal stripes that can arise in areas of transition to saturation. Limit block 50 implements the function:
  • Y ⁇ is typically equal to approximately 220.
  • limit block 50 i.e., Y ⁇
  • edge detection block 56 which outputs edge information for each pixel.
  • Edge detector block 56 convolves the Y ⁇ , value and its 8 immediate neighbors, with a high-pass or edge detecting kernel.
  • the 3x3 Laplacian operator may be used:
  • the following kernel may be used:
  • edge detector block 56 could be implemented as separate horizontal and vertical convolution operations (such as a l x 3 or 3 x l matrix) , with additional logic to avoid overemphasis of diagonal edges.
  • This alternative embodiment is less hardware intensive and gives improved picture quality in some circumstances.
  • Edge suppress block 58 receives the vertical low-pass intensity (Y v ⁇ ) signals from vertical low-pass filter 52 and outputs edge suppression function f ⁇ to edge multiplier 60.
  • the edge suppression function varies between 0 and 1 in the long exposure processing block 24 only. In the short exposure processing block 26, the function is set to 1, i.e., no edge suppression at this point.
  • the function is typically implemented in block 24 in a piecewise linear fashion as follows:
  • LOWSAT is set to approximately 190 and
  • Edge multiplier 60 receives input from blocks 56, 58 and generates suppressed edge ed ⁇ to intensity (Y) result calculation.
  • Edge multiplier 60 multiplies the edge output of the edge detector block 56 by the edge suppression function f ⁇ from block 58 to generate an output value ed ⁇ p to joint operations block 64 (see Figure 1) .
  • the purpose of this multiplication is to suppress distorted large edges that may appear in the long exposure at intensity (Y) values near saturation, at the same time as they appear in the short exposure at lower values of intensity (Y) .
  • the double appearance of such edges was found empirically to cause the resulting displayed edges to be overemphasized and sometimes smeared on account of blooming in the long exposure.
  • the long exposure edge is suppressed so that only the short exposure edge will pass through to the output image.
  • the edge suppress function may also be used to reduce the amplitude of edges from the long exposure which may be otherwise exaggerated due to the higher gain of the long exposure relative to the short exposure.
  • an optional multiplier or LUT may be added to multiply the output of block 56 times the ratio of exposure times (duration of long exposure/duration of short exposure) or the corresponding gain ratio, or some function of the exposure and/or gain ratio. This reflects the ratio of scales of these two values.
  • Y path block 30 outputs processed luminance Y ⁇ , edge, and edge. ⁇ to point processing block 62 and joint operations block 64.
  • point processing block 62 includes four point processing functions, all of which receive output intensity (Y ou ,) values from the long and short exposure processing blocks 24, 26 (see Figure 1). These functions may typically be implemented as LUTs in RAM or ROM memory. Point processing block 62 generates arbitrary function values for input to the joint operations block 64 ( Figure 1) .
  • the four tables of block 62 are:
  • the intensity (DC result) block 66 which generates a LUT value of intensity (Y ⁇ ) for the joint operations block 64.
  • Block 66 controls the amount of point (“DC”) luminance that is summed with the edge information in generating the output luminance, Y ban___.
  • DC point
  • this does not cover all cases, e.g. when neither exposure carries any information (long is saturated and short is cut-off, or both saturated, or both cut-off) .
  • Blocks 68 and 70 control the proportions of mixing the color values, dr and db, from the long and short exposures, respectively, that will be used to generate the output color values, dr ⁇ ⁇ and db, ⁇ .
  • w, and w are chosen so as to give predominant weight at each pixel to the color values taken from the exposure in which the intensity (Y) luminance values are in the linear portion of the range, and to give a smooth transition over luminance gradient regions of the image.
  • W ! and w are deter ⁇ mined on the basis of Y om (long) alone, except for cases where the long exposure is near saturation while the short is near cutoff, so that neither gives a linear reading.
  • the outputs of blocks 68 and 70 are normalized by division by the corresponding values of Y ou , for the long and short exposures.
  • a floating point representation for the output values of blocks 68, 70 is used so as to maintain sufficient accuracy to prevent noticeable quantization in the output image.
  • An additional edge color suppression factor, Z ⁇ is computed in the joint operations block (as will be described herein ⁇ after) .
  • the purpose of the saturation color suppression function is to reduce the appearance of color artifacts that arise due to CCD saturation.
  • the linear relationships between the ⁇ , ⁇ , 7, and ⁇ CCD outputs and the true RGB colors break down as the CCD 14 approaches saturation. As non-linear deviations cannot be readily corrected, suspected distorted colors are "whitewashed". Similar techniques are used in the analog domain in conventional CCD cameras.
  • Figure 7 discloses the joint operations block 64 (also see Figure 1) .
  • Joint operations block 64 combines the chrominance and luminance data from the long and short exposure processing blocks 24, 26, together with data from point processing block 62, to generate a combined Y/dr/db result.
  • Block 64 then converts this result to output in standard RGB or Y/Cr/Cb (luminance, chrominance (red) and chrominance (blue)) color space.
  • a color suppression factor Z is computed and applied to the chrominance outputs in order to reduce color artifacts (by reducing chroma saturation) around edges and areas of luminance signal saturation.
  • Joint operations block 64 includes:
  • the dr, db, Y block 74 (recalling that dr and db are the differences between successive readings in even and odd lines, respectively) which receives dr, db values from the color path outputs of long and short exposure processing blocks 24, 26 respectively; ed ⁇ from the intensity (Y) path output of long exposure processing block 24 and edge data from the intensity
  • Block 74 generates combined intensity Y/dr/db results to color conversion block 78 (to be discussed) . Block 74 will be discussed in greater detail hereinafter.
  • the color suppression factor block 76 which receives ed l00g and ed, ⁇ from edge detector block 56 and saturation color suppression factor (Wht) from point processing block 62 and generates chroma suppression factor Z for color conversion block 78.
  • Block 76 will be discussed in greater detail hereinafter.
  • the color conversion block 78 which receives Y « « ⁇ , dr, ⁇ , db ⁇ from block 74 and Z, the color suppression factor from block 76 and generates R o *, G o ,,,, and B oumble t and Cr and Cb. Block 78 will be discussed in greater detail hereinafter.
  • the dr, db, Y block 74 is shown in further detail in Figure 8.
  • Block 74 includes an intensity (Y) calculation which is performed by adders 79, 80 and edge limiting block 81.
  • Adder 79 receives ed Bjpp (long) data from long exposure processing block 24, and ed* ⁇ from short exposure processing block 26. These two inputs are added to give edge re ⁇ , which is then input to the edge limiting block 81.
  • Edge limiting is implemented as a piecewise linear function with 6 inflection points (A ⁇ .-A and 4 slopes (S ⁇ ...S 4 ), as shown in the upper right inset of Figure 8. Generally the inflection points and slopes are chosen so as to enhance the smaller edges (i.e., S 2 and S 3 > 1) , while large edges (edge > A 5 or ⁇ A 2 ) are suppressed.
  • a 3 and A 4 may be set to 0, but it is sometimes desirable to set them to small non-zero values in order to suppress false edges due to noise. The best results appear to be obtained with
  • Block 74 further includes a dr, db calculation which is performed by the remaining sections of block 74.
  • the dr, db calculation receives low-pass color components dr, db from the color paths of long and short exposure processing blocks 24, 26; w,/Y, and w,/Y, from block 62; and Yi wu i t s calculated by adder 80.
  • the dr, db calculation outputs dr ⁇ and db ⁇ .
  • the long and short values of dr and db are multiplied by the respective normalized color weights, w,/Y, and w s /Y, by multipliers 82, 84, 86, 88.
  • These normalized, weighted color values from the two exposures are summed together by adders 90, 92 and then multiplied by Y ⁇ ⁇ by multipliers 94, 96 to give the scaled values:
  • dr ⁇ ⁇ and db ⁇ ⁇ may be generated by selection between the long and short normalized dr and db inputs (and possibly their long/short average values) .
  • the color suppression factor block 76 of Figure 7 is shown in more detail in Figure 9.
  • Maximum value block 100 selects the higher of the two absolute values of ed ⁇ and ed ⁇ , ⁇ as calculated by absolute value blocks 98, 99.
  • the result of the calculation of block 100, ed ⁇ is input to edge chroma suppression factor block 102 to calculate Z ⁇ .
  • minimum value of Z ⁇ , Th is ordinarily set to zero, to give complete chroma suppression at very strong edges. Th ⁇ 0 is used only in replay of images stored in mosaic format (see generate mosaic block 120 described hereinafter) , in which case Z ⁇ serves to suppress color anomalies resulting from the reinterpolation of the pixel values. Thereafter, as shown in Figure 9, minimum value block 104 selects the minimum of the two color suppression factors, Z j j and Wht, thereby determining the edge criterion or saturation criterion that should be used to provide the required degree of chroma suppression at the given pixel.
  • color conversion block 78 receives Y, ⁇ , dr ⁇ ⁇ , and db, ⁇ from block 74 and Z from block 76 and generates outputs in both the RGB and Y/Cr/Cb formulations.
  • block 78 takes the interim dynamic range enhancement results Y/dr/db, and converts them into conventional color components for system output.
  • Block 78 includes horizontal low-pass filter 106 which receives Y,, ⁇ and calculates Y ⁇ ⁇ (lp) for the color matrix block 108.
  • Horizontal low-pass filter 106 is identical to the low-pass color component block 36 in the color path block
  • Color matrix block 108 receives Y, ⁇ (lp) from horizontal low-pass filter 106 and dr, ⁇ and db, ⁇ from block 74 and generates low-pass RGB color component outputs.
  • RGB white balance multipliers 109 A , 109 B , 109 c receive low-pass RGB signals from color matrix block 108 and generate normalized low-pass RGB signals.
  • Multipliers 109 A , 109 B , 109 c multiply each of the RGB low-pass values by a pre-computed white balance correction factor, adjusted by the normalization factor 0.7 required by the color matrix calculation.
  • conventional RGB white balancing uses only two multiplicative factors, correcting R and B while G is held constant, this "short cut" does not preserve constant Y achromatic luminance. This loss of normalization may lead to the appearance of artifacts and incorrect luminance in the output. It is necessary, therefore, to use three multiplicative factors, normalized to preserve constant luminance Y.
  • Output signal enhancement block 110 (which includes chroma suppression and RGB output functions) receives corrected low-pass RGB color component signals from color matrix block 108 via multipliers 109 A , 109 B , 109 c ; £ meet purpose & from block 74; (lp) from block 106; and chroma suppression factor Z from block 76.
  • the RGB values output from color matrix block 108 are low-pass values.
  • K is an arbitrary constant between 0 and 1, chosen according to the degree of high-frequency enhancement required. Values in the range 0.4 ⁇ K ⁇ 0.8 are typically used.
  • RGB components can alter the original values of R/G and B/G, with the result that the correct hue of the image is not preserved. Therefore, in an alternative embodiment, R, G and B are multiplied by a high-pass enhancement function.
  • the chroma suppression factor, Z is best applied to chrominance-only components, by adders 113 A , 113 B , 113 c :
  • the final RGB values from the processed image are used to generate equivalent, simulated mosaic values of ⁇ , ⁇ , y, and ⁇ .
  • ⁇ , ⁇ , y, and ⁇ are used to generate equivalent, simulated mosaic values of ⁇ , ⁇ , y, and ⁇ .
  • the simulated mosaic values are generated by the following matrix in matrix block 122, based on the color equivalencies given hereinabove.
  • multiplexer 124 selects which one of the four mosaic values to output for each pixel according to the table:
  • Apparatus 10 has three modes of operation: normal, adaptive sensitivity (AS) , and replay.
  • Normal mode emulates the performance of a mosaic color CCD camera without adaptive sensitivity. In this mode only the long exposure portion of the pipeline operates.
  • the processing functions are limited to decoding the mosaic input into conventional color components: Y/Cr/Cb or RGB, while additionally performing filtering operations for anti-aliasing, detail (edge) enhancement and chroma suppression where required.
  • Adaptive sensitivity mode uses all the resources of the processing pipeline to generate wide dynamic range images as described hereinabove.
  • Replay mode is required for displaying images that have been stored in RAM or disk. Apparatus 10 stores these images in a regenerated mosaic format in order to save on storage memory requirements. Replay mode is similar to normal mode, except that most of the enhancement operations are not performed: since the stored data have already been filtered once, it is for the most part not desirable to filter them again.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Image Processing (AREA)
  • Processing Of Color Television Signals (AREA)

Abstract

Un appareil couleur à gamme dynamique large (10) comporte un filtre (12) placé juste devant des éléments de couleurs répétitifs de façon que chaque pixel représente un élément de couleur donnée d'une vue. On utilise au moins deux niveaux d'exposition par vue et les sorties de pixels sont décodées de façon à produire les signaux de luminance et de chrominance vidéo. Les images des deux niveaux d'exposition au moins sont combinées pour former une image finale.
EP94907434A 1993-02-08 1994-02-07 Camera couleur a gamme dynamique large utilisant un dispositif a transfert de charge et un filtre mosaique Withdrawn EP0739571A1 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US1454593A 1993-02-08 1993-02-08
US14545 1993-02-08
PCT/US1994/001358 WO1994018801A1 (fr) 1993-02-08 1994-02-07 Camera couleur a gamme dynamique large utilisant un dispositif a transfert de charge et un filtre mosaique

Publications (1)

Publication Number Publication Date
EP0739571A1 true EP0739571A1 (fr) 1996-10-30

Family

ID=21766100

Family Applications (1)

Application Number Title Priority Date Filing Date
EP94907434A Withdrawn EP0739571A1 (fr) 1993-02-08 1994-02-07 Camera couleur a gamme dynamique large utilisant un dispositif a transfert de charge et un filtre mosaique

Country Status (2)

Country Link
EP (1) EP0739571A1 (fr)
WO (1) WO1994018801A1 (fr)

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6040858A (en) * 1994-11-18 2000-03-21 Canon Kabushiki Kaisha Method and apparatus for expanding the dynamic range of sensed color images
JP3134784B2 (ja) * 1996-08-05 2001-02-13 松下電器産業株式会社 画像合成回路
JP3806211B2 (ja) * 1997-01-08 2006-08-09 株式会社リコー 撮像信号処理方法及び撮像信号処理装置
FI103627B (fi) * 1997-06-12 1999-07-30 Fours Oy Digitointilaite
GB2341029B (en) * 1998-08-29 2002-12-31 Marconi Gec Ltd Cameras
US7084905B1 (en) 2000-02-23 2006-08-01 The Trustees Of Columbia University In The City Of New York Method and apparatus for obtaining high dynamic range images
KR100850729B1 (ko) * 2000-07-06 2008-08-06 더 트러스티스 오브 콜롬비아 유니버시티 인 더 시티 오브 뉴욕 데이터 해상도를 향상시키는 방법 및 장치
US7149262B1 (en) * 2000-07-06 2006-12-12 The Trustees Of Columbia University In The City Of New York Method and apparatus for enhancing data resolution
JP2002290838A (ja) 2001-03-27 2002-10-04 Matsushita Electric Ind Co Ltd 映像信号処理方式及び撮像装置
JP4822630B2 (ja) 2001-08-14 2011-11-24 キヤノン株式会社 色信号処理装置、撮像装置およびそれらの制御方法
JP4272443B2 (ja) * 2003-02-05 2009-06-03 パナソニック株式会社 画像処理装置および画像処理方法
JP4139276B2 (ja) 2003-06-17 2008-08-27 オリンパス株式会社 電子内視鏡装置及び信号処理装置
US20070024721A1 (en) * 2005-07-29 2007-02-01 Rogers Sean S Compensating for improperly exposed areas in digital images
CN103489165B (zh) * 2013-10-01 2016-04-06 中国人民解放军国防科学技术大学 一种面向视频拼接的小数查找表生成方法
KR20230010842A (ko) * 2017-11-06 2023-01-19 에이조 가부시키가이샤 화상처리장치, 화상처리방법 및 화상처리 프로그램을 기록한 기록 매체

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS5934772A (ja) * 1982-08-20 1984-02-25 Olympus Optical Co Ltd 画像信号処理装置
US4584606A (en) * 1983-09-01 1986-04-22 Olympus Optical Co., Ltd. Image pickup means
US4647975A (en) * 1985-10-30 1987-03-03 Polaroid Corporation Exposure control system for an electronic imaging camera having increased dynamic range
IL79485A0 (en) * 1986-07-21 1986-10-31 Technion Res & Dev Foundation Random scan system
EP0263295B1 (fr) * 1986-09-09 1994-06-08 Fuji Photo Film Co., Ltd. Caméra électronique pour image fixe pour la compensation de la dépendance de la température de couleur de signaux vidéo couleurs
IL87306A0 (en) * 1988-08-02 1989-01-31 Technion Res & Dev Foundation Wide dynamic range camera
US5247366A (en) * 1989-08-02 1993-09-21 I Sight Ltd. Color wide dynamic range camera
US5138458A (en) * 1989-12-22 1992-08-11 Olympus Optical Co., Ltd. Electronic camera apparatus capable of providing wide dynamic range image signal

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO9418801A1 *

Also Published As

Publication number Publication date
WO1994018801A1 (fr) 1994-08-18

Similar Documents

Publication Publication Date Title
US8736723B2 (en) Image processing system, method and program, including a correction coefficient calculation section for gradation correction
US5247366A (en) Color wide dynamic range camera
US8194160B2 (en) Image gradation processing apparatus and recording
US8488026B2 (en) Image capturing system and computer readable recording medium for recording image processing program
US7072509B2 (en) Electronic image color plane reconstruction
JP3548504B2 (ja) 信号処理装置、信号処理方法、及び撮像装置
US9936172B2 (en) Signal processing device, signal processing method, and signal processing program for performing color reproduction of an image
EP0739571A1 (fr) Camera couleur a gamme dynamique large utilisant un dispositif a transfert de charge et un filtre mosaique
JP2004074793A (ja) 画像の色補正の方法
JP2009124552A (ja) ノイズ低減システム、ノイズ低減プログラム及び撮像システム。
EP2360929B1 (fr) Dispositif de traitement de l'image
CN1937781B (zh) 图像处理设备和图像处理方法
CN101238485B (zh) 图像处理装置
KR101133520B1 (ko) 색 노이즈 제거 방법 및 장치
US7684636B2 (en) Edge correction apparatus, edge correction method, program, and storage medium
EP1530364A1 (fr) Système et procédé pour l'accentuation des bords
JP4086572B2 (ja) 映像信号処理装置
JP2006148248A (ja) 撮像システム及び撮像処理プログラム
JP4122082B2 (ja) 信号処理装置およびその処理方法
KR102051367B1 (ko) Ｗｒｇｂ ｃｆａ 영상을 위한 처리 장치 및 그 처리 방법
JP3992403B2 (ja) 画像信号処理装置および画像信号処理方法
JP7814728B2 (ja) 画像信号処理方法
US8035706B2 (en) Image processing apparatus and digital camera
JPH06269009A (ja) カラーワイドダイナミックレンジカメラ
JPH11164309A (ja) 画像信号処理方法及び電子カメラ

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 19950908

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): DE FR GB

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 19960830