WO2006120585A2 - Amelioration d'image - Google Patents

Amelioration d'image Download PDF

Info

Publication number
WO2006120585A2
WO2006120585A2 PCT/IB2006/050641 IB2006050641W WO2006120585A2 WO 2006120585 A2 WO2006120585 A2 WO 2006120585A2 IB 2006050641 W IB2006050641 W IB 2006050641W WO 2006120585 A2 WO2006120585 A2 WO 2006120585A2
Authority
WO
WIPO (PCT)
Prior art keywords
pixel
sub
images
image
calculating
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/IB2006/050641
Other languages
English (en)
Other versions
WO2006120585A3 (fr
Inventor
Henricus W. P. Van Der Heijden
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Koninklijke Philips NV
Original Assignee
Koninklijke Philips Electronics NV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics NV filed Critical Koninklijke Philips Electronics NV
Publication of WO2006120585A2 publication Critical patent/WO2006120585A2/fr
Publication of WO2006120585A3 publication Critical patent/WO2006120585A3/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform

Definitions

  • the disclosure relates in general to changing pixel values within images, for instance, still images or video images.
  • image enhancement by using local contrast boosting or other algorithms is known.
  • the image content can be enhanced by "boosting" the image features, e.g. the pixel values on all frequencies of the spatial spectrum except the lowest frequency.
  • a wide variety of techniques for processing and filtering signals for instance, representing two-dimensional images, for example, still image or video images, have been developed. The clarity, contrast, and sharpness of images might be required to be improved due to transmission noise or other factors. It may also happen that the original image itself is insufficiently clear and needs to be sharpened.
  • a blurred or perceptually blurred image may be enhanced by enhancing high frequency spatial components of the image.
  • high frequency components are usually degraded more significantly during transmission than lower frequency components.
  • enhancement of high frequency components may be effective in compensating for high frequency components lost during transmission.
  • image processing techniques modifying or supplementing the high spatial frequency components of an image have been developed.
  • the Burt-pyramid algorithm permits splitting the original image into a plurality of sub- images, with a hierarchy of separate component images.
  • Each of the sub- images can be a laplacian image comprised of different spatial frequency ranges of the original image plus a remnant Gaussian image.
  • Boosting can result in changing the pixel values within the corresponding frequency ranges.
  • An original image G 0 can be enhanced into an enhanced image G 0 ' with
  • D n can be understood as a set of sub- image derived from "primary" sub- images G n .
  • both D n and G n can be understood as sub- images, whereby D n are derived from G n , as will be described in more detail below.
  • sharpening techniques which typically only work on the highest spatial frequencies, representing edges, the described boosting of pixel values in all frequencies, but the lowest frequency range can enhance both sharpness and contrast in smooth image areas, with little contrast.
  • an object of the invention to provide an image enhancement which takes into account different contrast in different image areas.
  • Another object of the invention is to provide image enhancement changing pixel values only where necessary.
  • a further object of the invention is to provide image enhancement, where pixel change values of higher frequency ranges are taken into account when calculating pixel change values in lower frequency ranges.
  • Yet another object of the invention is to reduce the "light emission" effect of local contrast boosting techniques.
  • Another application area of the current invention is the removal of unwanted image details, for example noise and digital coding artifacts such as blocking, ringing and mosquito noise.
  • An object of the invention in this area is to determine which image detail is worth keeping, and which details should be reduced.
  • a further object of the invention is to provide noise and artiiact reduction, where pixel change values of higher frequency ranges are taken into account when calculating pixel change values in lower frequency ranges.
  • each of the sub- images represents a corresponding spatial frequency range of the image
  • calculating a pixel detail signal for at least one of the sub- images, depending on at least a pixel detail signal of another frequency range calculating a pixel change value for pixels within the sub- images depending on the corresponding pixel detail signal
  • calculating changed sub- images by changing pixel values within the sub- images depending on the corresponding pixel change value, and combining the changed sub- images into an output image.
  • the sub- images can, for example, be produced by convolving and decimating using a convolution filter.
  • the convolution filter can be an FIR filter. Enhancing, for example, 5x5 or 7x7 pixel image segments with a separable FIR filter with Id coefficients is possible.
  • the filter coefficients can, for example, be (1, 4, 6, 4, 1)/16 or (1, 6, 15, 20, 15, 6, l)/64.
  • the output of such a filter can be fed back to the input, for example, after being down- sampled both horizontally and vertically with a factor of two. For example, an original 256x256 image gives rise to a 128x128 filtered and down-sampled image, than a 64x64 image etc.
  • the output of the FIR filter can be fed back to its input, resulting in a sequence of images with less contrast due to a reduced amount of high frequency components.
  • the sub- images can represent separate component images of the original image in corresponding spatial frequency ranges. For each of the sub- images a pixel detail signal depending on at least a pixel detail signal of a sub- image in another frequency range can be calculated. This allows creating pixel detail signals which account for pixel detail already detected in sub- images of a higher frequency. Thus, boosting of pixels in higher frequencies can already be accounted for.
  • a pixel change value enabling to change the pixel values in the respective frequency ranges can be calculated from the corresponding pixel detail signal.
  • the values of the pixels in corresponding frequency ranges e.g., the corresponding sub-images, can be changed.
  • the changed sub-images can be combined into an output image with enhanced sharpness and contrast characteristics.
  • the pixel detail signal can be a cumulative pixel detail signal depending on at least a pixel detail signal of a neighboring frequency range. For example, when calculating the pixel detail signal, a loop can be created to input the pixel detail signal of a higher frequency for calculating the pixel detail signal of the next frequency range. This allows accounting for detected pixel detail in a higher frequency range.
  • embodiments can comprise calculating a maximum pixel value within an aperture of KxL pixels of a pixel detail signal within another frequency range, the aperture surrounding the corresponding pixel.
  • K and L can denote integers.
  • the aperture can represent a 5x5 pixel filter. In an area of 5x5 pixels around the respective pixels in the pixel detail signal of the previous frequency range, the maximum value can be obtained.
  • the pixel detail signal from a higher frequency band must be accumulated to this working area using a function that spreads the value over a similar larger area.
  • the pixel detail signal is up- sampled before calculating the maximum pixel value. Up-sampling can be carried out by increasing the number of pixels with a factor, for example 2, by interpolating.
  • embodiments can provide down-sampling the pixel detail signal after calculating the maximum pixel value.
  • the down-sampling can be in the same amount as the previous up-sampling.
  • embodiments can provide calculating the pixel change value comprising decreasing the pixel change value with an increased cumulative pixel detail signal.
  • the pixel detail signal accounts for the pixel change value.
  • the pixel change value for contrast boosting is calculated as
  • CATS value is zero, i.e. no cumulative pixel detail signal has been detected, yet, the frequency band can be boosted with the full gain factor f. With an increasing CATS, the gain is transited to 1, which is reached when the CATS value exceeds the threshold value T.
  • a global gain factor f can, for instance, be between 2 and 3.
  • the threshold value can typically be around 64 for 8 bit image data.
  • the pixel change value for noise and artiiact reduction is calculated as
  • g the pixel change value
  • fmin a minimal gain factor between 0 and 1
  • Tnoise a threshold value
  • CATS the pixel detail signal
  • i an integer representing the corresponding
  • the MAX ( ) results in the maximum value between and 1-fmin, noise thus having only positive values. If the CATS value is zero, i.e. no cumulative pixel detail signal has been detected, the frequency band is expected to mostly contain noise and other unwanted details, so the band is boosted with the minimal gain factor fmin. With an increasing CATS, the gain is transited to 1, which is reached when the CATS value exceeds the threshold value Tnoise.
  • the Tnoise threshold value can typically be around 16 for 8 bit image data.
  • a derived sub- image can be calculated, according to embodiments, for at least one of the at least three sub- images.
  • the derived sub- images can, for instance, be Differential of Gaussian (DOGS) images.
  • DOGS Differential of Gaussian
  • the sub- images can be subtracted from the sub- image of the next higher frequency range, producing the Differential of Gaussian image.
  • the pixel detail signal can be calculated as
  • CATS 1 abs D 1 (x,y) + max CATS 1-1 .
  • the maximum pixel value (max) of the pixel detail signal (CATS) of a higher frequency range can be added to the absolute value of a derived sub- image (D) obtaining the pixel detail signal (CATS) of the corresponding frequency range.
  • the integer i can denote the corresponding frequency range.
  • the pixel detail signal for the highest frequency range is the absolute value of the first derived sub- image. This accounts for that for the highest frequency range there is no pixel detail signal of a higher frequency range to be used as input for calculating the pixel detail signal.
  • Embodiments can provide splitting the image into at least three sub- images, comprising applying at least a spatial low-pass filtering, iteratively.
  • the spatial low-pass filtering can be an FIR filter.
  • the output of the low-pass filter can be fed back to the input to obtain an iteration of low-pass filtering.
  • Embodiments can provide down-sampling the low-pass filtered image after low-pass filtering. In case the down-sampled sub- images are used for calculating the derived sub- images, these are interpolated to allow subtracting the sub- image from the sub- image of the next higher frequency range.
  • Embodiments can provide combining the changed sub- images into an output image by calculating a summed value of the changed sub- images and the sub- image in the lowest frequency range. This can be done, for example, by calculating
  • G 0 the output image
  • G N the sub- image in the lowest frequency range
  • g! the pixel change value
  • D 1 the derived sub- image.
  • N denoting the absolute number of sub- images
  • i denoting the corresponding frequency range.
  • This calculation of the enhanced output image by adding the modified values of the derived sub- images with the pixel change value involves interpolating the pixels in different grids.
  • the derived sub- images of the frequency range i have a grid of M/2 1 xN/2 1 , thus, the derived sub- images need to be up-sampled before being summed-up.
  • Another aspect of the invention is an image enhancement device comprising first filter means arranged for splitting an image into at least three sub- images, wherein each of the sub- images represents a corresponding spatial frequency range of the image, first combination means arranged for calculating a pixel detail signal for at least one of the sub- images, depending on at least a pixel detail signal of another frequency range, second combination means arranged for calculating a pixel change value for pixels within the sub- images depending on the corresponding pixel detail signal, calculation means arranged for calculating changed sub- images by changing pixel values within the sub- images depending on the corresponding pixel change value, and third combination means arranged for combining the changed sub- images into an output image.
  • a further aspect of the invention is a computer program product tangibly embodied in an information carrier, the computer program product comprising instructions that, when executed, cause at least one processor to perform operations comprising: splitting an image into at least three sub- images, wherein each of the sub- images represents a corresponding spatial frequency range of the image, calculating a pixel detail signal for at least one of the sub- images, depending on at least a pixel detail signal of another frequency range, calculating a pixel change value for pixels within the sub- images depending on the corresponding pixel detail signal, calculating changed sub- images by changing pixel values within the sub- images depending on the corresponding pixel change value, and combining the changed sub- images into an output image.
  • a further aspect of the invention is a use of such a method in image processing and video processing.
  • Fig. 1 illustrates a block diagram for obtaining sub-images, derived sub- images, and pixel detail signals
  • Fig. 2 illustrates a block diagram of a further embodiment
  • Fig. 3 illustrates a combination of pixel detail values, derived sub- images and sub- images into an output image, according to embodiments.
  • Fig. 1 illustrates a block diagram of a method for obtaining an enhanced image.
  • G denote sub- images
  • i denotes the respective frequency range
  • G 0 denotes an original image
  • D represents derived sub- images.
  • CATS denotes a pixel detail signal
  • g denote pixel change values and gD changed sub-images.
  • An input image G 0 is input to a Gaussian low-pass filter 2.
  • the Gaussian low- pass filter 2 can be an FIR filter.
  • the input of Gaussian low-pass filter 2 is convolved with a filter function obtaining a low-pass filtered sub- image.
  • the pixel range of this image is MxN, with M and N integers denoting the size of a pixel range.
  • the output of Gaussian low-pass filter 2 is input to a filter 4 for reducing the number of samples in the vertical and horizontal direction.
  • the filter factor of filter 4 can, for instance, be 2.
  • the output sub- image has M/2xN/2 samples. This sub- image is input for the next iteration of this algorithm in a feedback loop (not shown).
  • Each sub- image is fed to interpolator 6, where the number of samples is increased by a respective factor.
  • the samples are reduced by a factor of 2 in filter 4, the samples are interpolated in interpolator 6 to obtain an image with two times the number of samples.
  • the output of interpolator 6 is fed to subtracter 8.
  • subtracter 8 the sub- image is subtracted from the image in the next higher frequency range. In the first iteration, this is the input image G 0 subtracted by the first sub- image G 1 , in the second iteration this is G 1 subtracted by the sub- image G 2 , etc.
  • the output of subtracter 8 is derived sub- image D in the respective frequency ranges i.
  • the subtracter can also be arranged such that the output of Gaussian low-pass filter 2 is subtracted directly from the input image Gi. This would allow omitting the interpolator 6. All other elements are the same as in Fig. 1.
  • filter 4 can be arranged after the branch to subtracter 8.
  • Filter 12 provides obtaining the absolute value of the corresponding derived sub- image.
  • a maximum filter 10 is fed by a pixel detail signal of a previous frequency range CATSi -1 .
  • the pixel detail signal of the previous frequency CATSi -1 can first be applied to an upsampling filter 9 to account for the different aperture in image segments of different frequency ranges.
  • the input to the maximum filter 10 is 0, and thus, the CATS 0 value is set equal to the absolute value of the derived sub- image D 0 , which is added to the CATSi signal through filter 12 in adder 14.
  • the pixel detail signal of the previous frequency range CATSi -1 is passed through the maximum filter 10 with an KxL aperture.
  • the KxL aperture which can be a 5x5 aperture, allows finding the maximum pixel value in the vicinity of 5x5 pixel of the corresponding pixel in the input signal.
  • the value of the pixel detail signal CATSi in the corresponding frequency range is set to the maximum value of the pixel detail signal CATSi -1 in the next higher frequency range in a 5x5 neighborhood around the pixel at position x, y.
  • the CATSi signal output from maximum filter 10 is down-sampled in filter 16 and fed to adder 14.
  • the CATSi signal is added with the absolute value of the derived sub- image in the corresponding frequency range. Having obtained the pixel detail signal for each of the frequency bands, this signal can be used to obtain a pixel change value, the pixel change value can be calculated such that the higher the pixel detail signal is, the lower the pixel change value is.
  • the pixel detail signal CATS can be a cumulative signal, taking into account the maximum value of the pixel detail signal in the neighboring frequency range. Thus, a pixel detail signal CATS is increased during each iteration with the maximum value around corresponding pixels of the pixel detail signal of a previous frequency range.
  • the pixel change value can be calculated within combination means 22 as
  • the aim of the image enhancement is contrast boosting
  • the aim of the image enhancement is noise and artifact reduction.
  • the calculator 20 can multiply the derived sub images D 1 within the respective frequency ranges to obtain the changed derived sub- images g,D,.
  • the derived sub- images D 1 are multiplied with the pixel change values g! for each frequency range i in the calculator 20.
  • the changed derived sub- images gA can be fed to summer 24, where a sum over all changed derived sub- images gA is created within all frequency ranges i except the lowest frequency range denoted by N.
  • a hold element 18 can be provided, only feeding the lowest frequency sub image G N to the adder 24.
  • the derived sub- images are up-scaled to the original resolution, as necessary.
  • the up-scaling can be linearly or bilinear, bicubic , or any other interpolation mechanism.
  • the calculation of the pixel change value has the effect that the frequency boosting is reduced for lower frequencies once the respective areas have been boosted already by a pixel change value in a higher frequency range. If there is, at a given image position, a pixel detail signal, or a so-called activity, on a higher frequency range, for example, due to a hard edge, the following lower frequencies will no longer be boosted.
  • Each frequency band contains the signal from the previous frequency range extended over a larger area plus the activity from the band itself.
  • An area with, for example, edges has a high pixel detail value from the start, reducing the boosting at all following frequencies.
  • the method can be applied to image enhancement in television sets or video processing software or any video equipment in general.
  • Determining whether noise/artifact reduction or contrast boosting should be applied is preferably based on characteristics of the sub- images.
  • calculating the pixel change value comprises decreasing or increasing the pixel change value with an increased cumulative pixel detail signal, depending on characteristics of the sub- images.
  • characteristics of the sub images is meant e.g. estimated noise level or compression parameters like the amount of applied compression. That means that an analysis of the sub images is performed to compute characteristics in order to control changing pixel values within images, i.e. image enhancement.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Facsimile Image Signal Circuits (AREA)

Abstract

Cette invention concerne un procédé d'amélioration d'image. Des débordements d'amplification d'image dans des zones présentant des contours peuvent être évités par division d'une image en au moins trois sous-images. Chaque sous-image représente une gamme de fréquence spatiale respective de l'image. Le procédé consiste à: calculer un signal de détail de pixel pour au moins une des sous-images, sur la base d'au moins un signal de détail de pixel d'une autre gamme de fréquence; calculer une valeur de changement de pixel pour des pixels situés à l'intérieur des sous-images, sur la base d'un signal de détail de pixel correspondant; calculer des sous-images modifiées par changement de valeurs de pixel à l'intérieur des sous-images sur la base de la valeur de changement correspondante; et combiner les sous-images modifiées pour obtenir une image de sortie.
PCT/IB2006/050641 2005-03-03 2006-03-01 Amelioration d'image Ceased WO2006120585A2 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
EP05101656 2005-03-03
EP05101656.6 2005-03-03
EP05106507 2005-07-15
EP05106507.6 2005-07-15

Publications (2)

Publication Number Publication Date
WO2006120585A2 true WO2006120585A2 (fr) 2006-11-16
WO2006120585A3 WO2006120585A3 (fr) 2007-04-05

Family

ID=37396941

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2006/050641 Ceased WO2006120585A2 (fr) 2005-03-03 2006-03-01 Amelioration d'image

Country Status (1)

Country Link
WO (1) WO2006120585A2 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111899176A (zh) * 2020-07-31 2020-11-06 罗雄彪 一种视频图像增强方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0574969B1 (fr) * 1992-06-19 2002-03-20 Agfa-Gevaert Méthode et dispositif de réduction du bruit
DE69331719T2 (de) * 1992-06-19 2002-10-24 Agfa-Gevaert, Mortsel Verfahren und Vorrichtung zur Geräuschunterdrückung
DE19849090A1 (de) * 1998-10-24 2000-04-27 Philips Corp Intellectual Pty Verfahren zur Verarbeitung eines Eingangsbildes

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111899176A (zh) * 2020-07-31 2020-11-06 罗雄彪 一种视频图像增强方法
CN111899176B (zh) * 2020-07-31 2022-08-12 罗雄彪 一种视频图像增强方法

Also Published As

Publication number Publication date
WO2006120585A3 (fr) 2007-04-05

Similar Documents

Publication Publication Date Title
US8228560B2 (en) Image contrast enhancement
Zhang et al. Adaptive bilateral filter for sharpness enhancement and noise removal
US8144253B2 (en) Multi-frame approach for image upscaling
US7529425B2 (en) Denoising method, apparatus, and program
US8457429B2 (en) Method and system for enhancing image signals and other signals to increase perception of depth
EP2294808B1 (fr) Procédé et système de traitement vidéo efficient
US8351725B2 (en) Image sharpening technique
JPH06245113A (ja) ノイズ及び他のアーティファクトを除去して画像をより向上させるための装置
US7257271B2 (en) Noise reduction in color digital images using pyramid decomposition
US7876979B2 (en) Resolution-converting apparatus and method
EP1610267B1 (fr) Agrandissement d'image non-linéare adaptatif utilisant des coefficients de transformée d'ondelettes
JP4498361B2 (ja) レティネックス型アルゴリズムを高速化する方法
KR100565065B1 (ko) 필터 뱅크를 이용한 이미지 세부묘사 향상 방법 및 장치
JP4663349B2 (ja) スクリーン周波数を推定する方法及び装置
US20040086201A1 (en) Fast edge directed polynomial interpolation
KR101675117B1 (ko) 다계층의 저해상도 영상을 이용한 고해상도 영상 생성 방법 및 그 장치
WO2006092763A1 (fr) Amelioration de contraste et de definition d'image
Kang et al. Real-time super-resolution for digital zooming using finite kernel-based edge orientation estimation and truncated image restoration
WO2006120585A2 (fr) Amelioration d'image
Chen et al. Image interpolation using across-scale pixel correlation
JP2009070407A (ja) 画像処理方法および装置並びに記録媒体
US6687417B1 (en) Modified kernel for image interpolation
Saito et al. Demosaicing method using the extended color total-variation regularization
JP3059287B2 (ja) ２値画像変倍装置
Feng et al. Perceptual thumbnail generation

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

NENP Non-entry into the national phase

Ref country code: RU

WWW Wipo information: withdrawn in national office

Country of ref document: RU

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 06765655

Country of ref document: EP

Kind code of ref document: A2

122 Ep: pct application non-entry in european phase

Ref document number: 06765655

Country of ref document: EP

Kind code of ref document: A2

WWW Wipo information: withdrawn in national office

Ref document number: 6765655

Country of ref document: EP