JPH1132333A - Device for processing compressed picture - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress the noise of an output signal by operating an arithmetic operation, including an optimal restoration equation related with a block noise and a mosquito noise based on a decoded discrete cosine transformation(DCT) coefficient, and calibrating the obtained value. SOLUTION: When an image is intra-encoded (an I-picture of JPEG and MPEG), this algorithm can directly restore the image by using a decoded DCT coefficient. All encoding processing is converted into the DCT coefficient at first for inter-frame encoded video signals. A predication signal is converted into the DCT coefficient by a DCT part (FDCT) 10. The decoded DCT coefficient (difference) is added to the signal by an adder 16. Then, an artifact (block noise or mosquito noise) is removed by a reproduction algorithm (a reproducing part 18). Thus, an arithmetic operation including an optimal restoration equation for a block noise and a mosquito noise can be performed based on the decoded DCT coefficient, and the obtained value is calibrated.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

TECHNICAL FIELD The present invention relates to the processing of video signals,
In particular, it relates to the processing of noise in DCT-based video compression signals.

[0002]

2. Description of the Related Art DCT (Discrete Cosine Transform) for a pixel block is positioned as a main means of video compression and multimedia video storage because of its high compression performance. However, when the compression ratio exceeds a certain threshold, there is a problem that block distortion (block noise) appears. Heretofore, no practical means has been found for removing block noise in decoded images.

The compression of a video signal by motion-compensated DCT coding involves two types of distortion called mosquito noise and block noise. Conventional solutions for reducing mosquito noise have been developed based on an analysis of the properties of quantized DCT coefficients in both frequency and signal space.

[0004] Mosquito noise is a dominant compression artifact when the compression ratio is relatively low (10: 1 to 20: 1). As the compression ratio increases,
Spatial filters that dominate block noise and reduce mosquito noise gradually become ineffective because there is no effective means to distinguish block distortion from the image signal itself.

In recent studies, there are several solutions to this problem, but these solutions directly or indirectly identify noise by appropriate high-pass operations on the decoded image signal. It often fails, or at least is impractical, because of the basic prerequisite that it can be done.

[0006] To reduce or eliminate block noise, macroscopic signal processing on the entire image signal is required, in contrast to the microscopic signal processing of conventional solutions. This necessarily leads to a solution involving a matrix operation of the number of pixels of the whole image, but this is not practical. However, in the form of iterative microscopic calculations, a good solution can be derived, which leads directly to a good real-time ASIC / DSP solution.

[0007]

According to the present invention, there is provided a transform unit for decoding a DCT coefficient from a quantized value, and an optimal restoration equation for block noise and mosquito noise based on the DCT coefficient decoded by the transform unit. Provided is a compressed image processing apparatus including: a reproducing unit that performs an operation including the above;

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment including theoretical considerations will be described below. Reducing block noise requires slightly changing the DCT coefficients, which affects mosquito noise. Therefore, this problem is caused by the mosquito noise E
It is formulated as reducing the block noise D ² under the constraint that ² does not increase. This is represented by the following minimization:

[Number 1] min (D ² + λE ²⁾ Equation (1) where, lambda represents a multiplier for determining the balance of the block noise and mosquito noise in a minimized process.

[0009] The quantization of the DCT coefficient is performed by dividing the block into 8 × 8 blocks. Although the distribution of quantization noise in the image signal space is not uniform within each block, it is believed that the noise in one block of the image is independent of the noise in adjacent blocks. Therefore, the block noise D
² is most represented by the sum of the squares of the pixel value differences at the four block boundaries D _L ² , D _R ² , D _T ² , and D _B ² , as shown in FIG. 1 and equation (2). It is reasonable. FIG.
Indicates four block boundaries around the current block.
The quantization noise on both sides of one boundary is independent, and the relative difference is larger than the signal difference in the block.

The minimization of equation (1) is performed in the DCT coefficient space. Therefore, as shown in equations (3) and (4), D ²
Needs to be further expanded with the DCT coefficient F _uv . Equation (4)
Denotes a well-known discrete cosine basic vector.

Next, consider mosquito noise. To illustrate this description by way of example, FIG. 2 shows one-dimensional examples of DCT coefficients. When the height of the quantization step is higher than the minimum DCT coefficient, as shown in FIG.
Not noise. This results in visible artifacts in DCT-based image compression.

[0012]

(Equation 2)

In this example, the DCT coefficients of band 1 and band 2 are quantized and coded as 0. It is important to note, however, that the quantization noise in these bands is not the same as in bands not coded to zero.
The quantization noise in the band not coded to 0 is uniform, and the noise in the 0 coded band has relatively small energy. For this reason, when the height of the quantization step is reduced, the band that is not coded to 0 expands, and finally the 0 coded area is completely reduced, resulting in uniform white noise in both frequency space and signal space. Become. The non-uniform distribution of the quantization noise in the image block due to the quantization effect is the cause of the mosquito noise. The non-uniform distribution of noise in the signal space means that there are locations where the quantization noise is much smaller than in the rest of the block, and the DCT that affects the value of the signal at such locations. Changes in the coefficients inevitably degrade the signal rather than removing distortion. Equation (1)
In terms of the constraint on minimization of this, this state is the very form in which the power spectrum in the 0 coding band (band 1 and band 2) drops.

In order for the zero code power spectral band to be able to be estimated with good accuracy from the form of the non-zero encoded spectral segments, it is most useful if the missing segments are obtained by smoothing the spectrum which is not quantized to zero. preferable. A sufficient condition for the validity of this assumption is

(Equation 3) The right side of exists. In this equation, S (ω) and R
(X) represents the power spectrum and the autocorrelation function of the image signal, respectively. A sufficient condition for the power spectrum S (ω) to be continuous is from equation (5).

R (x) <σ ² x ⁻¹⁻⁰ Equation (6), where σ ² is the variance of the image signal. It is known that when the image signal is less noisy, the autocorrelation function is represented by the following model:

[0015]

(Equation 5)This satisfies the sufficiently required conditional expression (6). Suitable
For a first-order estimation of the proper form of the cost function
Is F that is not 0 ⁽⁰⁾ _{i, j}Uniform for coefficients with
(1.0) and 0 F⁽⁰⁾ _{i, j}For coefficients with
Must be inversely proportional to the smoothed or interpolated value.
is there. This is the function form of the weighting function represented by equation (8)
Lead to. In this case, the parameter κ is the quantization step
Which is half of the dimension of
Determine the position, g_{u, i}Is a coefficient of the smoothing filter. This
Acts on the DCT coefficient, and g_{u, i}Is constant
Or F⁽⁰⁾ _{i, j}Is determined according to the shape of the
Note that it may be specified.

[0016]

(Equation 6) Using this, the mosquito noise E ² is represented by the following equation.

[0017]

(Equation 7) Where F ⁽⁰⁾ _{i, j} represents the quantized DCT coefficients just decoded from the bit stream.

The direct solution of equation (1) is (number of pixels)
Not practical as it requires a matrix of size ^two . The following shows that a fast convergence solution can be created by applying the well-known Gauss and Seidel method of solving linear equations.

First _, an iterative coefficient updating process for the DCT coefficient F _{l, m} is defined as follows.

(Equation 8) Here, α _{l, m} ^(k) is D at the k-th iteration cycle.
This is the increment of the CT coefficient. When φ _{l, m} = 0 in equation (8), P _{l, m} = 0. For the moment, the matrix P is arbitrary.

Next, F _{u, v} in equation (1) is replaced with the right side of equation (10),

(Equation 9) Derives the optimal value of α _{l, m} ^(k) . The result is:

[0021]

(Equation 10) To simplify this iterative process, we introduce an alternative parameter that incorporates α _{l, m} ^(k) defined by

[0022]

[Equation 11] Here, P _{l, m} is defined as an (l, m) element (= 1) of a matrix in which all elements other than the element at (l, m) are zero. The choice of P _{l, m} is arbitrary, but is the only choice that allows the solution of equation (12) to be obtained without using matrix inversion.

Equations (13) and (14) give 1 for each image
Each time the DCT coefficients are updated, during which the updated coefficients are used to update blocks of successively occurring coefficients in the video. When all DCT coefficients in the image have been updated, the same process needs to be repeated until the minimization equation (1) converges. It will be shown later that the number of repetitions is about four.

[0024]

(Equation 12) At this time, the optimal value of α _{l, m} ^(k) is given by equation (15), where the following convenient relation is used to simplify the notation.

[0025]

(Equation 13) Next, the multiplier λ of the equation (1) is found.

First, stability conditions

[Equation 14] Apply This yields the following equation:

[0027]

(Equation 15) To derive an intuitive solution to equation (18), φ ^(k)
Suppose _{u, v} = φ ⁽⁰⁾ _{u, v} and φ ^(k) _{u, v} is not a function of F _{i, j} ^(k) . At this time, integrating equation (18) gives:

(Equation 16) Here, D ^{(0) 2} and D ^{(k) 2} represent the initial value and the convergence value of the block distortion.

Thus, it can be seen that λ is the ratio of the reduced block noise to the mosquito noise reduction.

[Equation 17] A value of λ much greater than 1.0 effectively removes mosquito noise but leaves some blockiness. On the other hand, values much less than 1.0 over-smooth block and signal boundaries. The optimal value of λ is near 1.0. λ can be varied according to the repetition, as well as the more detailed spectral information of the DCT coefficients within the block.

To test the performance of the new noise reduction algorithm, various test images were first compressed by DCT and quantization. The DC coefficients were not quantized, and the AC coefficients used a fixed quantization step. Extension to nonlinear quantization is straightforward. Next, equations (13) and (14) of the signal reproduction algorithm were repeatedly applied. FIG. 3 shows the target S
This shows an improvement in NR.

[0030]

(Equation 18) In this equation, fi _{, j} and f'i _{, j} represent the original signal and the reproduced image signal, respectively.

FIG. 3 shows the relationship between the improvement in SNR and the number of repetitions. After four iterations, the maximum noise reduction was achieved. The test sequence is SUSIE (frame # 70)
It is.

The SNR at iteration # 0 corresponds to the SNR of the decoded image itself, and iterations # 1 through # 10 represent the relationship between the number of iterations and the SMR improvement. It is also evident from the figure that this algorithm achieves the greatest improvement after 4 iterations, and this property was also observed in other test images.

FIG. 4 shows an example of a coding loop structure using the above-described novel signal reproduction algorithm for restoring an image including block distortion and mosquito noise in the DCT coefficient space. When the image is intra-encoded (JPE
G and MPEG I-pictures), this algorithm can directly recover the image using the decoded DCT coefficients. In an inter-frame coded video signal, all coded signals are first converted to DCT coefficients.
In FIG. 4, a DCT unit (FDCT) 10 converts a prediction signal into DCT coefficients. Then, in the adder 16,
The decoded DCT coefficients (differences) are added to these signals. Next, artifacts (block noise and mosquito noise) are removed by the above-described reproduction algorithm (reproduction unit 18). In the case of an intra-encoded image, there is no prediction signal.

FIG. 5 shows the structure of a decoder compatible with the encoder of FIG. Bit stream input is inverse quantization unit 3
The signal is added to the decoded DCT (difference) coefficient by the adder 32 via 0, and the reproduction unit 34 removes block noise and mosquito noise by the above-described reproduction algorithm.

FIG. 6 shows an example of the structure of a circuit for reproducing a signal outside the decoding loop. The decoded signal needs to be transformed into DCT coefficients, processed, and then back transformed into an image signal.
That is, the reproduction of the signal takes place outside of the coding loop, and thus the existing video compression standard, H.264. If complete compatibility with 20x and MPEG has to be maintained, the structure of the decoder needs to be as shown in FIG. The main difference from the decoder of FIG. 5 is that in the decoder of FIG. 5, both the QS (quantization step size) and the DCT coefficient are input from the inverse quantization unit 30 to the reproduction unit 34, but the structure of FIG. In Q
S is from the inverse quantization unit 30, and DCT coefficients are from the inverse DCT unit (I
DCT) 20.

Equations (13) and (1) of the signal reproduction algorithm
4) is a self-embedding iterative algorithm (self-embeddi
ng interative algorithm), and processes one frame of the image in a block-raster manner as shown in FIG. Then, the self-embedding form of the algorithm enables a spatially pipelined PE, and a desired processing speed can be realized as shown in FIG. 7B. As described above, in order for the restoration process to converge, it is necessary to repeat the entire processing for each frame four times. In this regard, compact and fast FDCT and IDCT cores can simultaneously be integral components for differential noise reduction techniques. An iterative solution allows the fast requirements to be met by a pipelined PE architecture. The processing results of (a) and (b) are exactly the same. The entire frame process needs to be repeated four times for the restoration to converge.

As mentioned above, DCT based video compression introduces two major distortions in the decoded image: mosquito noise and block noise. Reducing mosquito noise is microscopic in nature and, in fact, produces visually acceptable results with adaptive filtering in the image signal space.

However, block noise has been considered to be a more difficult problem to solve because of its macroscopic nature. In an embodiment, by applying Gauss and Seidel's method to the solution of the noise minimization problem, D
It has been described that an effective solution can be derived in the CT coefficient space. This is a reasonable solution since the quantization is performed in the DCT coefficient space and the important constraints (the mosquito noise condition in equation (9)) can be represented most effectively in this signal space.

On the other hand, the Gauss and Seidel method is not the only method for iterative solutions. Steepest descent and gradient methods can also be used, and it is important to understand the properties of block noise and mosquito noise and formulate them with optimal recovery equations.

Although the present invention has been described with reference to an embodiment as described above, the present invention is not limited to this.

[0041]

The present invention effectively reduces both mosquito noise and block noise, which are the main distortions of a decoded image in DCT-based video compression.

[Brief description of the drawings]

FIG. 1 illustrates four block boundaries around a current block.

FIG. 2 shows the relationship between DCT coefficients and quantization effects.

FIG. 3 is a diagram illustrating the effect of an algorithm used in the present invention.

FIG. 4 is a block diagram of a coding loop structure according to an embodiment of the present invention.

FIG. 5 is a block diagram of a decoding loop structure example compatible with the encoder of FIG. 4;

FIG. 6 is a block diagram of another example of a decoding structure.

FIG. 7 is a view for explaining the effect of the algorithm used in the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 DCT part 12 Quantization part 14 Inverse quantization part 16 Adder 18 Reproduction part 20 Inverse DCT part 22 Memory part 24 Motion compensation part 26 Subtractor

Claims (2)

[Claims] A transform unit for decoding a DCT coefficient from a quantized value; a reproducing unit for performing an operation including an optimal restoration equation for block noise and mosquito noise based on the DCT coefficient decoded by the transform unit; Means for calibrating the value obtained by the calculation in the reproduction unit to obtain an output signal with reduced noise. 2. The apparatus according to claim 1, wherein the operation performed by the reproducing unit includes a Gauss-Seidel iterative method.