JPH037190B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a predictive coding apparatus for television signals.

The predictive coding method is based on the operating principle of reducing the amount of transmitted information by transmitting the difference between the input signal to be transmitted and the predicted signal, that is, the prediction error signal, and the present invention is a prediction method that provides this predicted signal. This is a predictive coding method that uses a large number of methods and selects the optimal one from among them.

For example, with interframe coding, prediction error signals with large amplitudes occur less frequently for still images or quasi-still images that contain little movement, so the amount of information generated is small; The amount of generated information increases for videos that contain a lot of information. That is, in interframe encoding, the less motion there is, the better the encoding efficiency is, but the more motion there is, the lower the efficiency is. Therefore, attempts have been made to improve efficiency even when motion is involved.
For example, the motion contained in television signals is often considered to be translational, so interframe prediction that takes into account the amount of change in the subject's position between frames, or "motion compensation" interframe prediction, is used for video. This is the most effective method for achieving high coding efficiency. This "motion compensation"
In the interframe prediction method, many prediction methods are used to maintain high efficiency against various motions. Therefore, the "motion compensated" interframe prediction method can be rephrased as a prediction method that adaptively determines the optimal prediction method from among a large number of prediction methods.

Here, the principle of this "motion compensation" will be briefly explained.

Assume that, as shown in FIG. 1, a figure located near coordinates (x ₀ , y ₀ ) at time t=t ₀ moves to (x ₁ , y ₁ ) after one frame time (τ). At this time, in normal interframe prediction, t=
Pixels in the figure near (x ₀ , y ₀ ) at t=t ₀ are used to predict pixels in the figure near (x ₀ , y ₀ ) at t ₀ +τ. Therefore, as is clear from FIG. 1, non-zero prediction errors occur near both points (x ₀ , y ₀ ) and (x ₁ , y ₁ ) at t=t ₀ +τ.

Here, if we can somehow detect the amount of displacement of the figure from (x ₀ , y ₀ ) to (x ₁ , y ₁ ), then the figure near (x ₀ , y ₀ ) at t=t ₀ using t
The figure near (x ₁ , y ₁ ) at =t ₀ +τ can be predicted, and the amount of generated information is greatly reduced. This is the principle of so-called "motion compensation."

As a method for detecting this amount of displacement, that is, a method for determining the optimum prediction method in units of blocks consisting of multiple pixels, for example, the 1978 IEICE Technical Research Report
A paper by Ninomiya published in Vol.78No.39 "Motion compensation in interframe coding" (paper number
IE78-6) is applicable.

In this paper, applying ``motion compensation'' (referred to as ``motion correction'' in this paper) to video pixels reduces the amount of information generated by approximately half of the amount of information generated using only interframe coding. It has been shown that information compression as wide as possible is possible.

If a television signal transmission device is configured using "motion compensation" with such high coding efficiency, there will be no problem if the transmission speed of the transmission path is high, but if the transmission speed is low, It may be difficult to transmit both the information representing the optimal prediction method detected by "motion compensation" and the prediction error signal.

In conventional interframe predictive coding devices that do not use "motion compensation," when the amount of generated information of the input television signal is considerably larger than the set transmission rate, the frame (or field) is
By temporarily suspending the encoding operation in units, the amount of generated information is reduced and speed matching with the transmission speed of the transmission path is achieved.

However, this temporary suspension of the encoding operation in units of frames (or fields) causes jerky and unnatural motion, commonly known as jerkiness, in subsequently reproduced images. In other words, this unnatural motion occurs because a missing frame (or field) in which the prediction error was not transmitted is simply replaced with the immediately preceding frame (or field) that was transmitted. As shown in the figure, a figure that smoothly moves upward in the original signal as shown in FIG. 2a becomes unnatural when the i-th frame is simply replaced by the (i+1)th frame as shown in FIG. 2b.

In this respect, the optimal prediction method in the "motion compensation" interframe coding method essentially includes information regarding the direction and speed of movement. In other words, when performing motion-compensated interframe prediction of frame (i+1) using the previous frame (number i), in the prediction signal, the moving area is displaced by the amount indicated by the motion vector in block units. , assuming an ideal case where the temporary motion compensation is perfect and the prediction error is zero, the moving area in this predicted signal is located at the position of the moving object in the original signal at frame number (i+1). become. It can be considered that this deviation from the ideal state is the cause of a non-zero prediction error signal. The higher the motion vector detection accuracy, the closer to this ideal state. Therefore, using the motion information included in the optimal prediction method, according to the prior art, frames (or fields) are simply replaced as shown in FIG. 2b, as shown by broken lines in FIG. 2c. By shifting the moving parts by the amount of movement, smooth movement as shown in Figure 2a can be reproduced.

That is, an object of the present invention is to eliminate the above-mentioned unnatural motion that is likely to occur in television signal transmission at low transmission speeds by using information representing the optimal prediction method obtained in a coding method that uses "motion compensation." An object of the present invention is to provide an encoding device that reduces the number of problems. That is, the present invention uses a plurality of prediction methods including inter-frame prediction, and determines the optimum prediction method among the plurality of prediction methods for each block of a plurality of pixels. In predictive coding, means for detecting the optimum prediction method, prediction signal generation means for generating a prediction signal corresponding to the optimum prediction method, error signal generation means for generating a prediction error signal for the prediction signal, and the error signal generation means. quantization means capable of fixing the output level of the means, encoding control means capable of instructing to fix each output of the optimum prediction method detection means and the quantization means, the optimum prediction method detection means; means for compression-encoding the output of the quantization means; a buffer memory for matching the generation speed of the compressed-encoded information with the output speed of the information to the outside; and transmitting the operating status to the encoding control means; 1. A television signal encoding device comprising:

In the present invention, when the amount of generated information is considerably large compared to the transmission speed of the transmission path, "motion compensation" is applied.
With conventional interframe coding that does not use interframe coding, the coding operation would temporarily stop on a frame (or field) basis, but before suddenly stopping coding, only the information indicating the optimal prediction method is sent. This data is used to reproduce smooth motion on the receiving side. In other words, (1) normally information indicating the optimal prediction method and a prediction error signal are transmitted, but (2) as the amount of generated information increases, the prediction error signal is not transmitted, and (3) when the amount of generated information further increases, the optimal prediction method is transmitted. Encoding is controlled so that neither the information indicating the prediction method nor the prediction error signal is transmitted.

On the receiving side, (1) normally the image is reproduced using the transmitted information indicating the optimal prediction signal and the prediction error signal, but (2) the amount of generated information increases and only the information indicating the optimal prediction method is transmitted. If not, decoding is performed assuming that all prediction error signals are zero, and (3)
If both are not transmitted, they are replaced by the previous frame (or field).

By doing this, it is possible to provide high-quality images with natural movements, such as expanding the range in which smooth images can be reproduced even for images in which conventionally only unnatural movements could be reproduced. is extremely large.

Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 3 shows an embodiment of an encoding device and a decoding device according to the present invention.

Input television signals are on lines 1010, 1011
The signal is supplied to the delay circuit 10 and the optimal prediction determination circuit 11, respectively. The delay circuit 10 delays the input television signal by the time required for determining the optimal prediction method in the optimal prediction determination circuit 11, and the subtraction circuit 12
supply to The subtraction circuit 12 performs subtraction between this delayed input television signal and the optimal prediction signal supplied from the variable delay circuit 13 via the line 1312, and generates a difference signal, that is, a prediction error signal, which is quantized. 18. This quantizer 18 has a plurality of quantization characteristics,
One of them can output a fixed value regardless of the value of the input prediction error signal. In the present invention, this fixed value is, for example, zero.

Selection of the plurality of quantization characteristics is by a quantization characteristic selection signal provided via line 1618. The output of the quantizer 18 is supplied to the compression encoding circuit 17 and the addition circuit 14. The adder circuit 14 sums the output of the variable delay circuit 13, that is, the optimal prediction signal, and the output of the minimizer 18, that is, the quantized prediction error signal, and outputs this addition result as a locally decoded signal to the frame memory 15. supply The frame memory 15 can store approximately one frame of the television signal, and can store approximately one locally decoded signal.
Lines 1511 and 1513 are delayed by frame time.
The optimal prediction determination circuit 11 and the variable delay circuit 1
3 respectively. The optimal prediction judgment circuit 11 is line 1
The optimal prediction scheme is determined using the input television signal provided via line 1511, the frame memory 15 output signal provided via line 1511, and the prediction scheme control signal provided via line 1611.

FIG. 4 shows an example of the configuration of the optimal prediction determination circuit 11.

The comparison circuit 110 first detects the prediction method showing the minimum prediction error amount based on the comparison of the prediction error amounts, and the configuration of the comparison circuit 110 is, for example, the motion vector detection circuit described in FIG. 2 of the paper by Ninomiya. is applicable. (The motion vector in this paper corresponds to the optimal prediction method in the present invention.) Normally, the prediction method that shows the minimum amount of prediction error is the optimal prediction method, but when the amount of generated information becomes extremely large, information indicating the optimal prediction method If even transmission is not possible, the output of the gate circuit 111 operated by the prediction method control signal supplied via the line 1611 is fixedly interframe prediction. At this time, the output of information indicating the optimal prediction method is fixed. Return to the main topic. Information indicating the optimal prediction method, which is the determination result of the optimal prediction determination circuit 11, is indicated by a line 111.
3 and a line 1117 to the variable delay circuit 13 and the compression encoding circuit 17, respectively. In the variable delay circuit 13, the output signal of the frame memory 15 is temporarily stored, and when information indicating the optimum prediction method is supplied, the predicted signal is read out from the corresponding address. This predicted signal is the optimal predicted signal. The variable delay circuit 13 is basically constituted by a memory that uses a random access memory and can be read and written independently of each other.

In the compression encoding circuit 17, the sufficiency S, which is a parameter indicating the operating state of the buffer memory 100, which will be described later, a quantized prediction error signal, and information indicating the optimal prediction method are compressed using an unequal length encoding method. However, normally both vertical and horizontal synchronization signals are compressed and encoded together with these signals. The compression encoding circuit 17 then outputs these compressed codes to the buffer memory 100. The buffer memory 100 smoothes irregularly generated information from the compression encoding circuit 17, matches the transfer rate of the transmission line 1000, and outputs the smoothed information. The sufficiency S of the buffer memory 100 is also supplied to the encoding control circuit 16 via a line 1016. Here, the sufficiency S is, for example, the difference between the write address and the read address.

Note that the transmission line 1000 can also be replaced with a digital recording medium such as a magnetic tape or a magnetic disk. Next, the operation of the encoding control circuit 16 will be explained with reference to FIGS. 3, 4, and 5.

Note that FIG. 5 is a diagram for explaining the control operation of the encoding control circuit 16 shown in FIG. 3.

The maximum amount of buffer memory sufficiency S supplied to the encoding control circuit 16 is 1, the threshold value related to the quantization characteristic selection signal supplied to the quantizer 18 via the line 1618 is T ₁ , and the threshold value related to the quantization characteristic selection signal is supplied to the quantizer 18 via the line 1611. The threshold value related to the prediction method control signal supplied to the optimal prediction determination circuit 11 is assumed to be _T2 . However, T1 and T2 are 0<T1<T2
<1. At this time, if the value of S is within the range of 0≦S<T1, both the information V indicating the optimal prediction method and the quantized prediction error signal e^ are transmitted (V
+e^), when the value of S increases and is within the range of T1≦S<T2, the amount of generated information per unit time is greater than the amount of information output to the transmission path per unit time, reducing the amount of generated information. Since it is necessary to do so, only V is encoded, and the output for e^ is always set to zero level. Even so, when the value of S increases to the range T2≦S≦1, it is determined that the encoding operation is no longer possible, and interframe prediction is fixedly regarded as the optimal prediction method, and T2
Only information indicating that S is within the range of ≦S≦1 is compressed and encoded, and neither V nor e^ is compressed and encoded (stopped). At this time, the movement of the image is temporarily frozen until S<T2.

In other words, for S where 0≦S<T1, the quantization characteristic selection signal instructs the selection of a characteristic from among the quantization characteristics other than the characteristic that can output a fixed zero level, and the prediction method control signal is as shown in FIG. Comparison circuit 110
The gate circuit 111 is instructed to use the output of the optimal prediction determination circuit 11 as the output.

When S exists in T1≦S<T2, the quantization characteristic selection signal instructs selection of a quantization characteristic that can output a fixed zero level, and the prediction method control signal is transmitted to the comparison circuit 110.
The output of the optimal prediction determination circuit 11 is instructed to be set as the output of the optimal prediction determination circuit 11. When S exists in T2≦S≦1, the quantization characteristic selection signal 1618 instructs the selection of a quantization characteristic that can permanently set the output to zero level, and the prediction method control signal 1611 instructs the gate circuit 111 to select a quantization characteristic that can permanently set the output to zero level. The interframe prediction is instructed to be output from the optimal prediction determination circuit 11. The quantization characteristic selection signal is supplied via a line 1618, and the prediction method control signal is supplied via a line 1611 to the quantizer 18 and the optimal prediction determination circuit 11, respectively. Furthermore, both these signals and the sufficiency S are sent to the compression encoding circuit 1 via a line 1617.
7. When T2≦S≦1, the fact that S is within this range is encoded once every unit time (for example, several lines, fields, or frames) of encoding control performed using the value of S, and V, There is no need to encode e^ each time.

Next, the decoding device shown in FIG. 3 will be explained.

Encoded information supplied from the transmission path 1000 or the recording medium is first matched by the buffer memory 200 on the receiving side to speed match the decoding operations on the transmission path 1000 and the receiving side, and then is supplied to the code expansion circuit 24. . The code expansion circuit 24 separates both the vertical and horizontal synchronization signals of the television signal, and then extracts information V indicating the optimal prediction method and a prediction error signal e^.
is decoded and decompressed. Decoded and expanded V
is sent to the variable delay circuit 23 via a line 2423, and the decoded and expanded e^ is sent to the adder 2 via a line 2422.
2, respectively. The variable delay circuit 23 generates an optimal prediction signal based on V and supplies it to the adder 23. The adder 23 generates a decoded signal by adding the optimal predicted signal and the decoded and expanded e^.

This decoded signal is output from the decoding device as a reproduced television signal, but is also supplied to a frame memory 25 that can store approximately one frame of the television signal. The decoded signal delayed by approximately one frame time in the frame memory 25 is transferred to a variable delay circuit 2 in order to generate an optimal predicted signal.
3. When S is within the range T2≦S≦1, the encoded information is decoded by encoding that S is within this range once every unit time of encoding control. After detection, within this unit time, the optimal prediction method is interframe prediction, and the prediction error signal e^ is fixedly set to zero to perform decoding.

The adder 22, variable delay circuit 23, and frame memory 25 in the decoding device may all have the same configuration as the adder 14, variable delay circuit 13, and frame memory 15 in the encoding device.

A modified example of the embodiment of the present invention will be described next.

As explained in the embodiment, the encoding operation is controlled by the value of the sufficiency S of the buffer memory 100. T1 and T2 were used for explanation as threshold values at this time. In actual encoding operations, even if S becomes a little smaller once a certain encoding state is reached, it is often not possible to immediately switch to another encoding state in which the amount of generated information is larger. Therefore, at this time, the values of T1 and T2 are changed depending on whether S is increasing or decreasing. For example, when the trend is increasing, T1′ is the threshold value T1, and when the trend is decreasing, T1′ has the relationship of T1′<T1, T2′<T2,
T2′ can be used.

[Brief explanation of the drawing]

FIG. 1 is a diagram for explaining the principle of "motion compensation", and FIG. 2 is a diagram for explaining the principle of the present invention. Third
4 is a diagram showing an example of the configuration of the encoding/decoding device in the embodiment of the present invention, and FIG. 5 is a diagram explaining the control operation of the encoding control circuit 16 shown in FIG. 3. be. In the figure, 10 is a delay circuit, 11 is an optimal prediction determination circuit, 12 is a subtracter, 13 is a variable delay circuit, 14 is an adder, 15 is a frame memory, 16 is an encoding control circuit, 17 is a compression encoding circuit, 18 is a quantizer, 100 and 200 are buffer memories,
22 is an adder, 23 is a variable delay circuit, 24 is a code expansion circuit, 25 is a frame memory, 110 is a comparison circuit, and 111 is a gate circuit.

Claims (1)

[Claims] 1. A television that uses a plurality of prediction methods that utilize correlation in the time axis direction, including inter-frame prediction, and determines one optimal prediction method among the plurality of prediction methods for each block consisting of a plurality of pixels. In the optimal predictive coding of the prediction signal, means for detecting the optimal prediction method, prediction signal generation means for generating a prediction signal corresponding to the optimal prediction method, and error signal generation means for generating a prediction error signal for the prediction signal. quantization means capable of fixing the output level of the error signal generation means; and encoding capable of instructing each output of the optimum prediction method detection means and the quantization means to be fixed respectively. a control means, a means for compressing and encoding the outputs of the optimal prediction method detection means and the quantization means, and adjusting the generation speed of the compressed and encoded information and the output speed of the information to the outside to match the operation. 1. A television signal encoding device comprising: a buffer memory for transmitting a status to said encoding control means.