JPH0746597A - Motion compensation predictive coding device and decoding device - Google Patents

Abstract

(57) [Abstract] [Purpose] Panning is detected only with horizontal and vertical two-dimensional vectors, and highly accurate motion vector detection and motion compensation can be obtained without significantly increasing the amount of calculation. A prediction image signal of one frame before is obtained from a vector switching and motion detector 107 with respect to an input image signal,
A prediction error signal, which is the difference between the input image signal and the predicted image signal, is obtained from the subtractor 109, and the coding unit (110 to 11).
Input in 2). The motion vector detector 104 detects an image motion vector for each divided block, and the representative motion vector detector 106 detects an image motion vector in an area of one frame or less. When the panning detector 105 detects panning, the transmission motion vector switch 119
The representative motion vector is given to the variable length encoder 112 so that it is encoded and transmitted. The decoding apparatus does not perform motion compensation for each divided block when the representative motion vector is transmitted.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a coding device and a decoding device which are used in a high-efficiency video coding system or the like and which use motion-compensated interframe prediction.

[0002]

2. Description of the Related Art An interframe predictive coding system is one of the techniques for compressing a video signal with high efficiency. This method utilizes the fact that the correlation between consecutive frames is generally high, and only the difference signal between the current frame and the previous frame is encoded to reduce temporal redundancy.

In particular, an inter-frame predictive coding system using motion compensation is often used for a moving image such as a television signal. 261 Recommendation and MP
Used in EG etc.

Generally, in these encoding systems, the input image data is divided into regions of 8 dots × 8 lines, 16 dots × 16 lines, etc. (hereinafter, this region is called a pixel block), and the pixel block data is divided. Image motion vector detection using vector representation in the horizontal and vertical directions is performed for each data, and motion compensation is performed according to the image motion vector.

In the so-called two-dimensional vector motion detection using only horizontal and vertical image motion vectors, a surface generally weak against rotations other than parallel motions of an object, such as zooming and panning, that is, The image movement may not be detected accurately.

As a means for improving the accuracy of motion compensation, a global motion compensation system has been proposed which uses multidimensional vectors to represent global motion caused by zooming and panning of a camera. (Reference: Ota, Miyamoto, Shimizu; 1
990 Autumn National Convention of the Institute of Power, Information and Communication Engineers D-225) By using a multi-dimensional vector by associating a dedicated vector for rotation, zooming, panning, etc. of an object that is not good at motion compensation using a two-dimensional vector. , It is reported that more accurate motion compensation can be achieved.

[0007]

In order to improve the accuracy of motion detection by motion compensation using horizontal and vertical two-dimensional motion vectors, the pixel unit (pixel block unit) for detecting the motion is made finer. There is a method. However, the number of blocks increases dramatically, and as a result, the number of image motion vectors (information amount) that must be transmitted also increases, resulting in a high transmission rate, which is not realistic. In addition, it is difficult to obtain an accurate vector by the block matching method in a flat area where the difference in luminance signal is small, and if motion compensation is performed using the vector, it causes deterioration of image quality.

Further, in the above-mentioned global motion compensation, more accurate motion compensation can be expected, but the amount of calculation for vector detection becomes enormous, and in order to perform the processing in real time, the hardware constituting the apparatus is required. The burden increases. Also parameters
As the number of data increases, the amount of transmitted information increases.

Therefore, the present invention detects a panning only with a horizontal and vertical two-dimensional vector, and is capable of highly accurate motion vector detection and motion compensation without increasing the calculation amount so much. An object is to provide an encoding device and a decoding device.

[0010]

According to the present invention, an input image signal and a reference image signal obtained by delaying the input image signal by one frame period are divided into a plurality of small areas, and each area of the input image signal is divided. From the image motion vector detected by the motion vector detecting means for calculating an image motion vector indicating the correlation with the reference image signal, and the image motion vector detected by the motion vector detecting means, it is determined whether or not the image of the input image signal is panning. A representative that calculates a representative value (hereinafter referred to as a representative motion vector) of the image motion vector for a certain region of one frame or less of the input image signal from the panning detection means to be performed and the image motion vector detected by the motion vector detection means. Based on the panning detection information from the motion vector determination means and the panning detection means, the image motion vector And a coding means for performing one of encoding of the representative motion vector. That is, according to the present invention, as preprocessing of the motion compensation predictive coding apparatus, means for obtaining an image motion vector in advance, means for performing panning detection of the frame from the obtained image motion vector, and representative motion vector from the obtained motion vector are obtained. It has a means to seek.

[0011]

By the above means, as preprocessing of the encoding device,
If a motion vector is obtained in advance by a method of pixel unit, pixel block unit, etc., and if it is recognized that panning is performed by performing panning detection in a smaller area obtained by dividing the entire frame or one frame from the obtained motion vector. Calculates the representative motion vector obtained. When it is detected that panning is performed, motion compensation is performed using the representative motion vector, and this vector is transmitted. At this time, the motion vector obtained for each pixel block is not used for motion compensation and is not transmitted. As a result, a stable image is obtained without image quality deterioration during panning.

[0012]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows an encoding apparatus according to the first embodiment of the present invention. CCITT, which is a standard method for video conference encoding
DCT used in the H261 recommended model
It is a hybrid coding device based on motion compensation and inter-frame prediction.

Digital image data is input to the input terminal 101. This image data is buffered by the input buffer 102 and subjected to format conversion (pixel block conversion, etc.) of the data, and then the frame memory 1
03 and the motion vector detector 104. The image data is delayed by a predetermined one frame period in the frame memory 103, and then the subtractor 109 and the motion vector detector 10
It is output to 4.

The motion vector detector 104 detects an image motion vector from the input image data and the image data delayed by one frame. The image motion vector detected in image block units is output to the panning detector 105 and the representative motion vector determiner 106. Panning detector 10
5, the variance of one frame of the input image motion vector is calculated, and when the variance is smaller than the set threshold value, the frame is considered to be panning, and the detection result is the vector switching and motion compensator 107. And to the transmission motion vector switch 119 and the variable length encoder 112. Further, the representative motion vector determiner 106 takes a vector average value from the input image motion vectors for one frame and outputs it to the vector switching / motion compensator 107 and the transmission motion vector switch 119 as a representative motion vector of the frame. Is output. Here, when the representative motion vector is calculated as 0 and when the panning detector 105 detects that the image is panning, the image is regarded as a still image and motion compensation is not performed.

In this embodiment, the area for obtaining the representative motion vector is defined as the entire frame, but this area is not limited to the entire frame, and the representative motion vector may be defined for each smaller area obtained by dividing one screen. good. This state is shown in FIG. In FIG. 7 (A), when the representative motion vector X is determined for one frame as a whole, in FIG. 7 (B), one frame is divided into two and representative motion vectors xA, x are respectively divided into areas A and B.
The case where B is set is shown. For example, when only the upper half of the input image moves in the horizontal direction substantially uniformly, the representative motion vector can be set more finely in FIG. 7B.

In the subtractor 109, the frame memory 103
In addition to the pixel block data from, the vector switching and motion compensator 107 also provides the motion-compensated pixel block data of the previous frame, and obtains a prediction error signal and outputs it to the DCT unit 110. The DCT unit 110 performs DCT (discrete cosine transform) on the input block data,
Output to the quantizer 111. DC in the quantizer 111
The output of the T unit 110 is quantized, the bit rate is reduced, and the variable length coder 112 and the dequantizer 115 are supplied. In the variable length encoder 112, image data, motion vector,
The panning detection information is encoded. At this time, when it is detected that panning is performed, the image motion vector is encoded only at the beginning of the frame, and the image motion vector for each block is not transmitted. The encoded data is output to the output buffer 113.

The dequantizer 115 dequantizes the data quantized by the quantizer 111 and outputs the dequantized data to the inverse DCT unit 116. The data subjected to the inverse DCT by the inverse DCT unit 116 is output to the adder 117. In the adder 7, the inverse DCT prediction error data, the vector switching and motion compensator 1
The motion-compensated data from one frame before from 07 is added to reproduce the predicted pixel block data of the current frame and output to the frame memory 118. In the frame memory 118, the input pixel block data is delayed by one frame period to be the previous frame data, which is output to the vector switching / motion compensator 107 and the motion vector detector 108.

The motion vector detector 108 stores the previous frame data from the frame memory 118 in the frame memory 1
Current frame data is given from 03. The motion vector detector 108 sequentially obtains an image motion vector between the previous frame and the current frame for the input block data and outputs it to the vector switching / compensation unit 107 and the transmission motion vector switching unit 119. In the vector switching and motion compensator 107, when it is detected that the frame is panning according to the output from the panning detector 105, motion compensation is performed using the image motion vector of the representative motion vector determiner 106 and panning is performed. If not detected, motion compensation is performed using the image motion vector of the motion vector detector 108. The motion-compensated pixel block data is added by the adder 1 as the prediction signal of the previous frame.
17, output to the subtractor 109.

In the transmission motion vector switch 119, the image motion vector from the motion vector detector 108 and the image motion vector from the representative motion vector determiner 106 are used as motion compensation vectors by the flag from the panning detector 105. It is selected and output to the variable length encoder 112, and is encoded together with the image data and panning detection information.

FIG. 2 shows a first embodiment of a decoding device corresponding to the above coding device. An image transmission signal is input to the input terminal 201, buffered by the input buffer 201, and then output to the variable length decoder 203.

The variable length decoder 204 decodes the image data and the vector data and outputs them to the inverse quantizer 204 and the motion vector separator / generator 211, respectively. The inverse quantizer 204 dequantizes the image data and outputs it to the inverse DCT unit 205. The image data is the inverse DCT unit 205.
Is inversely DCTed and output to the adder 206. Adder 2
In 06, the prediction error signal from the inverse DCT unit and the motion-compensated previous frame signal are added and output as the current signal to the frame memory 207 and the output buffer 208. The data buffered by the output buffer 208 is obtained as a constant rate video signal at the output terminal 209.

In the frame memory 207 to which the output of the adder 206 is supplied, the signal of the current frame is delayed by one frame time and output to the motion compensator 210. The motion vector separation / generator 211 extracts the image motion vector data from the output of the variable length decoder 203, and outputs it to the motion compensator 210. At this time, when this frame is motion-compensated with the representative motion vector for panning, since only one vector is transmitted in one frame, this one vector is spread to the vector for each block and the motion compensator 210 Output to.

In the motion compensator 210, motion vector separation
Motion compensation of the previous frame data is performed according to the vector output from the generator 211. Second, instead of transmitting only one vector per frame during panning,
A second embodiment in which a vector is transmitted for each pixel block will be described.

FIG. 3 shows an encoding device according to the second embodiment.
The same parts as those in the first embodiment are designated by the same reference numerals as those in FIG. 1 and their explanations are omitted. The panning vector from the representative motion vector determiner 106 and the motion vector obtained by the motion vector detector 308 for each normal pixel block are input to the motion vector switch 319. One of these vectors is selected by the panning detection flag from the panning detector 105 and is output to the motion compensator 307 and the variable length decoder 312.

The motion compensator 309 motion-compensates the previous frame data from the frame memory 118 according to the selected image motion vector and outputs it as a prediction signal. The variable length decoder 312 encodes the image motion vector received from the motion vector switch 319 together with the image data.

In the second embodiment, when panning is detected, the panning vector is used for motion compensation, but the panning vector is encoded in the same manner as the motion vector of each pixel block. Therefore, the decoding device implements a decoding device compatible with panning without adding any special function.

FIG. 4 shows a third embodiment in which edge detection is incorporated as a method for improving the panning detection accuracy. The image data converted into a signal for each pixel block in the input buffer 102 is output to the frame memory 103, the edge detector 401, and the pixel block extractor with edge 402. The edge detector 401 detects whether or not there is an area with a large luminance difference in the input image data pixel block, and if there is, the information that the pixel block has an edge is determined as the pixel block with edge detector 402.
Output to. The pixel block with edge extractor 402 extracts only the block with edge and outputs it to the edge motion vector detector 403.

The edge motion vector detector 403 detects the image motion vector of only the pixel block in which an edge is detected, and outputs it to the panning detector 105 and the representative motion vector determiner 106.

In the third embodiment, since the image motion vector obtained by the preprocessing is obtained only by the pixel block having the edge, the accuracy of the panning detection is higher than that in the case of judging from the motion vector in all the blocks. Is improved.

FIG. 5 is an explanatory diagram of an embodiment of a method for panning detection and representative motion vector determination. Here, the preprocessing unit of the encoding apparatus of the present invention obtains the motion vector in pixel block units of 16 dots × 16 lines, and the area surrounded by the diagonal lines is the pixel block. Also,
It is assumed that a representative motion vector is obtained in an area of 128 dots × 80 lines in which 40 blocks of 16 dots × 16 lines are collected as shown in FIG. When 40 image motion vectors V0 to V39 are obtained in a pixel block of 16 dots × 16 lines, panning detection is determined using the variance of these vectors.

[0031]

[Equation 1]

It is When this variance σ ² is smaller than a predetermined threshold σth ² , this region is considered to be panned, and when it is σth ² or more, this region is considered not to be panned.

The representative motion vector of this area is the average value Va.
And Also, when detecting that it is a still image,
Detect both variance and average. When variance sigma ² is smaller than? Th ^2, and the absolute value of the average value Va is calculated as zero,
Considered to be a still image.

FIG. 6 illustrates the configurations of the panning detector and the representative motion vector determiner of the above embodiment. A representative motion vector determiner 601 surrounded by a dotted line is a panning detector.

The detected image motion vector is input to the terminal 603 and output to the memories 604 and 606. The motion vector average value calculator 605 controls the address of the memory 604 so as to read out the image motion vector corresponding to the screen area for which the average value is to be calculated. Memory 604
With respect to the image motion vector output from, the average value of the vectors in the area is calculated by the motion vector average value calculator 605, and the average value of the image motion vectors is output as the representative motion vector.

The motion vector variance calculator 608 is stored in the memory 6
The address control of 06 is performed so that the motion vector corresponding to the screen area for which the variance value is to be obtained is read.
The motion vector output from the memory 606 is the subtractor 60.
7 and outputs to the motion vector variance calculator 608 after subtraction from the motion vector average value. The motion vector variance calculator 608 calculates the variance of the motion vector in the area and outputs it to the variance value comparator 609. Variance value comparator 60
9 compares the motion vector variance and the variance threshold,
Outputs the panning detection flag.

As described above, according to this system, it is possible to detect an image motion vector that copes with panning without increasing the calculation amount, while using the two-dimensional vector. When panning is performed in the first embodiment, the amount of motion vectors generated can be suppressed as compared with the case of normal transmission, so that the image quality is improved.

In a flat area where the difference in luminance is small, it is difficult to detect an accurate vector, but in the present invention, motion is uniformly compensated when it is detected that panning occurs. Does not cause image quality deterioration. Therefore, when a panning image is input, the performance of motion-compensated interframe prediction is improved, and higher-quality digital video transmission becomes possible.

[0039]

As described above, according to the present invention,
It is possible to detect panning using only horizontal and vertical two-dimensional vectors and to perform highly accurate motion vector detection and motion compensation without increasing the amount of calculation.

[Brief description of drawings]

FIG. 1 is a diagram showing a first embodiment of an encoding device of the present invention.

2 is a first decoding device corresponding to the encoding device of FIG. 1;
FIG.

FIG. 3 is a diagram showing a second embodiment of the encoding device of the present invention.

FIG. 4 is a diagram showing a third embodiment of the encoding device of the present invention.

FIG. 5 is an explanatory diagram of panning detection, representative motion vector determination, and still image determination according to the present invention.

FIG. 6 is a configuration diagram of a panning detector and a representative motion vector determiner according to the present invention.

7 is an explanatory diagram for explaining an operation example of the panning detector in FIG. 1. FIG.

[Explanation of symbols]

102 ... Input buffer, 103 ... Frame memory, 10
4 ... Motion vector detector, 105 ... Panning detector,
106 ... Representative motion vector determiner, 107 ... Vector switching and motion compensator, 108 ... Motion vector detector, 11
0 ... DCT device, 111 ... Quantizer, 112 ... Variable length encoder, 113 ... Output buffer, 115 ... Inverse quantizer, 1
16 ... Inverse DCT device, 117 ... Adder, 118 ... Frame memory, 119 ... Transmission motion vector switching device, 203 ... Input buffer, 203 ... Variable length decoder, 204 ... Inverse quantizer, 205 ... Inverse DCT device, 206 ... Adder, 207 ...
Frame memory, 208 ... Output buffer, 210 ... Motion compensator, 211 ... Motion vector separation / generator, 308 ...
Motion vector, 309 ... Motion compensator, 319 ... Transmission motion vector switch.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Tatsuya Ishikawa 8 Shinsita-cho, Isogo-ku, Yokohama, Kanagawa Prefecture

Claims (6)

[Claims] 1. An input image signal and a reference image signal obtained by delaying the input image signal by one frame period are divided into a plurality of small regions, and the correlation with the reference image signal is obtained for each region of the input image signal. Motion vector detecting means for calculating the image motion vector shown, panning detecting means for judging whether or not the image of the input image signal is panning from the image motion vector detected by the motion vector detecting means, and the motion Representative motion vector determination means for calculating a representative value (hereinafter referred to as representative motion vector) of the image motion vector for a certain region of one frame or less of the input image signal from the image motion vector detected by the vector detection means, and the panning. Either the image motion vector or the representative motion vector, based on the panning detection information from the detection means. A motion-compensated predictive coding apparatus, comprising: a coding means for coding either one of them. 2. The encoding means, when it is detected from the output result of the panning detection means that the input image is panning, the representative motion vector calculated by the representative motion vector determination means is used as the input image. Is detected as not being panned, the image motion vector obtained for each image block data of 8 dots × 8 lines or 16 dots × 16 lines is subjected to variable length coding. Item 1. The motion compensation predictive coding device according to Item 1. 3. The encoding means, when the input image is detected to be panned by the panning detection information, performs variable length encoding of only one image motion vector for one area for which a representative motion vector is obtained. The motion compensation predictive coding apparatus according to claim 1, further comprising: 4. The encoding means, when it is detected that an input image is panned, with respect to one area for which the representative motion vector is obtained, motion-compensated image block data existing in this area. 2. The motion compensation predictive coding apparatus according to claim 1, further comprising means for variable-length coding a representative motion vector for each pixel block data as the motion vector. 5. The prediction means for generating a prediction signal for generating a prediction signal in which the reference image signal of one frame before is motion-compensated according to a motion vector, and a still image as the input image signal. 7. A means for stopping the motion compensating operation of the prediction signal generating means when the inputted image motion vector obtained by the motion vector detecting means indicates a still image. 1. The motion compensation predictive coding device according to 1. 6. An input image signal and a reference image signal obtained by delaying the input image signal by one frame period are divided into a plurality of small regions, and the correlation with the reference image signal is obtained for each region of the input image signal. The image motion vector shown is calculated, and it is determined from the image motion vector whether or not the image of the input image signal is panned to obtain panning detection information. From the image motion vector, there is one frame or less of the input image signal. Motion-compensated prediction in which a representative value of image motion vectors (hereinafter referred to as representative motion vector) for a region is calculated, and either the image motion vector or the representative motion vector is coded and transmitted based on the panning detection information. In a receiving device for receiving a transmission signal from an encoding device, a decoding means for variable length decoding the transmission signal, and variable length decoding When the motion vector component is separated from the No. signal, and it is the representative motion vector for panning, the representative motion vector is added to the image motion compensator that performs the motion compensation of the motion compensation pixel block in the area corresponding to the representative motion vector. And a means for performing image motion compensation, the motion compensation predictive decoding apparatus.