JPH08163575A - Method and device for decoding picture - Google Patents

Abstract

PURPOSE: To provide a method and a device for decoding pictures capable of obtaining picture output with less noise even when an I frame is not provided inside a sequence. CONSTITUTION: At the time of predicting picture frames from plural prediction frames and decoding encoded picture data, the first P1 frame and the next B2 frame of ant input bit stream 100 are the frames predicted from the two frames that are a P0 frame before being inputted and the P1 frame inputted at first originally in a movement compensation picture reproducing part 60. Thus, for the next B2 frame of the first P1 frame, the first P1 frame is read from a frame memory, another reference picture is defined as entirely gray instead of the uninputted P0 frame and reproduction is performed in the movement compensation picture reproducing part 60. For the second P4 and succeeding frames, a processing similar to the normal processing is performed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image decoding method and device for decoding encoded image data.

[0002]

2. Description of the Related Art When transmitting or storing digitally represented image data, encoding is performed to reduce the amount of data. As an encoding method, there is a method of reducing redundancy by utilizing temporal or spatial correlation of image information (image data).

As a method of utilizing temporal correlation, there is a method of encoding a difference between two consecutive pixels (frames) or detecting motion of an image to perform motion compensation. As a method of utilizing the spatial correlation, the image is divided into blocks of a predetermined size (for example, 8 pixels in each of the vertical direction and the horizontal method), the data in each block is orthogonally converted, and the conversion coefficient is scan-converted. However, there is one that performs variable length coding (for example, rearranges in order from low frequency components to high frequency components). A pixel coding method (hereinafter abbreviated as MPEG2), which is being standardized by the Moving Picture Experts Group (MPEG), uses both of the above two methods in combination.
The provisional recommendation for MPEG2 is "Generic Coding of Moving Pi
ISO / IEC 13818 entitled "Ctures and Associated Audio"
-2.

Since the present invention can be applied to the process of decoding all MPEG2 images, the decoding process will be described.

FIG. 5 is a block diagram showing an example of a conventional structure of an image decoding apparatus for decoding data encoded by such a method. In FIG. 5, the buffer control unit 1,
Decoding processing is executed by the variable length decoder 2, the scan converter 3, the inverse quantizer 4, the inverse DCT unit 5, and the motion compensation image reproducing unit 6. Reference numeral 50 denotes a memory, which includes a buffer memory 51 and a frame memory (three I, P, and B frame memories described later) 52, 53, and 54. Also, 100
Is the input bitstream representing the encoded image,
Reference numeral 200 indicates a reproduced image.

Next, the operation will be described. The input bit stream 100 is controlled by the buffer control unit 1.
The data 40 is stored in the buffer memory 51. The data 41 read from the buffer memory 51 is variable length decoded by the variable length decoder 2.

Although not all data is variable length coded, it is assumed that fixed length code is also decoded by this variable length decoder 2. Next, the order of the data is rearranged by the scan converter 3 and then dequantized by the dequantizer 4. Next, the inverse DCT unit 5 performs inverse discrete cosine transform. The motion-compensated image reproduction unit 6 performs reproduction in consideration of the movement of the image. In MPEG2, a temporally intermediate frame (here, B frame) is predicted from both a temporally previous frame (here, I frame) and a temporally later frame (here, P frame). Therefore, to reproduce the B frame, the I frame and P that have been decoded in advance are used.
It is necessary to read the predicted frame data 42 and 43 of a frame from the frame memories 52 and 53 (in MPEG2, a P frame that is temporally later is decoded before a B frame). Predicted frame data 42, 43 and inverse DCT
The B frame is reproduced by the motion compensation image reproducing unit 6 according to the prediction error output from the unit 5, and is written in the frame memory 54 as reproduced pixel data 44. Frame memory 52,
The I, P, and B frames in 53 and 54 are read from each memory in a predetermined order (the data 45 of the B frame is read in FIG. 5), and the reproduced image 200 is output.

The present invention can be applied to an apparatus for processing all MPEG2 images, but as an example, consider the case of reproducing an NTSC image. One frame of an NTSC image consists of 720 horizontal pixels and 480 vertical lines, as shown in FIG. This is divided horizontally and vertically into 16 pixels each. 1
A unit of division is called a macroblock (hereinafter abbreviated as MB). The NTSC image is divided into 45 MB horizontally, 30 MB vertically, and 1350 MB in total. In MPEG2, an aggregate of MBs enclosed in one horizontal row is called a slice, and is called an NTS.
The C image is divided into at least 30 slices.

In MPEG2, the types of MB are roughly divided into two types, an intra MB and an inter MB. The intra MB is an MB coded only with the image data in the MB, while the inter MB is , The MB in which the differences from the reproduced images of the previous and / or future frames in display order are encoded. In addition, inter MB,
In addition to the difference signal, a motion vector representing the amount of movement of the image from the previous or future reproduced image in display order is also added.

MPEG2 input bitstream 100
Has a hierarchical structure, and from the upper layer, the sequence layer, the group of picture layer, the picture layer, the slice layer, and the MB
Layers and block layers are called. The picture layer is a layer forming one image, and the group of picture layer is composed of a plurality of picture layers starting with an I frame. The sequence layer is a series of pictures, and usually the sequence layer is composed of a plurality of group of picture layers.
In some cases, there is no group of picture layer and the picture layer is composed of multiple picture layers. When starting decoding, if the sequence layer includes a group of picture layers, the first picture in the sequence layer is always an I frame, so decoding can be started immediately without the need for a reference image. However, if the sequence layer does not include a group of picture layer, the first picture in the sequence layer may be either an I frame or a P frame, and if it starts with a P frame, decoding is started without a reference image. Either, or the decoding is not started until the I frame of the reference image is input.

The data amount of I frame is P frame, B frame
The number of frames is significantly larger than that of frames, and it takes a lot of time to store them in the buffer memory 51. Therefore, depending on the application, a sequence that does not include an I frame may be adopted. For example, assuming a broadcast application, an example of using a sequence composed of only P frames and B frames without I frames can be considered in order to reduce variations in the data amount of frames to be transmitted. In addition, assuming an image communication application, it is required to decode the transmitted stream with a short delay time, so that in addition to I frames, P frames that do not include B frames with reorder delays are included.
An example of using a sequence composed of only frames can be considered.

FIG. 7 shows an example of an image decoding apparatus for decoding a sequence composed of only P and B frames which does not include I frame. In FIG. 7, frame memories 152 and 53 are frame memories that store P frame image data 142 and predicted frame data 43 that have been decoded in advance to reproduce a B frame. The decoding process is the same as the above-described decoding procedure including the I frame and the process other than the selection of the reference image.
Decoding is performed by using a frame as a reference image, and a B frame is decoded by using a temporally preceding P frame and a temporally subsequent P frame as reference images. In FIG. 7, the same symbols as those in FIG. 5 indicate the same parts.

FIG. 8 is an example of an image decoding apparatus for decoding a sequence composed of only P frames which does not include I frames or B frames. In FIG. 8, a frame memory 252 is a frame memory that stores image data 242 of a temporally previous P frame that has been decoded in advance to reproduce the next P frame. The decoding process is the same as the above-described decoding procedure that does not include the I frame and the process of the B frame is removed, and the P frame is predicted from the P frame that is temporally preceding. In FIG. 8, the same symbols as those in FIG. 5 indicate the same parts.

In the coding using inter-frame prediction as in MPEG2, since the previously decoded image is used as the reference image, error components are accumulated and the image cannot be restored correctly. Therefore, in MPEG2, an image is normally refreshed in an intra-coded frame, that is, an I frame. However, as described above, a sequence that does not include an I frame cannot be refreshed by an I frame, so the intra M
A slice configured only with B, that is, an intra slice may be used. In the I frame, the entire screen is refreshed, while the intra slice refreshes a part of the screen, that is, a slice portion of the intra slice. For example, in a sequence including only P frames, the entire screen can be refreshed in several frames by changing the position of the intra slice for each frame (screen) as shown in FIG.

Further, when one bitstream is selected from a plurality of video bitstreams for decoding, when the video sequence is switched, decoding is usually performed until a sequence header or a group of picture header appears in the switched bitstream. Do not start. This is because the sequence header contains the data necessary to decode the sequence, but if these data are known in advance by some method, the sequence header and group of picture header You don't have to wait for it to appear. In such a case, when decoding is started from a P frame or a B frame, there is no reference image, so a method of waiting until the appearance of an I frame and then starting decoding can be considered. In this method, the bit stream is input to the image decoding device. The delay from when the image is output to when the image is output increases.

[0016]

As described above, according to the prior art, when an I frame is not included in the sequence,
Since decoding must be started without a reference image, a correct image cannot be restored in parts other than the intra slice and the intra MB. As described above, in MPEG2, in encoding using inter-frame prediction, the restored image is referenced and the subsequent images are restored. Therefore, if the succeeding images cannot be restored correctly, the intra slice of the subsequent frame is an intra MB.
It was outputting a noisy image until it was refreshed.

When the reproduction of the bit stream is started by switching the video sequence as described above, the I-frame of the reference image is input, and when decoding is started, a correct image is obtained from the image that has started decoding. be able to. However, no image is output while waiting for the I frame.

It is an object of the present invention to provide an image decoding method and apparatus which solves the above-mentioned drawbacks of the conventional apparatus.

[0019]

An image decoding method according to the present invention is an image decoding method for predicting an image frame from a plurality of prediction frames and decoding encoded image data, in which a frame where decoding is started is a frame. When the frame is coded using inter prediction, the inter-frame predicted and coded image data in the inter-frame predicted frame is replaced with the predetermined image data set in advance to be decoded. It was done.

Then, a gray image is used as the predetermined image data set in advance.

In the image decoding apparatus according to the present invention, when the first frame which starts decoding is not the I frame,
Until the decoding of the first P frame is completed, when writing the decoded image to the frame memory, the preset image data such as gray color is written to the MB portion other than the intra MB, and the intra MB portion is written. Only has a control means for writing the decoded image.

As for the frame subsequent to the frame decoded as described above, the control means restores the image by using the frame decoded up to that time as the reference image.

[0023]

According to the present invention, when the first frame in which decoding is started is not an I frame, image data set in advance for a portion other than the intra MB is stored in the frame memory until the decoding of the first P frame is completed. Write to
When this is used as a reference image for the subsequent frames, an image to be displayed can be output with a relatively small noise instead of a large noise as in the past, and an image decoding method that waits until an I frame is input and starts decoding Images can be output faster than devices.

Further, since a gray color image is used as the preset image data, the gray color is an intermediate color in a decoding means such as MPEG which handles pixels by separating them into a luminance signal and a color signal. When restoring a reference image that does not exist, noise is reduced by using a gray color.

[0025]

【Example】

[Embodiment 1] In FIG. 1, the input bit stream 100 is I
In the configuration example in the case of being composed of only P-frames and B-frames without frames, 20 is a variable length decoder, which is basically the same as the variable length decoder 2 of FIG. 7, but as will be described later. The control peculiar to the present invention is also performed. Reference numeral 60 denotes a motion-compensated image reproducing unit, which is basically the same as the motion-compensated image reproducing unit 6 in FIG. 7, but also controls the present invention as described later. Reference numeral 70 is a control signal for performing the characteristic control of the present invention, which will be described later, and is input from the variable length decoder 20 to the motion compensation image reproducing unit 60. Then, the variable length decoder 20 has a function of discriminating whether or not the first frame in which decoding is started is an I frame, and if the discriminated frame is not an I frame, the intra frame is decoded until the P frame is completely decoded. M
In the function of decoding only B, the control signal 70 of the execution request of the present invention, and the motion-compensated image reproducing unit 60, when the execution request of the present invention is requested from the variable length decoder 20, other than the intra MB. A control means for replacing the MB (inter MB) with a previously designated image is configured.
The same reference numerals as those in FIG. 7 indicate the same parts.

Next, the operation will be described. The variable length decoder 20 determines whether or not the first frame in which decoding has started is an I frame, and if the frame is not an I frame, a request to execute the processing of the present invention is issued via the control signal 70 to the motion compensation image. Notify the reproduction unit 60. Further, when the first frame is the P frame or the B frame, the variable length decoder 20 scans only the data of the intra MB until the decoding of the first P frame is completed.
The data of the inter MB is discarded. In the motion compensation image reproducing unit 60, when the execution request from the control signal 70 is notified from the variable length decoder 20, if the first frame is a P frame, the frame memory 152
MB not sent from the inverse DCT unit 5 when writing to the
For, write gray color. The first frame is B
In the case of a frame, when writing to the frame memory 54, for the MB that is not sent from the inverse DCT unit 5,
The same process is repeated until the P frame is sent by rewriting the gray color image data, and when writing the P frame to the frame memory 152, the gray color is changed for the MB that is not sent from the inverse DCT unit 5. Write.

The B2 frame next to the first P1 frame is a frame predicted from the P0 frame originally input before and the first input P1 frame as shown in FIG. Therefore, for the B2 frame next to the first P1 frame, the first P1 frame is read from the frame memory 152, and the P0 that has not been input is read.
Assuming that the other reference image is gray, instead of the frame, the motion-compensated image reproduction unit 60 reproduces the B2 frame and writes it into the predetermined frame memory 54. B
The same applies to three frames.

After the second P4 frame, the normal processing is performed. [Embodiment 2] This embodiment is also a configuration example in which the input bit stream 100 is composed of only P frames and B frames, and the processing up to the first P frame is the same as that of the first embodiment.

As shown in FIG. 3, the data of B2 and B3 frames between the first P1 frame and the second P4 frame in the input order of the bit stream 100 are discarded, and the second P4 frame is predicted from the frame memory 152. The frame data is read out and the decoding result is displayed in the frame memory 53
Write in. As for the subsequent B5 frame, the B frame is reproduced from the frame memories 152 and 53 by the motion compensation image reproducing unit 60 and written in the frame memory 54, as in the normal processing.

After the second P4 frame, the normal processing is performed. [Embodiment 3] In FIG. 4, the input bit stream 100 is P
In the embodiment in the case of being composed of only frames, the processing up to the first P frame is the same as in the first embodiment.

As for the first P frame, the gray color is written when writing to the frame memory 252 for the MB which is not sent from the inverse DCT unit 5 as in the first embodiment. The same reference numerals as in FIG. 1 indicate the same parts.

The subsequent P frames are processed in the same manner as normal processing. [Embodiment 4] In this embodiment, the first frame for decoding is not an I frame. The variable length decoder 20 determines whether or not the first frame started to be decoded is an I frame, and if the frame is not an I frame,
The process execution request from the control signal 70 of the present invention is notified to the motion compensation image reproducing unit 60, and the process for the MB is performed in the same manner as the normal process regardless of whether the MB is an intra MB.

When writing to the frame memory, if the MB type is an intra MB, write to the frame memory as it is, and if it is an inter MB, discard the data sent from the inverse DCT unit 5 and write a gray color. .

Except for the above processing, it is the same as the first embodiment.

[0035]

As described above, the image decoding method according to the present invention is an image decoding method for predicting an image frame from a plurality of prediction frames and decoding encoded image data. Is a frame encoded using inter-frame prediction, the inter-frame predicted and encoded image data in the inter-frame predicted frame is replaced with the predetermined image data set in advance to be decoded. Since this is done, decoding can be started with less noise even for a bitstream that does not start with an I frame, and decoding can be started immediately.

Since a gray image is used as the predetermined image data set in advance, the gray color becomes an intermediate color and noise is reduced.

Furthermore, the image decoding apparatus according to the present invention is
In a device that predicts an image frame from a plurality of prediction frames and decodes encoded image data, inter-frame prediction is performed when a frame to start decoding is a frame encoded using inter-frame prediction. Since there is provided a control means for replacing the image data predicted and encoded between the frames in the frame by the predetermined image data set in advance, the control means is provided, so that there is little noise even in the bit stream not starting with the I frame. The decoding can be started with, and the decoding can be started immediately.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention.

FIG. 2 is a diagram for explaining the operation of the embodiment of FIG.

FIG. 3 is a diagram for explaining the operation of another embodiment of the present invention.

FIG. 4 is a block diagram showing the configuration of still another embodiment of the present invention.

FIG. 5 is a block diagram showing a configuration of a conventional image decoding device.

[Fig. 6] Fig. 6 is a diagram illustrating the configuration of an MB of one frame of an NTSC image.

FIG. 7 is a block diagram showing a configuration of a conventional image decoding apparatus that decodes a sequence that is configured by only P frames and B frames that does not include I frames.

FIG. 8 is a block diagram showing a configuration of a conventional image decoding apparatus that decodes a sequence configured by only P frames that include neither I frames nor B frames.

FIG. 9 is a diagram showing an example of a method of refreshing the entire screen.

[Explanation of symbols]

 1 Buffer Control Unit 2 Variable Length Decoder 3 Scan Converter 4 Inverse Quantizer 5 Inverse DCT Unit 6 Motion Compensated Image Reproducing Unit 20 Variable Length Decoder 40 Data 41 Data 42 Prediction Frame Data 43 Prediction Frame Data 44 Reproduction Pixel Data 50 Memory 51 Buffer memory 52 Frame memory 53 Frame memory 54 Frame memory 60 Motion compensation image reproduction unit 70 Control signal 100 Input bit stream 142 Image data 152 Frame memory 200 Reproduced image

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Kaoru Kawamura 4-36-19 Yoyogi, Shibuya-ku, Tokyo Within Graphics Communications Laboratories, Inc. (72) Inventor Tomoyuki Shindo 4-yoyogi, Shibuya-ku, Tokyo 36-19 No. 19 in Graphics Communications Laboratories Co., Ltd. (72) Inventor Shigeru Komatsu 4-36-yoyogi, Shibuya-ku, Tokyo No. 36-19 In Graphics Communications Laboratories Co., Ltd. (72) Inventor Takayuki Kobayashi Tokyo 4-36-19 Yoyogi, Shibuya-ku, Tokyo Within Graphics Communications Laboratories Inc. (72) Inventor Ryuji Nishito 4-36-19 Yoyogi, Shibuya-ku, Tokyo Graphics Communications Inc. Laboratories in the

Claims (3)

[Claims] 1. An image decoding method for predicting an image frame from a plurality of prediction frames and decoding encoded image data, wherein a frame for starting decoding is a frame encoded using inter-frame prediction. In the image decoding method, the inter-frame predicted and encoded image data in the inter-frame predicted frame is replaced with predetermined image data set in advance to perform decoding. 2. A gray-color image is used as the predetermined image data set in advance.
The described image decoding method. 3. An apparatus for predicting an image frame from a plurality of prediction frames and decoding encoded image data, when a frame for starting decoding is a frame encoded using inter-frame prediction, An image decoding apparatus comprising: a control unit that replaces the inter-frame predicted and encoded image data in the inter-frame predicted frame with predetermined image data set in advance and decodes the image data.