JPH09154135A - Error processing unit - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow the processing unit to conduct compensation in the case of absence of a reference frame or a reference field in a frame or a field with an error or of absence of a motion vector just above a macro block having an error or of a head slice of a frame or a field when an error is compensated to decode an image signal. SOLUTION: When a macro block of a slice S11 is disabling of decoding, a mirror image of a slice S10 having been already decoded just above the position is generated and used for interpolation of the slice S11. Furthermore, in the case of absence of a motion vector in a macro block just above or of a head slice, the interpolation is conducted by using a mirror image of a lower slice.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an error processing device in image decoding processing, and more particularly to MPE.
The present invention relates to an error processing device in a G decoding process.

[0002]

2. Description of the Related Art Conventionally, MPEG has been known as a standard image coding technique for mainly compressing moving image data. Here, the image data in MPEG is sequence G
It has a hierarchical structure of 6 layers of OP (group of pictures), pictures, slices, macroblocks (MB), and blocks.
If a start code is attached to the beginning of each of the above-mentioned sequences, GOPs, pictures, and slices that are constituent units, and if an error occurs during decoding, the decoding can be recovered immediately by skipping to the next start code. It is like this.

In the skipped portion, the motion vector in the macroblock immediately above the macroblock in which an error has occurred and which has been omitted is omitted as the error concealment motion vector of the omitted macroblock. Interpolate macro blocks.

More specifically, as shown in FIG. 6, in a frame F1 of a picture, a slice S
0 is provided with macro blocks M0-1 to M0-n,
It is assumed that the slice S1 is provided with macroblocks M1-1 to M1-n. A motion vector is stored in each macro block. The slice mentioned here means a normal slice, and one slice is formed by 16 lines and is not divided into a plurality of slices.

Here, as shown in FIG. 7A, it is assumed that decoding of a macroblock having a slice S1 becomes impossible. For example, the macroblock M1- in FIG.
It is assumed that 2 is omitted. Then, this macro block M
Using the motion vector of the macro block M0-2 located directly above 1-2, the image block specified by this motion vector is cut out from the frame F0 which is the reference frame as shown in FIGS. 6 and 7B. Then, the macroblock M1-2 that cannot be decoded is interpolated. That is, as shown in FIG. 6, the macroblock M1-corresponding to the position of the macroblock M1-2 in the frame F0.
An image block at a position displaced from the position 2 ′ by the motion vector is extracted and interpolation is performed. Then, as shown in FIG. 7C, the start code is skipped and the decoding process is continued.

In the above processing, since the minimum unit capable of error compensation is a slice, when complete error processing is performed, if a macroblock in a slice cannot be decoded, that slice Will interpolate all of the macroblocks in. In the above description, the case of processing for each frame has been described, but in the case of an interlaced image, the same processing as above is performed for each field.

In MPEG, not all macroblocks have a motion vector,
There may be no motion vector in the macroblock directly above the macroblock in which the error occurred. Further, in the case of the first slice of a certain frame or field, since the slice does not exist directly above, interpolation cannot be performed by the above method. Therefore, in the above case, the interpolation is performed by setting the error concealment motion vector to (0, 0).

According to the MPEG2 standard, MPE
G code "concealment" motion
If you set "vectors" to 1 and turn it on,
Intra macroblocks can also have motion vectors. For example, the first GO in multiple GOPs
In the I picture of P, the motion vector is not normally stored, but the motion vector can be stored by performing the above setting.

[0009]

However, the error compensation as described above cannot be performed for all macroblocks, and the first intraframe having no reference frame or the first intraframe having no reference frame is used. The first intra-field has a problem that interpolation cannot be performed because there is no substitute frame or field. That is, since there is no reference frame or reference field in the first I picture of the first GOP, interpolation cannot be performed. Also, when there is no motion vector in the macroblock immediately above the macroblock in which the error has occurred, or in the case of the first slice of a certain frame or field, interpolation is performed with the error concealment motion vector as (0,0). Cannot be said to be sufficient error compensation for an image suitable for motion compensation.

Therefore, the present invention can perform error compensation when there is no reference frame or reference field in the frame or field in which the error has occurred, and the motion vector directly above the macroblock in which the error has occurred is It is an object of the present invention to provide an error processing device capable of performing error compensation even when there is no slice or when the slice is at the beginning of a certain frame or field.

[0011]

SUMMARY OF THE INVENTION The present invention was created to solve the above problems, and, firstly, an error processing device for performing error compensation when decoding an image signal. When a certain macroblock is interpolated, image data of a mirror image of a macroblock adjacent above or below the macroblock is generated. In the error processing device having the first configuration, when performing interpolation processing for a certain macroblock, image data of a mirror image of a macroblock adjacent above or below the macroblock is generated. Then, for macroblocks of adjacent slices, the macroblocks are interpolated by the image data of the generated mirror image. According to the error processing device of the first configuration,
Since the interpolation is performed by the mirror image, the interpolation can be performed even in the first intra frame or the intra field, and the interpolation can be performed even in the first slice in a certain frame or field. In particular, since a mirror image is created and interpolated, the image can be made continuous in the vertical direction and can be an interpolated image that is good in terms of both visual and frequency.

Secondly, in the first configuration, when the macroblock of the first slice in a frame or field of a picture is interpolated, the macroblock immediately below the macroblock is first decoded, It is characterized in that interpolation is performed by a mirror image of the decoded macroblock. In the error processing device having the second configuration, when interpolating the macroblock of the first slice of a certain frame or field, the macroblock immediately below the macroblock is first decoded. Then, the macroblock of the first slice is interpolated by the decoded mirror image of the macroblock. According to the error processing device of the second configuration, even the first slice in a certain frame or field can be interpolated, and since interpolation is performed by the mirror image, continuous images can be obtained.

Thirdly, in the above-mentioned first configuration, when the macroblock is interpolated in each of two or more slices which are continuous upward or downward, the slice of any macroblock to be interpolated. To generate a mirror image of a macroblock of a decodable slice adjacent to, and perform a interpolation, and to generate a mirror image of a macroblock of an interpolated slice adjacent to a position above or below the slice to be interpolated Is sequentially performed once or a plurality of times. In the error processing device of the third configuration, image data of a mirror image of a decodable slice adjacent to the upper or lower position of the two or more slices is generated. Then, the slice adjacent to the decodable slice is interpolated by the image data. Thereafter, for macroblocks of slices adjacent to positions above or below the slice on which the interpolation has been performed, interpolation is performed by generating a mirror image of the macroblock of the slice on which the interpolation has been performed, and thereafter The generation of a mirror block mirror image is repeated as many times as necessary. According to the error processing device of the third configuration,
Interpolation can be performed even when two or more slices that require interpolation are consecutive in the top and bottom, and interpolation is performed by creating a mirror image of macroblocks of adjacent slices, so that continuous images can be obtained. You can

Fourth, in the first or third configuration, when there is a decodable macroblock above the macroblock to be interpolated, a mirror image of the decodable macroblock is displayed. It is characterized by generating. According to the error processing device of the fourth configuration, since the interpolation is performed by the mirror image generated from the macro block at the upper side, it is possible to make the decoded image of a certain picture continuous from the upper side, and it is possible to visually Images with continuity can be obtained.

Fifth, in any one of the first to fourth configurations, the present invention is characterized by being applied to error compensation of the first intra-frame or intra-field. According to the error processing device of this configuration, the first intraframe or intrafield is interpolated by the mirror image, so the first intraframe without reference frame or the first intrafield without reference field is used. Can also be interpolated.
A sixth feature is that in any one of the first to fifth configurations, the error processing device further has an error processing function based on an ordinary MPEG standard error concealment motion vector. According to the error processing device of the sixth configuration, with respect to an error that can be handled by the error processing by the normal MPEG standard error concealment motion vector, the error processing is performed, and in the case of the first intra frame or intra field, In the case of the first slice of a certain frame or field, interpolation using a mirror image can be performed to perform more appropriate error compensation as a whole.

[0016]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described with reference to the drawings. As shown in FIG. 1, a decoding device A according to the present invention includes a decoder 10 and a concealment motion vector storage device 12 (hereinafter,
“Vector storage device 12”), a decoded image storage device 14, and an error processing device 16. Here, the decoder 10 receives a bit stream sent from the outside and decodes encoded image data. The vector storage device 12 stores a motion vector for each macroblock.
The decoded image storage device 14 also stores the image data decoded by the decoder 10. further,
The error processing device 16 performs error compensation and interpolates macroblocks in which data is missing. The interpolation is performed by creating a mirror image of adjacent decodable macroblocks. Details will be described below.

The operation of the decoding device A having the above configuration will be described. It should be noted that the slice here means a normal slice, and one slice is formed by 16 lines, is not divided into a plurality of slices, and does not straddle the next 16 lines. Shall mean. The bit stream is sent to the decoder 10 from the outside, and the decoder 10 performs a decoding process. Here, the decoder 10
In the case where the decoding is performed in a state where no error normally occurs, the decoded image is sent to and stored in the decoded image storage device 14 for each macroblock, and at the same time, the motion vector stored in the macroblock is stored as a vector. Device 12
Sent to and stored. This motion vector is used as an error concealment motion vector at the time of error processing of a macroblock immediately below the macroblock in position.

On the other hand, if the bitstream becomes undecodable in a certain macroblock, the decoder 10
The picture number at that point and the head macroblock number of the slice to which the macroblock in which the error has occurred belong are sent to the error processing unit 16. Then, the error processing device 16 stores the motion vector of the macroblock positioned immediately above the macroblock in the vector storage device 12.
After that, the substitute macroblock is extracted from the reference frame in the already decoded decoded image. That is, the image block specified by the above motion vector is cut out from the reference frame for the frame in which the error has occurred (this is referred to as the “current frame”), and the slice in which the error has occurred is interpolated. Then, the work is continued until the next slice of the decodable start code, and then the normal decoding process is resumed. The content of this error compensation is the same as the content described in the above-mentioned conventional technique.

Next, when an error occurs in the slice in the first intra frame or intra field, the above interpolation cannot be performed because there is no reference frame or reference field. Interpolation like this is performed.

First, the error processing when an error occurs in the first intra-frame or the second and subsequent slices in the intra-field and decoding becomes impossible will be described. In this case, a mirror image of the macroblock immediately above the position is created to interpolate the macroblock. Since the minimum unit capable of error compensation is a slice, in reality, a mirror image is created for the entire decodable slice and the slice in which the error has occurred is interpolated.

That is, as shown in FIG. 2A, the case where an error occurs in a macro block in the second and subsequent slices S11 in a normal slice will be described. The error processing device 16 is shown in FIG. As shown in
The decoded image data of the slice immediately above the slice S11 in which the error has occurred, that is, the decoded image data of the slice S10 is received from the decoded image storage device 14, and a mirror image is created from this image data. The interpolated image data is sent to the decoded image storage device 14. Then, when the interpolation is completed, as shown in FIG. 2C, the start code is skipped and the decoding is continued.

Next, a case will be described in which it is impossible to continuously decode two or more slices of the first and second intra frames and the second and subsequent slices in the intra field. This is a case where an error occurs in macroblocks in two or more consecutive slices of the second and subsequent slices of a normal slice. In this case, a mirror image of the macroblock immediately above the position is created, the macroblock is interpolated, and then the same operation is applied to the following slices one after another. Again, actually, create a mirror image for the entire decodable slice, interpolate the slice directly below, then create a mirror image of the interpolated slice, interpolate the slice below it, and so on. Will be processed.

That is, as shown in FIG. 3A, the slices S21 and S22 which are the second and subsequent slices.
In the case where the macroblock in FIG.
As shown in FIG. 3, the decoded image data of the slice immediately above the slice S21 in which the error has occurred, that is, the decoded image data of the slice S20 is received from the decoded image storage device 14, and the image data of the mirror image is obtained from this image data. create. The image data of this mirror image is for interpolating the slice S21. Further, image data of a mirror image of the image data created by interpolation of the slice S21 is created. This image data is for interpolation of the slice S22. Then, the image data of these mirror images is sent to and stored in the decoded image storage device 14. Then, when the interpolation is completed, FIG.
As shown in, skip the start code and continue decoding. In addition, in the above example, the case where 2-slice decoding cannot be performed continuously has been described. However, even when 3 or more slices cannot be decoded, the slices immediately below the decodable slices are sequentially downward. Create a mirror image on and interpolate.

Next, a case will be described in which the first slice cannot be decoded in the very first intra frame or intra field. In this case, the next decodable slice is decoded first, and then a mirror image of the macroblock in that slice is created, and the missing macroblock immediately above, that is, the macro in the previous slice, is created. Interpolate blocks. Here again, in practice, a mirror image is created and interpolation is performed for the entire decodable slice.

That is, as shown in FIG. 4A, a case where an error occurs in a macroblock in the first slice S31 in a normal slice will be described. First, as shown in FIG. 10, the next slice S32 that can be decoded by skipping the start code is decoded first, and this decoded image is stored in the decoded image storage device 1
Store in 4. Then, the error processing device 16 creates a mirror image of the decoded image of the slice S32 and interpolates the slice S31.

Next, a case where two or more slices cannot be decoded consecutively from the first slice in the first intraframe or intrafield will be described. In this case, the decodable slice is decoded first, the mirror image of the macroblock in that slice is created, the macroblock immediately above is interpolated, and then the same operation is performed one after another. Will be applied to. Here again, in the same manner as described above, actually, a mirror image is created and interpolation is performed for the entire decodable slice.

That is, as shown in FIG. 5 (a), the first slice S4 to the slice S4n-1 in the normal slices.
In the case where an error occurs in each macroblock in the above, first, as shown in FIG. 5B, the decoder 10 skips the start code and decodes the next slice S4n that can be decoded first. Next, this decoded image is stored in the decoded image storage device 14. Then, the error processing device 16 creates a mirror image of the decoded image of the slice S4n. This image data is for interpolation of the slice S4n-1. Furthermore, slice S4n-
The image data of the mirror image of the image data created in the interpolation of 1 is created and the slice S4n-2 is interpolated. In this way, the image data of the mirror image is created one after another and interpolation is performed.

As described above, according to the decoding apparatus A of this embodiment, since mirror images of adjacent slices are created and interpolation is performed, it is possible to perform error processing within the first intra frame or intra field. Become. In addition,
The interpolation with the mirror image described above is useful for error handling within the first intra-frame or intra-field, but can also be used for frames or feeds other than the first intra-frame or intra-field. Also, in the above error handling,
Since the mirror image is created and interpolated, the image can be made continuous in the vertical direction and can be an interpolated image that is good in terms of both visual and frequency.

When looking at a screen on which a certain image is displayed, since human visual action is characterized by looking downward from above, a slice that can be decoded is located above the slice in which an error has occurred. In some cases, it may be desirable to interpolate with the mirror image of the decodable slice. Further, in the above description, when the image data of the mirror image is generated, it is described that a mirror image of a certain slice is generated. However, when interpolation is possible in macroblock units, the mirror image of a certain macroblock is It may be generated and interpolated.

[0030]

According to the error processing device of the first aspect of the present invention, since the interpolation is performed by the mirror image, the interpolation can be performed even in the first intra-frame or intra-field. Also, interpolation can be performed even for the first slice in a certain frame or field. Especially, since a mirror image is created and interpolated, the image is made continuous in the vertical direction,
It is possible to obtain an interpolated image that is good both visually and in terms of frequency.

Further, in particular, according to the error processing device of the second aspect, even the first slice in a certain frame or field can be interpolated, and since interpolation is performed by the mirror image, continuous images are obtained. Can be obtained. Further, in particular, according to the error processing device of the third aspect, interpolation can be performed even when two or more slices that require interpolation are consecutive in the vertical direction, and macroblocks of adjacent slices can be interpolated. Since a mirror image is created and interpolation is performed, continuous images can be obtained. Further, in particular, according to the error processing device of claim 4,
Since the interpolation is performed by the mirror image generated from the macro block located above, the decoded image of a picture can be a continuous image from above, and a visually more continuous image can be obtained.

Further, in particular, according to the error processing device of the fifth aspect, since the interpolation by the mirror image is performed on the first intra frame and the intra field, the first intra frame having no reference frame and the reference frame Interpolation can also be performed on the very first intra field that has no fields. Further, in particular, according to the error processing device of the sixth aspect, it is possible to perform more appropriate error compensation as a whole.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an error processing device according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing a procedure of error processing.

FIG. 3 is an explanatory diagram showing a procedure of error processing.

FIG. 4 is an explanatory diagram showing a procedure of error processing.

FIG. 5 is an explanatory diagram showing an error processing procedure.

FIG. 6 is an explanatory diagram illustrating conventional error processing, which is performed by using an error concealment motion vector.

FIG. 7 is an explanatory diagram illustrating a conventional error processing, which is an error processing procedure using an error concealment motion vector.

[Explanation of symbols]

A Decoding device 10 Decoder 12 Concealment motion vector storage device 14 Decoded image storage device 16 Error processing device S10, S11, S21, S20, S21, S22, S
31, S32, S41, S42, S4n, S4n-1,
S4n-2 slice

Claims (6)

[Claims] 1. An error processing device for performing error compensation when decoding an image signal, wherein image data of a mirror image of a macroblock adjacent above or below the macroblock when the macroblock is interpolated. An error handling device characterized by: 2. When interpolating a macroblock of a first slice in a frame or field of a picture, a macroblock immediately below the macroblock is first decoded, and interpolation is performed using a mirror image of the decoded macroblock. The error processing device according to claim 1, wherein 3. When interpolating a macroblock in each of two or more consecutive slices that are continuous upward or downward,
Interpolation is performed by generating a mirror image of a macroblock of a decodable slice adjacent to a slice of any macroblock to be interpolated, and also at an upper or lower position of the slice to be interpolated. The error processing device according to claim 1, wherein the process of generating the mirror image of the macroblock is sequentially performed once or a plurality of times. 4. The mirror image of a decodable macroblock is generated when there is a decodable macroblock above the macroblock to be interpolated. Error handling device. 5. The error processing device according to claim 1, wherein the error processing device is applied to error compensation of the first intra frame or intra field. 6. The error processing device further comprises a normal MP.
The error processing device according to any one of claims 1 to 5, which has an error processing function based on an EG standard error concealment motion vector.