JPH10271510A - Image data encoding / decoding method and encoding device / decoding device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image data encoding / decoding method and an encoding device / decoding device for compressing and encoding image data of different moving image sequences with extremely high efficiency and decoding the image data. . SOLUTION: On the encoding side, a predictive encoding unit 102
On the condition that the corresponding frame does not exist in the lower layer, the image data is encoded by predictively encoding the shape information of the image object of the lower layer located temporally before or after the frame. On the decoding side, on the other hand, the prediction decoding unit 202 predictively decodes the shape information of the image target that has been predictively coded, and decodes the image data coded on the coding side. Therefore, on the encoding side, the shape information of the image target is predictively encoded and the image data is encoded, so that the data amount after encoding is reduced.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention belongs to the field of digital image processing, and relates to an image data encoding / decoding method and an encoding apparatus / decoding apparatus for encoding and decoding moving image data with high efficiency.

[0002]

2. Description of the Related Art Conventionally, in encoding moving image data,
A method of combining different moving image sequences is being studied. For example, “Image coding using hierarchical representation and multiple templates (IEICE IE94-159, pp99-10)
6, 1995)], a new moving image sequence is obtained by superimposing a moving image sequence of a background and a moving image sequence of a part moving image (for example, a human image or a video of a fish cut out by the chroma key technique) as a foreground. Is described.

[0003] Further, "temporal scalability coding based on image contents (" Temporal Scalability coding "
goritm based on image co
nent ", ISO / IEC JTC1 / SC29 /
WG11 MPEG95 / 0377, 1995) ", a method of superimposing a moving image sequence of a component image obtained by encoding only a specific region in an image on a moving image sequence having a low frame rate to improve the frame rate of this region. Is stated.

In this method, as shown in FIG. 10, the entire lower layer image is motion-compensated and predictively coded at a low frame rate. In the upper layer, a component region of a frame not coded in the lower layer (see FIG. Only the halftone portion is subjected to motion compensation prediction coding. When decoding, the upper layer is motion compensated using the lower layer decoded frame and the upper layer decoded frame as reference frames. As the component area, a part that attracts the viewer's attention, such as a person part, is selected.

In this method, if only the lower layer is decoded, the entire image is reproduced at a low frame rate. On the other hand, if both the lower layer and the upper layer are decoded, only the component area is reproduced at a high frame rate. At this time, for a frame in which the decoded image does not exist in the lower layer, the lower layer is synthesized using the two decoded lower layer frames. By superimposing the decoded image (component region) of the upper layer on the frame obtained by the synthesis, it is possible to obtain a reproduced image in which the frame rate of the component region such as the person portion is improved.

Next, a method of synthesizing the lower layer will be described with reference to FIG. Here, in FIG. 11A, images A and C are two decoded images of the lower layer, and image B is a decoded image of the upper layer. The order of the display time of each image is A, B, C. The hatched area indicates the component area.

In FIG. 1, the decoded image B of the upper layer is obtained by encoding only the component area, and the images of the images A and C are shown by broken lines for reference. In the following description, for example, a component area (foreground area) of the image X is expressed as F (X), and an area (background area) other than the component area F (X) of the image X is expressed as B (X). I do.

First, a background image shown in FIG. 11B is synthesized from images A and C shown in FIG. That is,
In FIG. 10A, pixel values are copied from an image C for an area of a component area F (A) and a background area B (C),
The pixel values of the component area F (C) and the background area B (A) are copied from the image A. The pixel values of the other areas are copied from the closest one of the images A and C.

The image thus copied and synthesized is
As shown in FIG. 11B, an image is formed of only the background region excluding the regions of F (A) and F (C). If this image is used as a background image and a coded image (component area) of an upper layer of the image B is superimposed, a decoded image of the image B with the synthesized background is obtained as shown in FIG. In the above-described conventional technique, in order to perform frame synthesis of a lower layer, the shape of a component region of a lower layer frame (images A and C) positioned before and after a frame (image B) of an upper layer is encoded.

Here, the shape data representing the shape of the component region is binary image data having a pixel value of 1 inside the component region and having a pixel value of 0 outside the component region. , MMR coding (a coding method used in facsimile, etc.) independently for each frame, chain coding (a method in which pixels located on the outer periphery of an area are sequentially traced from a certain pixel, and the moving direction of the pixels is coded), etc. This is performed using In the MMR encoding, a change point of data is detected for each line of the shape data, the relative position between the position of the change point and the change point of the previous line is calculated, and this relative position is encoded.

[0011]

By the way, the shape data of the component area has a correlation between frames, and a sufficient compression performance cannot be obtained by a method of encoding the component area independently for each frame. Therefore, in order to remove the correlation between frames, motion compensation prediction coding of shape data using MMR coding has been studied (ISO / IEC J).
TC1 / SC29 / WG11 MPEG96 / N146
9). However, when encoding the shape data of the component area used for the synthesis of the lower layer frame, the correlation with the shape data of the upper layer is not taken into account, and there is still a problem in the compression performance.

SUMMARY OF THE INVENTION The present invention has been made in view of the foregoing problems, and has an image data encoding apparatus and an image data encoding apparatus for compressing and encoding image data of different moving image sequences with extremely high efficiency and decoding the same. It is an object to provide a device.

[0013]

Means for Solving the Problems The present invention has the following arrangement to achieve the above object. The image data encoding / decoding method according to the first aspect of the present invention integrates image data of a lower layer having a low frame rate and an upper layer having a high frame rate, and encodes and decodes the same as one moving image sequence. An encoding / decoding method for image data, wherein (a) on the encoding side, a frame corresponding to a frame position of an upper layer does not exist in the lower layer, and is temporally earlier than the frame. Or predictively encode the shape information of the image object of the lower layer located later and encode the image data,
(B) On the decoding side, predictive decoding of the shape information of the image object that has been predictively encoded, synthesizes the lower layer frame that does not exist, and decodes the image data encoded on the encoding side. It has a configuration of an encoding / decoding method of image data as a feature.

According to a second aspect of the present invention, an image data encoding apparatus integrates image data of a lower layer having a low frame rate and an upper layer having a high frame rate and encodes the integrated data as one moving image sequence. An image data encoding device, wherein a frame corresponding to a frame position of an upper layer does not exist in the lower layer, and a frame of the lower layer located temporally before or after the frame is provided. The image data encoding apparatus has a configuration in which the shape information of the image object to which the image data belongs is predictively encoded with reference to the already encoded image object shape information of the lower layer or the upper layer.

According to a third aspect of the present invention, there is provided an image data decoding apparatus which integrates a lower layer having a low frame rate and an upper layer having a high frame rate and encodes the image data as one moving image sequence. An apparatus for decoding image data to be decoded, wherein a frame of the lower layer positioned temporally before or after the frame is provided on the condition that a frame corresponding to a frame position of an upper layer does not exist in the lower layer. Predictive decoding with reference to the already decoded lower layer or upper layer image target shape information of the image target shape information belonging to
The image data decoding apparatus is characterized in that the non-existent lower layer frame is synthesized.

According to a fourth aspect of the present invention, there is provided an image data encoding apparatus which encodes an upper layer frame by referring to at least image object shape information belonging to an already encoded lower layer or upper layer frame. 3. The predictive coding of shape information of an image object to which the image belongs.
Has a configuration of an image data encoding device described in (1).

According to a fifth aspect of the present invention, there is provided an image data decoding apparatus which refers to at least the already decoded image object shape information belonging to a lower layer or an upper layer frame, the image belonging to an upper layer frame. The image data decoding apparatus according to claim 3, wherein the target shape information is predictively decoded.

According to a sixth aspect of the present invention, in the image data encoding apparatus, when predictive encoding of shape information of an image target is performed, the motion vector of the shape information of the image target is encoded. An image data encoding apparatus according to any one of claims 2 and 4.

According to a seventh aspect of the present invention, in the image data decoding apparatus, when predictive decoding of shape information of an image target is performed, a motion vector of the shape information of the image target is decoded. Or a configuration of the image data decoding device according to any one of 5.

According to an eighth aspect of the present invention, there is provided an image data encoding apparatus, wherein a motion vector for predictively encoding shape information of an image object is used for encoding pixel information of a frame to which the image object belongs. The method according to claim 1, further comprising: using a motion vector for encoding the pixel information when predictive encoding the shape information of the image target, on condition that the motion vector coincides with the motion vector. 5. The image data encoding device according to any one of items 2 and 4.

According to a ninth aspect of the present invention, in the image data decoding apparatus, a motion vector for predictive encoding of shape information of an image object is a motion vector for encoding pixel information of a frame to which the image object belongs. The motion vector for encoding the pixel information is decoded when predictive decoding of the shape information of the image object is performed on condition that the shape information coincides with a vector.
Item has a configuration of a decoding device for image data.

According to a tenth aspect of the present invention, in the image data encoding apparatus, a motion vector for predictive encoding of shape information of an image object is used for encoding pixel information of a frame to which the image object belongs. The method according to claim 1, wherein, when the shape information of the image object is predictively encoded, a motion vector for encoding the pixel information is used, and the motion vector is encoded on condition that the shape information does not match the motion vector. 5. The image data encoding device according to any one of items 2 and 4.

According to an eleventh aspect of the present invention, in the image data decoding apparatus, a motion vector for predictively encoding shape information of an image target is a motion vector for encoding pixel information of a frame to which the image target belongs. When predicting and decoding the shape information of the image target, on condition that they do not match the vector, a motion vector for encoding the pixel information is decoded, and the shape information of the image target is decoded based on the decoded motion vector. The image data decoding apparatus according to any one of claims 3 to 5, wherein a motion vector for predictive decoding is calculated.

According to a twelfth aspect of the present invention, there is provided an image data encoding apparatus, wherein a motion vector for predictively encoding shape information of an image object is used for encoding pixel information of a frame to which the image object belongs. When predictive coding of the shape information of the image target is performed on condition that they do not match the motion vector, a motion for predictively coding the shape information of the image target based on a motion vector for coding the pixel information. The image data encoding apparatus according to any one of claims 2 and 4, wherein the motion vector is encoded by obtaining a vector.

According to a thirteenth aspect of the present invention, in the image data decoding apparatus, a motion vector for predictive encoding of shape information of an image object is a motion vector for encoding pixel information of a frame to which the image object belongs. On the condition that they do not match the vector, when predictive decoding the shape information of the image target, decoding a motion vector for predictive decoding of the image target based on a motion vector for encoding the pixel information. An image data decoding apparatus according to any one of claims 3 and 5, characterized in that:

According to a fourteenth aspect of the present invention, in the image data encoding apparatus, when predictive encoding is performed on the shape information of the image object, the motion of the shape information of the image object with respect to the motion vector of the pixel value of the image object 5. The apparatus according to claim 2, wherein a prediction error vector of the vector is obtained, and the prediction error vector is encoded.

According to a fifteenth aspect of the present invention, in the image data decoding apparatus, when decoding shape information of an image object,
4. A motion vector for predictive decoding of shape information of an image object by decoding a prediction error vector of a motion vector of shape information of the image object with respect to a motion vector of a pixel value of the image object. Or a configuration of the image data decoding device according to any one of 5.

The operation of the present invention will be described below. According to the image data encoding / decoding method according to the first aspect of the present invention, on the encoding side, a frame (lower frame) corresponding to an upper layer frame (upper frame) having a high frame rate is placed in a lower layer. If not, the shape information of the image object belonging to the frame of the lower layer positioned temporally before or after the upper frame is predictively encoded, and the image data is encoded.

On the other hand, when decoding the encoded image data, the decoding side predictively decodes the shape information of the image object belonging to the lower layer frame positioned temporally before or after the upper frame. Then, the non-existent lower layer frame is synthesized using the lower layer frame positioned before or after the above, and the encoded image data is decoded.

According to the image data encoding apparatus of the present invention, when a frame (lower frame) corresponding to an upper layer frame (upper frame) having a high frame rate does not exist in the lower layer, Predictive coding is performed on the shape information of the image object belonging to the lower layer frame positioned temporally before or after the upper frame.

Here, the predictive coding is performed by coding the motion vector of the shape information of the image object of the lower layer with reference to the already encoded image object shape information of the lower layer or the upper layer. As a result, the shape information of the image target for synthesizing the lower layer frame corresponding to the upper frame is predictively encoded, and the data amount after encoding is reduced as compared with the case where the information is independently encoded.

According to the image data decoding apparatus according to the third aspect of the present invention, when decoding the image data encoded by the encoding apparatus according to the second aspect,
If the lower frame corresponding to the upper frame does not exist, the shape information of the image object belonging to the lower layer frame temporally located before or after the upper frame is predicted and decoded, and the lower layer frame corresponding to the upper frame is decoded. And decode the image data.

According to the image data encoding apparatus of the present invention, the upper layer frame is referred to by referring to the already encoded image object shape information belonging to the lower layer or upper layer frame. Predictively encodes shape information of an image object belonging to.

According to the fifth aspect of the present invention, the image data decoding apparatus refers to the already decoded image object shape information belonging to the lower layer or upper layer frame and belongs to the upper layer frame. Predictively decode the shape information of the image target.

According to the image data encoding apparatus of the present invention, a motion vector for predictively encoding shape information of an image coincides with a motion vector for encoding pixel information. In this case, instead of the motion vector of the shape information of the image object, a motion vector for encoding pixel information of a frame to which the image object belongs is encoded, and the shape information of the image object is predictively encoded.

According to the image data decoding apparatus according to the ninth aspect, when decoding the image data encoded by the encoding apparatus according to the eighth aspect,
A motion vector for encoding pixel information included in the encoded image data is obtained by decoding, and the motion vector is used to predictively decode the shape information of the image target and decode the image data.

According to the image data encoding apparatus of the present invention, a motion vector for predictively encoding shape information of an image object does not match a motion vector for encoding pixel information. In this case, instead of the motion vector of the shape information of the image object, a motion vector for encoding the pixel information of the frame to which the image object belongs is encoded to predictively encode the shape information of the image object.

According to the image data decoding apparatus according to the eleventh aspect of the present invention, when the image data encoded by the encoding apparatus according to the tenth aspect of the present invention is decoded, the encoded image data is decoded. The motion vector of the pixel information included in the data is decoded. Then, a motion vector for predictively decoding the shape information of the image target is obtained from the motion vector obtained by decoding, and using this, the shape information of the image target is predictively decoded to decode the image data.

According to the twelfth aspect of the present invention, a motion vector for predictively coding shape information of an image object does not match a motion vector for coding pixel information. In this case, a motion vector for predictively encoding the shape information of the image target is obtained from the motion vector of the pixel information, and the obtained motion vector is encoded to predictively encode the shape information of the image target.

According to the image data decoding apparatus according to the thirteenth aspect of the present invention, when decoding the image data encoded by the encoding apparatus according to the twelfth aspect of the present invention, the encoded image data is decoded. The motion vector of the shape information of the image object included in the data is decoded using the motion vector of the pixel information, and the image data is decoded.

According to the image data encoding apparatus of the present invention, the prediction error vector of the motion vector of the shape information of the image object with respect to the motion vector of the pixel value of the image object is obtained. The vector is encoded, and the shape information of the image target is predictively encoded.

According to the image data decoding apparatus according to the present invention, when decoding the image data encoded by the encoding apparatus according to the present invention, the encoded prediction is performed. Determining a motion vector for predictive decoding of shape information of the image target by decoding the error vector,
This is used to predictively decode the shape information of the image target.

[0043]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing a configuration of an image data encoding apparatus according to a first embodiment of the present invention; In each figure,
The common elements or the corresponding elements are denoted by the same reference numerals, and the description thereof will not be repeated.

The encoding apparatus according to the first embodiment integrates image data of a lower layer having a lower frame rate and an upper layer having a higher frame rate and encodes the data as one moving image sequence. Area (image object) belonging to a lower layer frame temporally preceding or following the upper layer frame, provided that the frame to be performed does not exist in the lower layer.
Is predictively coded. The details will be described below.

FIG. 1 is a block diagram showing a configuration of an encoding device according to the present embodiment. In the figure, 101 is a first shape coding unit, 102 is a prediction coding unit, and 103 is a second shape coding unit. Further, the prediction encoding unit 102
Motion vector detection unit 102A, motion compensation prediction unit 102
B and a memory 102C.

The operation of each section will be described below. The first shape encoding unit 101 encodes shape data (hereinafter, referred to as “first shape data”) of a component region (image object) of an upper layer. For this encoding, for example, MMR encoding or chain encoding is used. The encoded first shape data is sent to an encoded data integration unit (not shown) and integrated with other encoded data. Also, the first shape encoding unit 10
1 decodes the encoded first shape data and
Store in 2C.

The predictive coding unit 102 performs motion compensation prediction on the shape data (hereinafter, referred to as “second shape data”) of the component area (image object) used for lower-layer frame synthesis and performs coding (prediction coding). ). Specifically, first, the motion vector detection unit 102A detects a motion vector for the second shape data. Here, the second shape data is divided into blocks, and a motion vector is detected for each block.

As a reference image in the motion compensation prediction encoding, the first shape encoding unit 101 or the second shape encoding unit 103 encodes and decodes the first image already encoded and decoded.
The shape data or the second shape data is used. These data are supplied from the memory 102C. The motion vector of the second shape data detected by the motion vector detection unit 102A is sent to a motion vector encoding unit (not shown) and encoded, and the second shape data is predictively encoded.

The motion compensation prediction unit 102B stores the motion vector detected by the motion vector detection unit 102A and the memory 10
A motion compensated prediction is performed from the reference image stored in 2C to generate a predicted image. For example, when MMR coding is used, as described above, a data change point (a portion corresponding to the contour portion of the component area) is detected for each line of the shape data, and the position of the change point and the position on the prediction image are detected. Is calculated as prediction error data. Then, the second shape encoding unit 103 encodes the prediction error data together with the second shape data, and sends the encoded prediction error data to an encoded data integration unit (not shown).

As described above, the present encoding apparatus encodes the first shape data of the upper layer and the second shape data of the lower layer and then integrates the first shape data to form one image data of one moving image sequence. Encode as a part. At this time, the second shape data used for lower-layer frame synthesis is not independently encoded, but is motion-compensated and predicted and encoded.

Hereinafter, a decoding device for decoding the image data encoded by the above-described encoding device will be described. The decoding apparatus decodes image data obtained by integrating a lower layer having a low frame rate and an upper layer having a high frame rate and encoding the moving image sequence as a single moving image sequence. The image data is decoded by predictively decoding the shape data of the component region (image target) belonging to the lower layer frame positioned temporally before or after the upper layer frame, on condition that the frame does not exist. The details will be described below.

FIG. 2 is a block diagram showing the configuration of the decoding device according to this embodiment. In the figure, reference numeral 201 denotes a first shape decoding unit, 202 denotes a prediction decoding unit, and 203 denotes a second shape decoding unit. The prediction decoding unit 202 includes a motion compensation prediction unit 202A and a memory 202B. Further, the apparatus includes an encoded data separation unit (not shown),
As a result, the data integrated in the encoding apparatus described above is separated into data before the integration.

Here, the first shape decoding unit 201 decodes the coded data of the first shape obtained by the separation by the coded data separation unit (not shown), and obtains the shape data of the component area of the upper layer. The decrypted data is stored in the memory 202B
And is sent to an upper layer decoding unit (not shown) to be used for decoding pixel data of the upper layer.

The predictive decoding unit 202 decodes the shape data (second shape data) of the component area for frame synthesis in the lower layer. Specifically, the motion compensation prediction unit 202A creates a prediction pixel by performing the same operation as the motion compensation prediction unit 102B of FIG. The second shape decoding unit 203 performs predictive decoding of the second shape encoded data obtained by separation by an unillustrated encoded data separation unit using the prediction image, and generates second shape decoded data. The decoded data is stored in the memory 202B and sent to a lower layer synthesizing unit (not shown) to reproduce an image by using the lower layer frame synthesizing. As described above, in the present embodiment, the encoding device efficiently encodes the second shape data by motion compensation prediction, and the decoding device predictively decodes the second shape data to decode the image data.

Next, an image data encoding apparatus and a decoding apparatus according to a second embodiment of the present invention will be described. The encoding device according to the present embodiment is characterized in that the encoding of the first shape data and the encoding of the second shape data are performed by the same circuit,
Thereby, the circuit scale of the device is reduced.

FIG. 3 is a block diagram showing the configuration of the encoding device according to the present embodiment. The device shown in the figure has the same configuration as that of the device of the first embodiment shown in FIG. 1 except that a shape data rearranging unit 301 is provided at the preceding stage, and a first shape encoding unit 101 and a second shape encoding unit are provided. A configuration is provided including a shape encoding unit 304 in place of 103.

The shape data rearranging section 301 rearranges the shape data (first shape data) of the component area of the upper layer and the shape data (second shape data) of the component area of the lower layer in the order of encoding. To explain with reference to FIG. 11A, for example, an image B shown in FIG.
Is input and images A and C are input as the second shape data, these are rearranged in the order of images A, B, and C. As a method of rearranging, for example, a method of rearranging in the order of display time, a method of rearranging in the encoding order of pixel data, or the like is used.

Returning to FIG. Predictive coding unit 102
1, motion-compensated prediction of shape data being coded from already coded and decoded shape data, and coding and decoding of a prediction error into shape data obtained by the memory 102C. Write to. The difference from the device of the first embodiment is that the device of the first embodiment has the first
While the shape encoding unit 101 independently encodes the first shape data, in the present embodiment, the shape encoding unit 304 encodes the first shape data when encoding the first shape data. The point is that prediction and encoding are performed from the decoded data of the second shape data in addition to the decoded data of the shape data.

FIG. 4 is a diagram showing a configuration of a decoding device according to the present embodiment. The device shown in FIG. 13 includes a shape decoding unit 402 instead of the first shape decoding unit 201 and the second shape decoding unit 203 in the configuration of the device shown in FIG.
The subsequent stage is provided with a shape data reverse sorting unit 404. The shape decoding unit 402 corresponds to the second shape decoding unit 203 shown in FIG. 2, and decodes shape data that has been motion-compensated and predictively encoded by the same operation. Also, the shape decoding unit 4
02 is different from the second shape decoding unit 203 in that the first shape data and the second shape data are mixed in the data input from the motion compensation prediction unit 202A, and data mixed with these is processed.

The shape data decoded in this way is
The shape data reverse sorting unit 404 separates the data into first shape data and second shape data. In this case, the shape data reverse sorting unit 404 performs the reverse operation of the shape data sorting unit 301 shown in FIG. 3 to sort the shape data and separate it into first shape data and second shape data.

As described above, according to the apparatus of the present embodiment, the second shape data is efficiently encoded by the motion compensation prediction, and the first shape data is encoded by the same circuit as the second shape data. In addition, the circuit scale of the encoding device is reduced.

Next, an encoding device and a decoding device according to a third embodiment of the present invention will be described. The encoding apparatus according to the present embodiment obtains a motion vector used for motion compensation prediction of shape data from a motion vector used for motion compensation prediction encoding of pixel values, and thereby obtains a circuit scale of a motion vector detection unit in the encoding apparatus. , And the amount of data to be encoded is reduced without encoding the motion vector of the shape data.

The encoding apparatus and the decoding apparatus according to the present embodiment are configured by replacing the predictive encoding section 102 and the predictive decoding section 202 with those shown in FIG. 5 in the configuration of the apparatus shown in FIGS. Is done. Here, FIG.
, 501A is a motion vector creation unit, 501B is a motion compensation prediction unit, and 501C is a memory.

However, in the case of the present encoding apparatus, the motion vector creation unit 501A shown in FIG. 5 uses the pixel value instead of the shape data of the shape data rearranging unit 301 and the motion vector from the motion vector data decoding unit. A motion vector for prediction is input. Also, in the case of the decoding device of the present embodiment,
The decoded data of the motion vector for pixel value prediction is input to the motion vector creation unit 501A illustrated in FIG. 5 instead of the decoded data of the motion vector for shape data prediction. The motion vector creation unit 501A creates a motion vector for shape data prediction from a motion vector for pixel value prediction.

A specific description will be given with reference to FIG. This figure shows the concept of motion compensation prediction of the lower layer and the upper layer in the present encoding device. The solid arrow represents the motion compensation prediction of the pixel value, and the dotted arrow represents the motion compensation prediction of the shape data. Represent.

As shown in the figure, when the pixel value motion compensation and the shape data motion compensation are in one-to-one correspondence (to predictively encode the shape data of the image object belonging to the lower layer frame). Is used as the motion vector for coding the pixel information of the frame to which the image object belongs), and the motion vector used for the motion compensation of the pixel value is used as it is for the motion compensation of the shape data. be able to.

Therefore, the motion vector creating section 501A
The motion vector of the pixel value is output as it is as the motion vector of the shape data. However, it is assumed that the block for performing the motion compensation of the pixel value and the block for performing the motion compensation of the shape match.

As described above, according to the present encoding apparatus, the motion vector of the shape data is not encoded, and the decoding apparatus uses the decoded data of the motion vector of the pixel value as it is as the motion vector of the shape data. In this case, since the encoding device does not detect the motion vector of the shape data, the circuit scale of the device can be reduced. In addition, since the motion vector of the shape data is not encoded, the amount of encoded data can be reduced.

Next, in the decoding apparatus according to the present embodiment, the motion vector creating section 501A replaces the decoded data of the motion vector for shape data prediction with a motion vector for pixel value prediction (encodes pixel information). ) Is input. Further, the memory 501C of FIG.
Stores two types of shape data: decoded data of the first shape data and decoded data of the second shape. Then, the motion vector creation unit 501A generates a motion vector from the decoded data of the motion vector for pixel value prediction, and predictively decodes the second shape data using the generated motion vector.

Further, in the present embodiment, the prediction encoding unit 102 and the prediction decoding unit 202 shown in FIGS. 3 and 4 have the configuration shown in FIG. 5, but the prediction encoding unit 1 and the prediction encoding unit 1 shown in FIGS.
02 and the predictive decoding unit 202 may be configured as shown in FIG. In this case, in the encoding device, instead of inputting the component area shape data for lower layer synthesis and the motion vector from the motion vector decoding unit, the motion vector creation unit 501A shown in FIG. Is entered.

When the encoding device and the decoding device are configured as described above, the first shape data is encoded and decoded independently, so that the motion vector creating unit 5 shown in FIG.
The motion vector generated by 01A is used only for motion compensation prediction of the second shape data.

FIG. 7 illustrates the case where the directions of the motion compensation prediction of the pixel value and the motion compensation prediction of the shape data match, as shown in FIG. When the directions of the solid data arrow and the motion compensation prediction of the shape data (dotted arrows) do not match (when the direction of the motion vector of the pixel value does not match the direction of the motion vector of the shape data), the motion compensation of the pixel value is performed. The used motion vector (hereinafter, referred to as “first motion vector”) cannot be used as it is as a motion compensation vector of the shape data (hereinafter, referred to as “second motion vector”).

Therefore, in such a case, the motion vector generating section 501A shown in FIG. 5 obtains the second motion vector from the first motion vector by the method shown in FIG.
Here, FIG. 9 is a diagram for explaining a method of obtaining the second motion vector from the first motion vector, and is a virtual diagram viewed from the side with the respective frames overlapped.
Each frame is divided into a plurality of blocks.

The method of obtaining the second motion vector (the motion vector of the shape data) when the directions of the motion compensation using the first motion vector and the motion compensation using the second motion vector do not match will be specifically described. FIG. 9 (a)
FIGS. 8A and 8B are diagrams showing a method of obtaining a second motion vector when directions of motion compensation using a first motion vector and motion compensation using a second motion vector do not match, as indicated by reference numerals Pa and Pb in FIG. It is.

First, in the case where the direction of the motion compensation is indicated by the symbol Pa in FIG. 8, as shown in FIG.
The direction of the first motion vector MV1a representing the motion vector of the pixel value on the frame 2 with respect to is inverted to obtain a vector MV3a indicated by a chain line. This vector MV3
a represents a motion vector on frame 1 with respect to frame 2 and does not necessarily represent a motion vector at the center position of a block of frame 1.

Then, the first motion vector MV1a is obtained for all the blocks on frame 2, and the corresponding motion vector MV3a on frame 1 is obtained.
The second motion vector MV2a at the center position of the block on the frame 1 is obtained by weighting and averaging the peripheral motion vectors MV3a. The weight of the weighted average is, for example, a function of the distance between the center position of the block and the position of each vector MV3a, and the closer the distance, the larger the weight value. As described above, the motion vector (MV2a) for performing motion compensation of the shape data indicated by the symbol Pa in FIG. 8 is obtained.

In the case where the direction of the motion compensation is indicated by the symbol Pb in FIG. 8, as shown in FIG. 9B, first, the vector MV1b on the frame 3 is extended to change the vector MV3b indicated by the dashed line. Ask. This vector MV3b represents a motion vector of a pixel value from frame 4 to frame 3. The vector MV3b represents a motion vector of a pixel on the frame 4, but does not always represent a motion vector at the center position of a block of the frame 4.

Then, the vector MV1b is obtained for each block on the frame 3, and the vector MV1b corresponding to the vector MV1b is obtained.
After obtaining the upper vector MV3b, the vector MV2b at the center of the block of the frame 4 is changed to the peripheral vector MV3b.
From the weighted average. The method of weighted averaging is the same as described above. As described above, the second motion vector (MV2b) for performing the motion compensation of the shape data indicated by the symbol Pb in FIG. 8 is obtained.

As described above, even when the first motion vector cannot be used as it is as the second motion vector,
The second motion vector may be obtained from the first motion vector by the method as described above, and may be applied to the apparatus of the present embodiment.

Next, a description will be given of an apparatus for encoding and decoding image data according to a fourth embodiment of the present invention. According to the apparatus of the third embodiment, the encoding of the second motion vector is not required, but the second motion vector is indirectly obtained from the first motion vector. May not be accurately represented. Therefore, in the present embodiment, the motion vector detecting unit 101 shown in FIGS.
The motion vector obtained in A is efficiently encoded, and a large increase in the amount of encoded data is avoided while maintaining the accuracy of the motion vector.

FIG. 6 is a block diagram showing a part of the configuration of the encoding apparatus and the decoding apparatus according to the present embodiment. In the figure, reference numeral 601 denotes a motion vector creation unit, and 602 denotes a motion vector encoding unit. The motion vector creation unit 601
A motion vector used for prediction of a pixel value is input and a predicted motion vector is generated by a method similar to the motion vector generation unit 501A illustrated in FIG.

The motion vector coding unit 602 predicts the input shape data motion vector based on the predicted motion vector input from the motion vector creation unit 601 and calculates a prediction error vector representing an error between them. Is encoded. The encoded prediction error vector data is sent to an unillustrated encoded data integration unit, integrated with other encoded data, and stored or transmitted.

Normally, regarding the image object (part area),
There is a strong correlation between the motion vector of the pixel data and the motion vector of the shape data, and the motion vector of the pixel data and the motion vector of the shape data are similar. For this reason, the code amount of the prediction error vector is reduced, and the motion vector of the shape data is efficiently coded.

[0084]

As is clear from the above description, according to the present invention, the following effects can be obtained. According to the invention described in claims 1 to 7 (corresponding to the first and second embodiments), the shape information (part region shape) of an image to be used for the synthesis of the lower layer is stored in the already encoded lower layer. Alternatively, since encoding / decoding is performed using motion compensation prediction from the shape information (part region shape) of the image object of the upper layer, the image data is efficiently encoded using the inter-frame correlation positively. / Decrypt.

The shape information of the image object used for the synthesis of the lower layer and the shape information of the image object of the upper layer are rearranged in the order of the encoding of the pixel values, and the rearranged shape information of the image object is subjected to motion compensation prediction. Encoding / decoding using
It is possible to efficiently encode / decode the shape information of the two types of image targets of the lower layer and the upper layer.

According to the invention described in claims 8 to 11 (corresponding to the third embodiment), in the invention described in claims 2 to 5, the motion vector used for predicting the shape information of the image object is represented by a pixel value. Since it is configured to be obtained from the motion vector used for the prediction of, the encoding of the motion vector of the shape information of the image target becomes unnecessary, and the scale of the encoded data can be reduced.

According to the invention as defined in claims 12 to 15 (corresponding to the fourth embodiment), the invention according to claims 2 to 5 is used for predicting shape information (part region shape) of an image object. At the time of encoding a motion vector, an error (difference information) from a predicted motion vector obtained from a motion vector used for prediction of a pixel value of a frame to which the shape information of the image belongs belongs is configured. It is possible to efficiently encode and decode image data while maintaining the accuracy of a motion vector that predictively encodes shape information of an image target.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an encoding device according to a first embodiment of the present invention.

FIG. 2 is a block diagram illustrating a configuration of a decoding device according to the first embodiment of the present invention.

FIG. 3 is a block diagram illustrating a configuration of an encoding device according to a second embodiment of the present invention.

FIG. 4 is a block diagram illustrating a configuration of a decoding device according to a second embodiment of the present invention.

FIG. 5 is a block diagram illustrating a configuration of a prediction encoding unit / prediction decoding unit of an encoding device / decoding device according to a third embodiment of the present invention.

FIG. 6 is a block diagram illustrating a configuration of a prediction encoding unit / prediction decoding unit of an encoding device / decoding device according to a fourth embodiment of the present invention.

FIG. 7 is a diagram for explaining the concept of predictive encoding / predictive decoding according to the third embodiment of the present invention (when a pixel value and a motion vector of shape data match).

FIG. 8 is a diagram for explaining the concept of predictive encoding / predictive decoding according to the third embodiment of the present invention (when a pixel value and a motion vector of shape data do not match).

FIGS. 9A and 9B are diagrams for explaining a method of obtaining a motion vector of shape data from a motion vector of a pixel value.

FIG. 10 is a diagram for explaining a concept of integrating image data having different frame rates into one moving image sequence.

FIGS. 11A to 11C are diagrams for explaining the concept of conventional frame synthesis.

[Explanation of symbols]

 101 first shape coding unit 102 prediction coding unit 102A motion vector detection unit 102B motion compensation prediction unit 102C memory 103 second shape coding unit 201 first shape decoding unit 202 prediction decoding unit 202A motion compensation prediction unit 202B memory 203 2 shape decoding unit 301 shape data rearranging unit 304 shape encoding unit 402 shape decoding unit 404 shape data reverse rearranging unit

Claims (15)

[Claims] An image data encoding / decoding method for integrating image data of a lower layer having a low frame rate and an upper layer having a high frame rate and encoding and decoding the same as one moving image sequence, comprising: On the encoding side, on the condition that a frame corresponding to the frame position of the upper layer does not exist in the lower layer, the shape information of the image object of the lower layer positioned temporally before or after the frame is (B) on the decoding side, predictively decodes the predicted and coded shape information of the image object to synthesize the non-existent lower layer frame, and codes on the encoding side. A method for encoding / decoding image data, wherein the image data is decoded. 2. An image data encoding apparatus that integrates image data of a lower layer having a lower frame rate and an upper layer having a higher frame rate and encodes the moving image sequence as one moving image sequence. On the condition that the frame corresponding to does not exist in the lower layer, the shape information of the image object belonging to the frame of the lower layer positioned temporally before or after the frame is already encoded in the lower layer or An image data encoding device, which performs predictive encoding with reference to shape information of an image object of an upper layer. 3. An image data decoding apparatus for decoding image data encoded as one moving image sequence by integrating a lower layer with a low frame rate and an upper layer with a high frame rate, comprising: A lower layer in which shape information of an image object belonging to a frame of the lower layer located temporally before or after the frame is decoded, provided that a frame corresponding to a frame position does not exist in the lower layer. Alternatively, a decoding apparatus for image data, wherein predictive decoding is performed with reference to shape information of an image object of an upper layer, and the non-existent lower layer frame is synthesized. 4. A method for predictively encoding shape information of an image object belonging to a frame of an upper layer with reference to at least shape information of an image object belonging to a frame of a lower layer or an upper layer which has already been encoded. Claim 2
2. An image data encoding device according to claim 1. 5. The method according to claim 1, wherein at least the shape information of the image object belonging to the frame of the upper layer is decoded by referring to the shape information of the image object belonging to the frame of the lower layer or the upper layer which has already been decoded. Item 3. An image data decoding device according to Item 3. 6. The image data according to claim 2, wherein a motion vector of the shape information of the image target is encoded when predictive coding of the shape information of the image target is performed. Encoding device. 7. The decoding of image data according to claim 3, wherein, when predictive decoding of the shape information of the image target, a motion vector of the shape information of the image target is decoded. apparatus. 8. The shape of the image object, provided that a motion vector for predictively encoding shape information of the image object matches a motion vector for encoding pixel information of a frame to which the image object belongs. 5. The image data encoding apparatus according to claim 2, wherein a motion vector for encoding the pixel information is used when predictive encoding of information is performed, and the motion vector is encoded. Device. 9. The shape of the image object, provided that a motion vector for predictively encoding shape information of the image object matches a motion vector for encoding pixel information of a frame to which the image object belongs. The image data decoding apparatus according to claim 3, wherein, when predictive decoding of information, a motion vector for encoding the pixel information is decoded. 10. The shape of the image object, provided that the motion vector for predictive encoding of the shape information of the image object does not match the motion vector for encoding the pixel information of the frame to which the image object belongs. 5. The image data encoding apparatus according to claim 2, wherein a motion vector for encoding the pixel information is used when predictive encoding of information is performed, and the motion vector is encoded. Device. 11. The shape of the image object provided that the motion vector for predictive encoding of the shape information of the image object does not match the motion vector for encoding the pixel information of the frame to which the image object belongs. Upon predictive decoding of information, a motion vector for encoding the pixel information is decoded, and a motion vector for predictively decoding the shape information of the image target based on the decoded motion vector is obtained. Any one of claims 3 and 5
A decoding device for image data according to the item. 12. The shape of the image object, provided that the motion vector for predictive encoding of the shape information of the image object does not match the motion vector for encoding the pixel information of the frame to which the image object belongs. Upon predictive encoding of information, encoding a motion vector to obtain a motion vector for predictive encoding of the shape information of the image target based on a motion vector for encoding the pixel information; The image data encoding device according to any one of claims 2 and 4. 13. The shape of the image object, provided that a motion vector for predictive coding of shape information of the image object does not match a motion vector for coding pixel information of a frame to which the image object belongs. 6. The method according to claim 3, wherein, when predicting and decoding information, a motion vector for predictively decoding the image object is decoded based on a motion vector for encoding the pixel information.
The image data decoding device according to any one of the above. 14. When predictive encoding of shape information of an image object is performed, a prediction error vector of a motion vector of the shape information of the image object with respect to a motion vector of a pixel value of the image object is obtained, and the prediction error vector is encoded. The image data encoding apparatus according to claim 2, wherein the encoding is performed. 15. When decoding shape information of an image target, predictive decoding of shape information of the image target by decoding a prediction error vector of a motion vector of the shape information of the image target with respect to a motion vector of a pixel value of the image target. The image data decoding apparatus according to claim 3, wherein a motion vector for calculating the motion vector is obtained.