JP2000324491A - Video encoding device and video decoding device - Google Patents

Abstract

(57) Abstract: An object of the present invention is to realize the same error tolerance as that of a conventional rectangular image in image coding of an arbitrary shape. An encoding unit encoding an input moving image.
101, important information reconstructing means 102 for extracting important information from the encoded information in the encoding means, synchronization signal generation means 103 for generating a synchronization signal, and synchronization generation means for the bit sequence encoded by the encoding means. And a bit string reconstructing means 104 for reconstructing a bit string by adding the synchronization signal output from the PID and important information reconstructed by the important information reconstructing means.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for converting encoded moving images / still images into ISDN (Integrated Services Digital).
Network), an information transmission system for transmitting data using a wired communication network such as the Internet, or a wireless communication network such as PHS (registered trademark) or satellite communication, and an encoding / decoding device in an information transmission system to which the method is applied. .

[0002]

2. Description of the Related Art In recent years, with the development of digital coding technology for images and other types of information and broadband network technology, applications using these technologies have been actively developed. Systems for transmitting data using a network have been developed.

For example, in a videophone, a video conference system, and a digital television broadcast, a moving image and a sound are compression-encoded to a small amount of information, respectively, and the compressed moving image code sequence, audio code sequence, and other data code sequence are encoded. Are multiplexed into one code string and transmitted / stored.

Techniques such as motion compensation, discrete cosine transform (DCT), sub-band coding, pyramid coding, and variable-length coding have been developed as compression coding techniques for moving picture signals, and schemes combining these techniques have been developed. I have. In addition, ISO MPEG-1,
MPEG-2, ITU-TH. 261, H .; 262
H. There are ISO MPEG system, ITU-T as an international standard method for multiplexing a moving image, a code string obtained by compressing a sound / audio signal and other data.
H. 221, H.R. 223 are present.

In a conventional moving picture coding method such as the above-mentioned moving picture coding international standard method, a moving picture signal is divided for each frame, and a frame is divided into small areas, such as GOB (Group Of Block) or the like. Encoding is performed for each unit such as a macroblock, and header information indicating an encoding mode and the like is added to each frame, GOB, and macroblock. These header information are necessary information for decoding the entire frame, GOB, and the like.

For this reason, if an error is mixed in the header information in the transmission line / storage medium and the decoding cannot be performed correctly in the moving picture coding apparatus, the frame including the header information, the entire GOB, etc. are correctly decoded. Therefore, the quality of the reproduced moving image in the moving image decoding apparatus is greatly deteriorated.

That is, when transmitting a compressed and coded image using a communication network, the receiving side decodes meaningful information from the transmitted "0" / "1" bit string. Processing is required.

For this purpose, the header information described above is very important as information indicating under what rules a certain unit of a bit string has been encoded. The header information is, for example, the prediction type of the frame currently being decoded (whether it is encoding within the frame,
Whether the coding is between frames), information (time reference) indicating the timing at which the frame is displayed, or step size information when performing quantization.

Therefore, if the header information is lost, information transmitted thereafter cannot be correctly decoded.

For example, despite the fact that the prediction type of the frame originally indicates that the coding is between frames, an error is mixed in the bit string for some reason, and the bit pattern indicating the coding within the frame is changed to the bit pattern. Let's say it has changed.
In this case, even if the subsequent actual information is correctly transmitted, the decoding side determines that the signal is the result of the intra-frame encoding, so that the decoding is not correctly performed finally.

Therefore, the quality of the reproduced moving picture in the moving picture coding apparatus is greatly deteriorated.

Such mixing of errors frequently occurs particularly when a system for transmitting / accumulating moving images via a wireless transmission path is used, such as a wireless videophone, a portable information terminal, and a wireless digital television receiver. .

Conventionally, image transmission uses a system using a wired communication network, and it is assumed that satellite communication with a very low error rate is used even if a wireless communication network is used. Therefore, sufficient consideration has not been given to the error resistance of the structure of the coded sequence to be transmitted itself, and the transmission path error protection for important information such as header information has not been sufficient.

On the other hand, the error rate of PHS (simplified portable telephone), which is expected to become one of the mainstreams of mobile communication in the future, is about 100,000 to 1,000,000 times that of satellite communication. Performing error correction on the encoded bit string alone makes it impossible to perform sufficient correction.

[0015] In the Internet, which is expected to become the mainstream of communication in the future like PHS, it is not statistically clear when and what kind of errors will be introduced.
In some cases, appropriate error correction cannot be performed.

[0016]

In the conventional configuration, the information that can be duplicated using the HEC does not include information necessary for encoding an image of an arbitrary shape.
There has been a problem that, when an image of an arbitrary shape is decoded in units of objects and VOP header information is lost due to a transmission error, decoding cannot be performed correctly.

Therefore, there is a problem that when transmitting a code string coded by using image coding of an arbitrary shape, the error tolerance of transmitted data is reduced.

SUMMARY OF THE INVENTION An object of the present invention is to provide a moving picture coding apparatus and a moving picture coding data which can provide the same error resistance as the conventional rectangular picture coding even in the case of coding of an arbitrary shape picture. It is an object of the present invention to provide a transmission method and a recording medium for recording the data.

[0019]

According to a first aspect of the present invention, there is provided an encoding section which encodes an input moving image to generate a bit string,
An important information configuration unit that extracts and configures important information as information indicating under what rule a group of certain bit strings is encoded from the encoded information obtained by the encoding unit; A synchronizing signal output from the synchronizing signal generator and important information reconstructed by the important information forming unit are added to the bit sequence encoded by the encoding unit, and the bit sequence is reconstructed. There is provided a moving image encoding device having a bit sequence reconstructing unit.

According to a second aspect of the present invention, in the first aspect, the important information forming unit forms normal image related information which is important information in a normal coding method in which a frame is coded from the coded information in units of rectangular regions. An image-related important information forming unit and an arbitrary-shape image-related important information that constitutes arbitrary-shape image-related important information, which is important information in an arbitrary-shape encoding method that encodes an image in a frame in units of an arbitrary-shape image from encoded information. An information configuration unit, an arbitrary shape encoding determination unit that determines whether the image being encoded from the encoded information is an arbitrary shape image, and when the arbitrary shape encoding determination unit determines that the image is an arbitrary shape image. Provided is an image coding apparatus including a switching unit that outputs important information related to an arbitrary shape image and a multiplexing unit that multiplexes normal image related information and an output of the switching unit.

According to a third aspect of the present invention, a moving image is encoded to obtain a bit string including synchronization information, and information indicating a rule of a certain group of bit strings in this encoding is encoded. A decoding unit that decodes encoded data including a bit string obtained by adding important information as header information to an encoded bit string, a decoding unit that decodes an image bit string from an input bit string, and a decoding unit that decodes the image bit string. A synchronization signal detection unit that detects a synchronization signal from the bit string and notifies the decoding unit, and an error check unit that determines whether an error exists in decoding information of the decoding unit.
When the error check unit determines that there is no error, the present invention provides an image decoding apparatus including an important information configuration unit configured to configure important information from header information output from the decoding unit and to notify the decoding unit.

According to a fourth aspect, in the third aspect, the important information forming unit includes a normal image related important information forming unit that forms normal image related important information from the header information, and an image decoded from the header information. An arbitrary shape encoding determining unit that determines whether or not the image is an arbitrary shape image, an arbitrary shape image related important information configuring unit that configures the arbitrary shape image related important information from the header information, and the arbitrary shape encoding determining unit determines that the image is an arbitrary shape image In this case, the first switching unit that inputs the header information to the arbitrary shape image-related information reconstructing unit and the arbitrary shape image-related important information are output when the arbitrary shape encoding determination unit determines that the image is the arbitrary shape image. An image decoding device including a second switching unit is provided.

According to a fifth aspect of the present invention, there is provided an encoding unit for encoding an input moving image to obtain a bit sequence, and a set of constant bit sequences is determined based on encoding information obtained by the encoding unit based on what rules. An important information configuration unit that extracts and configures important information as information indicating whether the important information is encoded, a bit string division unit that divides a bit string encoded by the encoding unit, and an important information that is reconstructed by the important information configuration unit. An image decoding apparatus having a packet header creating unit that creates a packet header from information, and a packet forming unit that forms a packet using the bit string divided by the bit string dividing unit and the packet header generated by the packet header creating unit I will provide a.

According to a sixth aspect of the present invention, the important information forming unit according to the fifth aspect comprises an arbitrary shape image-related important information forming unit for forming the arbitrary shape image-related important information from the coded information, and an arbitrary shape image forming unit from the coded information. An extension header insertion determination unit that determines whether to add an extension header holding related important information to the packet header, and, when the extension header insertion determination unit determines that the extension header is to be inserted, the header information is transferred to an arbitrary shape image. An image decoding device comprising a switching unit for inputting information.

According to a seventh aspect of the present invention, a moving image is encoded to obtain a bit string, and important information as information indicating under what rule a group of certain bit strings in this encoding is encoded. A decoding device for decoding moving image encoded data having a bit sequence obtained by adding a character string as packet header information, comprising: a separating unit that separates an image bit sequence and packet header information from an input bit sequence; and a decoding unit that decodes the image bit sequence. And an error check unit that determines whether or not there is an error from the decoding information of the decoding unit, and indicates under what rules a set of certain bit strings is encoded from the packet header information. The important information as information is taken out, the important information is reconstructed, and the error check unit determines that there is an error, and the important information is used for decoding. To provide an image decoding apparatus and a key information composing unit that notifies the decoding unit key information the structure if necessary.

According to an eighth aspect of the present invention, the important information forming unit according to the seventh aspect comprises an arbitrary shape image-related important information forming unit that forms the arbitrary shape image-related important information from the packet header information, and an extended header from the packet header information. An extension header insertion determining unit for determining whether or not the packet is added to the packet header;
Provided is an image decoding apparatus comprising: a switching unit that inputs packet header information to important information relating to an arbitrary shape when the extension header insertion determination unit determines that an extension header is inserted.

According to the present invention, in moving picture coding, coded data is provided with a header, an extended header portion is further provided in the header, and stored in the header. Since the extension header can include important information in the arbitrary-shape image coding method, even if the header is partially broken, an image can be decoded for a portion having a sound header. Further, by inserting a synchronization signal into the image data, the problem of loss of synchronization of the video packet VP can be solved. That is, the video packet VP is a packet that starts with the synchronization signal RM, and can be resynchronized with the synchronization signal RM even if an error occurs before that and an out of synchronization occurs.

From these facts, it can be seen that moving picture coding which has high resistance to noise at the time of transmission and which has the same error resistance as that of conventional rectangular picture coding even in the case of arbitrary shape picture coding. Technology can be provided.

[0029]

Embodiments of the present invention will be described below with reference to the drawings.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 shows a basic configuration of a moving picture coding apparatus according to the first embodiment of the present invention. According to this, the output of the encoding section 101 is connected to the important information forming section 102 and the bit string reconstructing section 104. Important Information Composition Unit 10
The output of 2 is connected to the bit string reconstruction unit 104 together with the output of the synchronization signal generation unit 103. Bit string reconstruction unit 104
Are connected to the multiplexing unit 105. Multiplexing unit 105
Is connected to the transmission line 106.

The encoding unit 101 receives the input image signal 1
31 is encoded and output to the bit string reconstruction unit 104, and encoded information 133 at the time of encoding is output to the important information construction unit 102. Important Information Composition Unit 10
2 is encoded information 133 when encoded by the encoding unit 101
Then, only important information 134 necessary for decoding is selected and output.

The synchronizing signal generating section 103 generates the synchronizing signal 135 at an arbitrary interval.
04 inserts the synchronizing signal 135 from the synchronizing signal generator 103 into the bit string 132 and, after that, inserts the important information 134 output from the important information forming unit 102 if necessary.
It is configured to insert and output in a predetermined format.

The multiplexing unit 105 includes a bit string reconstructing unit 10
The bit stream 136 reconstructed in 4 is multiplexed with other data (for example, audio data, a bit string obtained by encoding another object, etc.) and output to the transmission path / storage medium 106 as a multiplexed bit stream 137. It is composed of

In the above configuration, the input video signal 131 of the moving image is encoded by the encoding unit 101. The bit sequence 132 encoded and output by the encoding unit 101 is input to the bit sequence reconstruction unit 104. The encoded information 133 encoded by the encoding unit 101 is input to the important information forming unit 102, and only important information 134 necessary for decoding is selected and output.

In the bit string reconstruction unit 104, the synchronization signal 135 output from the synchronization signal generation unit 103 at arbitrary intervals is output.
Is inserted into the bit string 132, and after that, if necessary, important information 134 output from the important information forming unit 102 is inserted in a predetermined format.

The bit sequence 136 reconstructed by the bit sequence reconstructing unit 104 is input to the multiplexing unit 105,
(E.g., audio data, bit strings encoding other objects, etc.)
And a multiplexing process is performed, and a multiplexed bit string 137 is output to the transmission path / storage medium 106.

As described above, in the present embodiment, the bit stream obtained by encoding a moving image is added to the synchronization signal generator 1 at an arbitrary interval.
03 is inserted, and after that, if necessary, the important information 134 output from the important information forming unit 102 is inserted by the bit string reconstructing unit 104 according to a predetermined format.

Therefore, information necessary for encoding / decoding an image of an arbitrary shape in MPEG-4, for example, information on the width VW of the image size and the height VH in the encoding of an image of an arbitrary shape
Of the image position and information of the x coordinate VHMSR of the image position for indicating the position at which the decoded image is displayed, and the y coordinate VVMSR
VOP indicating the encoding mode of the shape information
Shape coding type “vop_shape_coding_t
ype (VSCT) "or change_conv_ which is a flag indicating whether encoding is performed after converting the size of the shape information.
A ratio_disable (CCRD) is generated as important information 134. When this important information is duplicated and inserted into the VP header by the bit sequence reconstruction unit 104 in a predetermined format, arbitrary shape image encoding can be given the same error tolerance as rectangular image encoding. Thus, even if the VOP header or the VP is partially broken, the moving image can be decoded.

In the case of MPEG-4, a frame equivalent is referred to as a video object plane "Video Objec".
t Plane ”(Fig. 18).
A Plane (hereinafter, referred to as a VOP) can be divided into a plurality of packets, which is referred to as a video packet “Video Packet” (FIG. 19).

A video packet “Video Packet” (hereinafter referred to as VP) is a synchronization signal (Resync Marker,
(Hereinafter referred to as RM), and it is possible to perform resynchronization with this synchronization signal even if an error exists before that and an out-of-synchronization occurs.

Therefore, video packets VP other than the head
Then, even if information is destroyed / lost due to errors,
Subsequent video packets VP can be decoded correctly. It is a video object plane VOP
This is because the VOP header at the head of has been successfully decoded, and all the information necessary for decoding is prepared (FIG. 20).

As described above, the VOP header information includes the coding type of the video object plane VOP (intra-frame coding, inter-frame coding, etc.), time reference, step size, and the like. When this information was lost, all video packets VP could not be decoded (FIGS. 21A and 21B).

Therefore, in MPEG-4, a header extension code HEC is defined in the header of the video packet VP, and the value
Important information in the header could be described again.

Regarding this format, FIGS.
1B. That is, as shown in FIG. 21A, the video object plane VOP has a pattern in such a format that a VOP header and subsequent data come at the head, followed by a video packet VP header and then data. , Etc., in a format in which several patterns of such a format are repeated.

If a header extension code HEC is defined in the header of the VP and important information in the VOP header is described again by using this value, the video object plane VOP is broken. Unless otherwise, even if one or two pairs of data of the video packet VP and the data following it are broken, a normal VP header and data pair can be decoded using the information of the VOP header and the data. Met.

In the case of the example of FIGS. 22A and 22B, the header of the video object plane VOP and the data forming the pair are not broken, but the first video packet V
This shows a state in which only the header of P and the data forming a pair with the header are broken. In this case, as shown in FIG. 22B, since the video object plane VOP and its data portion are not broken, the first area of the image is decoded normally, and the next second area is degraded due to an error. Is decoded, and the subsequent third and subsequent areas are decoded normally.
Although partially broken, most could be decoded as beautifully reproduced images.

In the case of a system without the header extension code HEC, as shown in FIG.
If the VOP header is broken, there is no room for restoration even if other VPs are not broken, and as shown in FIG. 22B, no image can be obtained due to the influence of errors. -In the case of the method in which the extension code HEC was provided, the reproduction was possible. That is, in the case where the header extension code HEC is provided, when the header extension code HEC is true, important information is duplicated after the HEC, and when the HEC is false, important information is duplicated. Not taken form is taken. When using a transmission path with many errors, H
The EC is set to true, and important information is duplicated and added after the EC.

As a result, as shown in FIG. 23A, even if an error exists at the head of the VOP and decoding is not possible, the image in the head area can be restored normally by using the information protected by HEC. If not, it is possible to decode the video packet VP in the portion after the second region, and it is partially broken as shown in FIG.
Most of them can be decoded as beautifully reproduced images.

However, this can only be realized in units of rectangular image areas. That is, V
By duplicating the OP header information using HEC in the VP header, even if the VOP header is lost, the VP
If the VOP header is duplicated by the HEC in the header, the subsequent data can be correctly decoded by using the VOP header. However, the information that can be duplicated by the HEC includes an image of an arbitrary shape. It does not contain information required for encoding. For this reason, although there was no problem with a conventional rectangular image, decoding was not possible in a method such as MPEG-4 in which an image of an arbitrary shape could be encoded in object units.

This is because the encoding of an image of an arbitrary shape in an object unit in MPEG-4 adds more header information than the encoding of a rectangular image, and this is the object of duplication. And the inability to do so was a major problem.

Further, another viewpoint will be considered. As the use of the Internet, intranets, and the like has become generalized, communication using such networks has increased, and Internet videophones and the like have also been used. In this case, the moving image is transmitted in real time. However, when a moving image is transmitted in real time over the Internet, an intranet, or the like, there are many problems with the generally used TCP and UDP protocols. In particular, the problem is that the header has no time information.

Therefore, in recent years, as a protocol used for transmitting moving image / audio data, RTP (Real-time Trans
fer Protocol) is gaining attention. That is, TCP
In such a protocol, there is no time information attached to each packet, so that the receiving side cannot know when to reproduce the received data. Therefore, when data is transmitted in packets, the receiving side cannot successfully reproduce the data if the data is moving image data or audio / sound data.

However, in the RTP, time information is added to each packet, and the receiving side can reproduce moving image data and audio / sound data based on the time information. Thus, RTP is a protocol suitable for transmitting real-time data.

In this protocol, an extension header can be defined for each application.

As described in the prior art, MPEG-4
Then, by duplicating the VOP header information using the HEC in the VP header, even if the VOP header is lost, if the VOP header is duplicated by the HEC in the VP header, it can be used. Subsequent data could be correctly decoded.

However, the information that can be duplicated using the HEC does not include information necessary for encoding an image of an arbitrary shape. Therefore, although there was no problem with the conventional rectangular image, since the header information is added in the encoding of an arbitrary shape compared to the encoding of the rectangular image, it is a big problem that this cannot be duplicated. there were.

For example, in image coding of an arbitrary shape, each VO
Since the image size changes for each P, the width vop_width (hereinafter referred to as VW) and the height v of the image size are included in the VOP header.
op_height (hereinafter referred to as VH) is described. Also, the x coordinate vop_horizontal_mc_spatial_ref (hereinafter, VHMSR) of the image position for indicating the position at which the decoded image is displayed
, Y coordinate vop_vertical_mc_spatial_ref (hereinafter, referred to as VVMSR). FIG. 24 shows the relationship between these values.

The video packet VP without these information
If an attempt is made to decode a moving image using only the information described above, it will not be possible to decode correctly with image coding of an arbitrary shape.
That is, in image coding of an arbitrary shape, the width V of the image size
Decoding could not be performed correctly without the W information, the height VH information, and the x coordinate VHMSR information and the y coordinate VVMSR information of the image position for indicating the display position of the decoded image.

VO indicating the encoding mode of the shape information
P shape coding type “vop_shape_coding
_type (hereinafter, referred to as VSCT) "and change_conv_ratio_disable (hereinafter, referred to as CCRD), which is a flag indicating whether or not encoding is performed after converting the size of the shape information, are necessary for correct decoding.

The duplication of the VOP header by the MPEG-4 HEC does not protect such information.

As described above, the present system can provide the same error resistance as that of the conventional rectangular image encoding even in the case of encoding an arbitrary-shaped image. In the above configuration, the important information forming unit 102 and the bit string reconfiguring unit 104, which are important components of the present invention, will be described in detail with reference to FIG.

First, details of the important information forming unit 102 will be described.

As shown in FIG. 2, the important information forming unit 102 includes a normal image-related important information forming unit 206, an arbitrary shape image-related important information forming unit 207, and an arbitrary shape encoding determining unit 20.
8, a multiplexing unit 210.

Of these, the normal image-related important information forming unit 206 receives the coded information 133 from the coding unit 101 and receives information (for example, coding information) determined to be important in normal coding. Mode, time reference, etc.)
Is selected and output to the multiplexing unit 210 as the normal image-related important information 238. The arbitrarily-shaped image-related important information forming unit 207 outputs important information related to arbitrarily-shaped image encoding (for example, image size, position, encoding mode,
(Reduction conversion mode, etc.), and outputs this as the arbitrary shape image-related important information 239.

The arbitrary shape encoding judging section 208 judges whether the encoded image is a normal rectangular image or an arbitrary shape image, and outputs the judgment result as a judgment signal 240. .

The switching unit 209 includes an arbitrary shape encoding determination unit 2
08 and the arbitrary shape image-related important information 239 from the normal image-related important information forming unit 207 in response to the determination signal 240 from
Is switched to output whether or not to output to the multiplexing unit 210. The multiplexing unit 210 includes the normal image-related important information forming unit 2
It is configured to multiplex the normal image-related important information 238 from No. 06 and the arbitrary shape image-related important information 239 output from the arbitrary shape encoding determination unit 208 in the arbitrary shape image encoding, and to output the resulting information as important information 134.

In the above configuration, the encoding unit 10
1 is input to the normal image-related important information forming unit 207, which is a component of the important information forming unit 102. (For example, an encoding mode or a time reference) is selected, and the selected information is output to the multiplexing unit 210 as normal image-related important information 238. Therefore, the normal image-related important information 238 is a collection of information that is determined to be important at the time of normal encoding, such as an encoding mode and a time reference.

Next, an arbitrary shape image related important information forming unit 2
At 07, important information related to arbitrary shape image encoding (for example, image size, position, encoding mode, reduction conversion mode, and the like) is selected, and arbitrary shape image related important information 2 is selected.
38 and output to the multiplexing unit 210.

On the other hand, the arbitrary shape encoding determining section 208
It is determined whether the encoded image is a normal rectangular image or an image of an arbitrary shape, and the result of the determination is output as a determination signal 240. This determination signal 24
0, the switching unit 209 is controlled, and the arbitrary shape image related important information 23 from the normal image related important information forming unit 207 is controlled.
9 is controlled to be output.

The multiplexing section 210 multiplexes the normal image-related important information 238 and the arbitrary-shape image-related important information 239 in the case of arbitrary-shape image coding, and outputs the result as important information 134.

As a result, in the case of arbitrary shape image encoding, the multiplexing unit 210 can output the important information 134 in which the normal image related important information 238 and the arbitrary shape image related important information 239 are multiplexed. In the case of normal image coding, only the normal image related important information 238 can be output as important information 134.

Next, the details of the bit string reconstruction unit 104 will be described. As shown in the upper half area of FIG. 2, the bit string reconstruction unit 104 includes an MB boundary determination unit 201, a counter 202, a synchronization signal insertion determination unit 203, a header information insertion unit 205, and an addition unit 206.

Among them, the MB boundary determining unit 201
For the bit string 132 encoded and input by the encoding section 101 in the preceding stage, it is determined whether or not the data of the bit string falls on the boundary of the macroblock MB. This is for counting the code amount of the bit string 132 encoded and input by the encoding unit 101 of FIG.

The synchronization signal insertion determining unit 203 determines that the MB boundary determining unit 201 determines that the bit string 132 is an MB boundary, and that the count value of the code amount counter 202 for the bit string 132 exceeds a certain value. , And an insertion permission signal 234.

The header information insertion unit 205 creates header information from the input important information 134 and the synchronization signal 135, and when the synchronization signal insertion determination unit 203 determines that insertion is permitted, the encoded bit string 132 is output to the adding unit 206 in order to add the created header information 237 to the adding unit 206.

The addition section 206 adds the bit string 132 coded and input by the coding section 101 and the output of the header information insertion section 205, and uses this as a reconstructed bit string 136 of the bit string reconstruction section 104. This is the output section.

Bit string reconstructing section 104 having such a configuration
Is a bit string 13 encoded by the encoding section 101 in the preceding stage.
When 2 is input, it is input to the MB boundary determination unit 201 and the code amount counter unit 202, which are one of the components of the bit string reconfiguration unit 104. This MB boundary determination unit 201
It is determined whether or not the input bit string 132 is at the boundary of the MB.

The code amount counter unit 202 counts the code amount of the bit string 132. In the synchronization signal insertion determination unit 203, when the determination in the MB boundary determination unit 201 is determined to be an MB boundary, and the count value of the code amount in the counter 202 exceeds a certain value, the insertion permission signal 2
34, and operates to output it to the header information insertion unit 205.

On the other hand, the header information insertion unit 205 creates header information from the input important information 134 and the synchronization signal 135, and when the synchronization signal insertion determination unit 203 determines that insertion is permitted, the encoded bit string The added header information 237 is added to the adder 206 to add the created header information 237 to the adder 132. As a result, the adder 206 outputs the encoded bit string 132
To the reconstructed bit string 1
36 is output. This bit sequence 136 is the output of the bit sequence reconstructing unit 104.

As a result, the bit string 132 of the image data which is coded and inputted by the coding unit is examined, and when the bit which becomes the boundary position of the macroblock MB arrives, the code amount up to that point is a predetermined value. If so, an insertion permission signal 234 is generated. Input important information 13 created in header information insertion section 205
4 and the header information based on the synchronization signal 135 can be added to the bit string 132.

The important information forming unit 102 determines that the normal image related important information forming unit 206, which is one of the constituents, is important when performing normal coding based on the coded information 133 from the coding unit 101. Information (eg, encoding mode, time reference, etc.) to be used is selected as normal image-related important information 238. In addition, the arbitrarily-shaped image-related important information forming unit 207, which is one of the constituent elements in the important information forming unit 102, stores important information (for example, image size, position, encoding mode, reduction conversion) related to arbitrary-shaped image coding. Mode, etc.), and arbitrarily-shape image related important information 2
39. In the case of a normal image, only the normal image-related important information 238 is obtained, and in the case of encoding an image having an arbitrary shape, the normal image-related important information 238 and the arbitrary shape image-related important information 239 are multiplexed. Therefore, the header information to be inserted into the bit string can include normal image information related important information and arbitrary shape image related important information.
Information necessary for image reproduction of EG-4 encoded data is VP
Can be included in the header.

FIG. 3 shows a flowchart relating to the creation of header information. In the bit string reconstruction unit 104, first, the first
As a step (step S502), it is determined whether or not the bit string input from the encoding unit 101 is at a boundary position of an MB (macroblock).

As the second stage (step S503),
If it is MB, it is determined whether or not the synchronization signal RM should be inserted. This determination can be made by an arbitrary algorithm of the user. For example, an algorithm that inserts the synchronization signal RM when the number of bits exceeds a predetermined number of bits from the immediately preceding synchronization signal, or when the number of MBs exceeds the predetermined number of synchronizations from the immediately preceding synchronization signal, the RM is inserted along the shape in the image. Various methods are available, such as determining whether to insert.

The video packet VP is a packet starting with the synchronization signal RM. Even if an error exists before that and an out-of-synchronization occurs, resynchronization can be performed with the synchronization signal RM.

If it is determined in step S503 that the synchronization signal RM is to be inserted, the RM is inserted, and a VP header following the RM is inserted (step S504 in FIG. 3). Then, the process proceeds to step S505.

In the third stage (step S505), it is determined whether or not important information of the VOP header is duplicated as an extension header.

If it is determined that duplication is to be performed, HEC is set to true, and then important information in rectangular image encoding is selected from the VOP header and output (FIG. 3).
Step S506). Then, the process proceeds to step S507.

In the final fourth stage (step S507), it is determined whether or not the image is an arbitrary-shaped image. In the case of an arbitrary shape image, important information in the arbitrary shape image encoding in the VOL header is selected and output (step S50 in FIG. 3).
8).

Through the above four steps, a VP header portion is generated and inserted into a bit string.

FIG. 4 shows a configuration example of a VP header of an arbitrary-shaped image. An extension header Ex-Header is added to the conventional VP header shown in FIG. 20, and the extension header Ex-Header has important information in arbitrary shape image coding, that is, the width (VW) and height of an image. (V
H), an X coordinate (VHMSR) to paste the image, a Y coordinate (VVMSR), a flag (CCRD) indicating whether or not the shape information is reduced and converted and encoded, and an encoding type of the shape information (intra-frame encoding / Information (VSCT) for inter-frame coding and the like.

The important information in the arbitrary shape image coding is not limited to the above information, and it is possible to further increase other information or to reduce the information depending on the use of the application. . However, it is necessary for the transmitting side and the receiving side to commonly recognize the header format.

As described above, the function of extracting important information in the coding of an arbitrary-shaped image, the function of determining whether or not an arbitrary-shaped image is used, and the function of detecting a macroblock boundary are provided.
The P header is provided with an extended header part, so that important information in normal image coding can be included, as well as important information in arbitrary shape image coding in arbitrary shape image coding, including a synchronization signal. Even if the header is partially broken, an image can be decoded for a portion having a healthy header. Further, since there is a synchronization signal, the problem of out-of-synchronization of the video packet VP is also solved. That is, the video packet VP is a packet that starts with the synchronization signal RM, and can be resynchronized with the synchronization signal RM even if an error occurs before that and an out of synchronization occurs.

From these facts, it is possible to provide a moving image which has high resistance to noise at the time of transmission and has the same error resistance as that of the conventional rectangular image coding even in the case of arbitrary shape image coding. An image coding technique can be provided.

The above has described the configuration and processing on the encoding side in detail. Next, the configuration and processing on the decoding side will be described in detail.

The decoding section will be described. According to the decoding unit according to the first embodiment shown in FIG. 5, the output of the separation unit 302 to which the coded bit string is input is output from the decoding unit 303.
And a synchronization detection unit 304. Synchronization detector 30
4 is connected to the other input of the decoding unit 303. The output of the decoding unit 303 is connected to the error check unit 305. The output of the error check unit 305 is
03 and to the important information configuration unit 306.
The output of the important information forming unit 306 is connected to the decoding unit 303.

The separation section 302 is provided for the transmission path / storage medium 106
From the bit string 331 received from the
2 is provided to separate the data from the other data. The synchronization detection section 304 detects a synchronization signal RM from the bit string 332 output from the separation section 302. In addition, the decoding unit 303 performs a decoding process on the image bit string 332 separated and output from the separation unit 302 to generate image data. At this time, the decoding unit 303 is configured to perform the decoding process while synchronizing with the synchronization signal detected by the synchronization signal detection unit 304.

Further, the important information forming unit 306 obtains the data currently being decoded by the decoding unit 303, and obtains the VOP (video / video data) currently being decoded by the decoding unit 303.
If an object plane (VOP) header exists, the information is extracted, output, and
It is configured to give to.

[0099] The error check unit 305 is
3 to check whether or not an error has occurred during the decoding operation by checking the decoding information 334 output by the third information processing unit 3. If an error is detected, the error checking unit 305 causes the important information forming unit 306 to perform the decoding process. Is configured to notify that there is an error, and to suppress the output of important information to the decoding unit 303.

When an error occurs, the decoding unit 303
It is configured to perform processing corresponding to the error.
After performing processing corresponding to the error, decoding section 303 performs decoding work from the position of the next synchronization signal detected by synchronization detection section 304.

In such a configuration, the bit string 331 received from the transmission path / storage medium 106 is
2 separates the image into a bit string 332 for the image and other data. Other data is sent to the corresponding decoding units.

The image bit string 332 separated by the separation unit 302 is input to the decoding unit 303, and is decoded. At this time, the decoding process is performed while the synchronization signal is detected from the bit sequence 332 by the synchronization signal detection unit 304.

[0103] From the decoded information 334 obtained by performing the decoding process in the decoding unit 303, the error check unit 3
At 05, it is detected whether an error has occurred during the decoding operation.
When an error is detected, a process corresponding to the error is performed by the decoding unit 303, and then decoding is performed from the position of the next synchronization signal detected by the synchronization detection unit 304.

The decoding section 303 determines the type of the next synchronization signal. If the error signal 335 is true in the case of the synchronization signal RM, the important information forming section 306 sends the information 3 of the VOP header.
43 is acquired.

The important information forming unit 306 includes a decoding unit 303
If a VOP (Video Object Plane) currently being decoded has a VOP header, that information is output. If no VOP header is present in the VOP currently being decoded, if HEC has inserted important information into the VP header, the important information is output.

In the decoding process in decoding section 303, the important information obtained in important information forming section 306 is used. When the VOP header currently exists in the VOP (video object plane) currently being decoded by the decoding unit 303, the information is output to the important information obtained by the important information forming unit 306, and the decoding is performed. If there is no VOP header in the VOP, if important information has been inserted into the VP header by HEC, it is output. On the encoding processing side, in addition to important information in normal image encoding as important information, in arbitrary shape image encoding, important information in arbitrary shape image encoding is also included.
Even if the header is partially broken, the part with a healthy header is the data that encodes a normal image, or the data that encodes an image of any shape, Can be decrypted. Further, since there is a synchronization signal, the problem of out-of-synchronization of the video packet VP is also solved. That is, the video packet VP is the synchronization signal RM.
, And it is possible to perform resynchronization with the synchronization signal RM even if an error exists before that and an out of synchronization occurs. From these facts, decoding of a moving picture coding technique which is highly resistant to noise at the time of transmission and which can have the same error resistance as conventional rectangular picture coding even in the case of arbitrary shape picture coding. Technology can be provided.

In addition to important information in normal image coding,
In the arbitrary shape image coding, the important information in the arbitrary shape image coding is also given as header information and transmitted, thereby giving a resistance to noise at the time of transmission. It is important how to extract and pass it to the decoding unit 303 so that it can be used for the decoding process.

Therefore, the characteristic point of this embodiment lies in the important information forming unit 306. Thus, the important information forming unit 306 will be described in detail with reference to FIG.

As shown in FIG. 6, the important information forming unit 306 is composed of a normal image related important information forming unit 307, an arbitrary shape coding judging unit 308, switching units 309 and 311, and an arbitrary shape image related important information forming unit 310. Be composed.

When the VP header is found in the decoding unit 303, the normal image-related important information forming unit 307
It is configured to decode and output encoding mode information, time reference, and the like from the information of the VP header.

The arbitrary shape encoding determination unit 308 determines whether the image currently being decoded by the decoding unit 303 is an arbitrary shape image.
Alternatively, it is a unit that determines whether the image is a conventional rectangular image, and the switching units 309 and 311 are controlled to be switched according to the determination result. The switching units 309 and 311 are two system switching switches.

The arbitrarily-shaped image-related important information constructing unit 310 stores important information (for example, image size,
And the like. In the case of an arbitrary-shaped image, the switching units 309 and 311 are switched so as to be connected to the arbitrary-shaped image-related important information forming unit 310, and important information on the arbitrary-shaped image is reproduced. The important information related to the normal image and the important information related to the arbitrary shape image are also given to the decoding unit 303 by the normal image related important information forming unit 307, and the decoding unit 303 decodes the arbitrary shape image. Is also possible.

In important information forming section 306 having such a structure, decoding section 303 adds VP to the input bit string.
When the header is found, first, the normal image-related important information forming unit 307 decodes the encoding mode information, the time reference, and the like.

Further, the arbitrary shape encoding determining unit 308 determines whether the image currently decoded by the decoding unit 303 is an arbitrary shape image or a conventional rectangular image.
A control signal is generated according to the determination result.

The switching units 309 and 311 are controlled by a control signal from the arbitrary shape encoding determination unit 308. At this time, in the case of an arbitrary-shaped image, the arbitrary-shaped image-related important information forming unit 310 outputs important information (for example,
The important information 343 is finally created and given to the decoding unit 303 as an output of the important information forming unit 306. Thereby, if an extension header is provided in the header and important information relating to the arbitrary shape image is embedded in the extension header, the decoding side extracts the important information and decodes the important information necessary for decoding the arbitrary shape image into the decoding unit 303.
Can be given to.

As described above, according to the present embodiment, even in the case of encoding an arbitrary-shaped image, it is possible to provide the same error resistance as the encoding of a conventional rectangular image.

In the first embodiment and the second embodiment which will be described hereinafter, it is necessary to describe “image size” and “position information” in encoding an arbitrary-shaped image. In the case of MPEG-4, these pieces of information are each represented by 13 bits, and include “image size”,
Since “position information” requires vertical and horizontal information, a total of four pieces of information are required, and the required number of bits is 4 × 13 =
A total of 52 bits of 52 are required. This can be a great deal of redundancy for transmission at low bit rates.
Therefore, this data is compressed and transmitted as much as possible. Hereinafter, the method will be described.

The size of the video object plane VOP is represented by 13 bits in MPEG-4, but there are many cases where all 13 bits are not used. Therefore, a method of reducing the code amount by changing the size expression to a variable length is considered.

The basic policy here is “codeword length” +
It shall be expressed as a set of “value”. As shown in FIG. 7, a combination of a header part representing a code length and a data part following the header part is used. That is, the header “header1” and the header “header
2), the former has a 1-bit configuration, and the latter has a 3-bit configuration.It is adopted in a range from a value 1 to a value 542. The code word length has a 6-bit configuration from the value “3” to the value “6”, and the code word length has a 7-bit configuration from the value “7” to the value “14.” The value “15” to the value “30” , The code word length has an 8-bit configuration, the code word length has a 9-bit configuration from values 31 to 94, and the code word length has a 10-bit configuration from the values “95” to “158”. The code word length is 11 bits from 159 to 286 and the code word length is 12 bits from 287 to 542. The value from 543 to 8222 is The header “header1” and the header “header2” have a 1-bit configuration, and the latter has a 2-bit configuration. And, from the value "543" value until "1054" codeword length of 12 bits,
The codeword length is 1 from the value “1055” to the value “2078”.
It has a 3-bit configuration, and the value “2029” to the value “4126”
Up to a code word length of 14 bits and a value of "4127"
To a value “8222”, the code word length is 15 bits.

In this way, the word length is fixed at 13 bits, but becomes variable from 5 bits to 15 bits depending on the numerical value. As a result, a maximum of 18 bits is required even if the header is included, and the number of constituent bits of 34 bits can be reduced as compared with the conventional 52 bits.

In the case of a small image, it is generally required to be coded at a low bit rate, and in the case of a large image, the coded bit string also has a large size. It seems that there are many.
In this sense, it is effective to assign a short code to a variable size with a small size.

For example, the presentation layer “P
resentation layer "is QCIF (176 pixels x 144
In the case of an image of (pixel), VW and VH are 11 [bit] × 2 = 22 [bit] at the maximum, and 11 [bit] × 2 = 22 [bit] at the maximum for the positional information (VHMSR, VVMSR). With 44 bits, 8 bits can be compressed.

In addition, in the case of the image configuration as shown in FIG. 8, VW = 128 pixels = 10 [bit], VH = 80 pixels = 9 [bit], VHMSR = 32 pixels = 9 [bit], VVMSR = 20 pixels = 8 [bits], a total of 10 + 9 + 8 + 9 = 36 [bits], resulting in a reduction of 16 [bits].

FIG. 9 shows a basic configuration diagram of this modification. FIG.
In the figure, 1001 is a variable length coding unit, 1002 is a variable length code generation unit, and the variable length code generation unit 1002 receives size information and converts it into a code word.
Also, the variable-length encoding unit 1001 receives the input important information 1
031, and reads the size information 1
032 is sent to the variable-length code generator 1002, and the codeword 1033 obtained from the variable-length code generator 1002 is output as a codeword 1034.

In such a configuration, important information 103
When 1 is input, the important information 1031 is input to the variable length coding unit 1001. The variable length coding unit 1001 reads the size information from the input important information 1031 and converts the size information 1032 into the variable length code generation unit 1002.
To generate a codeword 1033.

The variable length coding unit 1001 outputs a code word 1034 obtained by converting the size information into the code word 1033 obtained from the variable length code generation unit 1002.

Although the present embodiment has described MPEG-4, error resilience can be improved by adding the same information to transmission of arbitrary shape encoding other than MPEG-4.

Next, another example will be described as a second embodiment.

FIG. 10 is a basic block diagram of a moving picture coding apparatus according to the second embodiment of the present invention. According to the moving picture coding apparatus shown in FIG. 10, the output of coding section 601 is connected to bit string dividing section 602 and important information forming section 603. The output of the important information forming unit 603 is connected to the packet header generating unit 604. Outputs of the bit string division unit 602 and the packet header generation unit are connected to the packet configuration unit 605. The output of the packet configuration unit 605 is
Connected to.

The encoding section 601 converts the input image signal 1
31 is encoded and output to the bit string division unit 602, and encoded information 634 at the time of encoding is output to the important information configuration unit 603.

Further, the important information forming unit 603 includes the encoding unit 6
This is a unit that receives the encoded information 634 obtained by encoding at 01, and selects and outputs only important information 635 necessary for decoding. In particular, in addition to important information related to normal images, MPEG-4
Information necessary for encoding / decoding images of arbitrary shapes in
For example, in image coding of an arbitrary shape, the width VW of the image size
, The information of the height VH, and the information of the x coordinate VHMSR of the image position for indicating the position to display the decoded image,
The VOP shape coding type “vo” indicating the information of the y coordinate VVMSR and the encoding mode of the shape information
Important information related to an arbitrary shape image such as "p_shape_coding_type (VSCT)" and change_conv_ratio_disable (CCRD) which is a flag indicating whether or not encoding is performed after converting the size of the shape information is acquired as important information 635. The information 635 is provided to the packet header generation unit 604. The normal image-related important information is normally reflected in the packet header, and the arbitrary shape image-related important information is determined in a newly provided extension header in the packet header. The packet header is generated in a format unique to the present invention, which is reflected in the format described in the present invention.

As described above, the packet header generator 604
Incorporates the important information 635 into the packet header and outputs the packet information to the packet configuration unit 605 as the packet header 636. The bit sequence division unit 602 divides the bit sequence 632 output from the encoding unit 601 into a packet size and outputs it. It is composed of

The packet configuration section 605 includes a divided bit string 633 output from the bit string division section 602 and a packet header 636 output from the packet header generation section 604.
And multiplexed data 637 is output to the transmission path / storage medium 106.

In such a configuration, the input image signal 131 of the moving image is encoded by the encoding section 601.
At this time, encoded information 634 at the time of encoding is output from the encoding unit 601 and input to the important information configuration unit 603. The important information forming unit 603 stores the input encoded information 6
Only important information 635 necessary for decryption is selected from 34 and output. The important information 635 is stored in the packet header generation unit 604.
In, the packet is embedded in the packet header and output as the packet header 636.

On the other hand, the bit string dividing section 602 divides the bit string 632 output from the encoding section 601 into a packet size and outputs it. Packet forming section 605 outputs divided bit string 63 output from bit string dividing section 602.
3 and the packet header 636 output from the packet header generation unit 604, and multiplex data 637
Output to the storage medium 106.

As described above, in the present embodiment, the important information 635 output from the important information forming unit 602 is inserted into the header by the packet header generating unit 604 in a determined format in the bit string obtained by encoding the moving image. Then, this is added to the encoded data of the moving image, packetized, and transmitted.

An extension header is provided in the packet header, and this extension header is a portion used for storing and transmitting important information other than the image-related important information.

Therefore, as important information other than the normal image-related important information, information necessary for encoding / decoding an image of an arbitrary shape in MPEG-4, for example, information of a width VW of an image size in image encoding of an arbitrary shape. And information on the height VH and information on the x-coordinate VHMSR, information on the y-coordinate VVMSR of the image position for indicating the position to display the decoded image,
Then, the VOP shape coding type “vop_shape_coding_type (V
SCT) "or change_conv_ratio_di which is a flag indicating whether or not encoding is performed after converting the size of the shape information.
sable (CCRD) etc. is obtained as important information 635,
This is inserted into the packet header as an extension header by the packet header generation unit 604 in a predetermined format. If the decoding unit is configured to perform decoding processing using information extracted from the extension header of this packet, an image of an arbitrary shape can be reproduced for each packet, and even in the case of encoding an arbitrary shape image, The same error resistance as that of the conventional rectangular image encoding can be provided, and even if the VOP header or a part of the VP is broken, the moving image can be decoded.

As described above, the present system can provide the same error resistance as the conventional rectangular image encoding even in the case of encoding an arbitrary-shaped image. A certain important information configuration unit 603 is shown in FIG.
This will be described in detail with reference to FIG.

FIG. 11 is a block diagram of the important information forming unit 603. The important information forming unit 603 is an important point in this embodiment. As shown in FIG. 11, the important information forming unit 603 includes a switching unit 2201, an extended header insertion determining unit 2202, and arbitrary shape image related important information. Configuration unit 220
And 3.

The extension header insertion determination section 2202 determines whether or not to add an extension header to a packet header.
From 34, it is determined whether or not the encoding unit 601 is performing image coding of an arbitrary shape. If the image coding of arbitrary shape is being performed, an extension header is added to the packet header.

The switching section 2201 is a section open / close switch. When the extension header insertion determination section 2202 determines that an extension header is to be added to a packet header, the switching section 2201 switches according to a control signal output from the extension header insertion determination section 2202. The unit 2201 is closed, and the encoded information 634 from the encoding unit 601 is used to
203.

Arbitrary Shape Image-Related Important Information Composition Unit 2203
Represents encoded information 634 input via the switching unit 2201
Is received as input encoded information 2233, and VOP header information related to arbitrary shape encoding is selected from the received encoded information and output as important information 635.

In such a configuration, encoding section 601
Information 63 input to important information forming section 603 from
In step 4, the extension header insertion determination unit 2202 determines whether to add the extension header to the packet header. If it is determined to add the input encoded information 223 from the switching unit 2201 to the arbitrary shape image related important information forming unit 2203.
3 is input. Arbitrary shape image-related important information forming unit 22
In step 03, VOP header information related to arbitrary shape encoding is selected from the input encoding information 2233, and important information 635 is output.

The following describes in detail how important information is incorporated into the packet header.

When encoding an arbitrary-shaped image, the width (VW), height (VH), X-coordinate (VHMSR), and Y-coordinate (VVMS) of the image are compared with those of the rectangular image.
R), a flag (CCRD) indicating whether or not the shape information is reduced and converted and encoded, and an encoding mode (VSCT) for the shape information are required. In addition, a flag (VCA) and a value (VCAV) for setting the alpha value at the time of alpha blending constant, and a flag indicating a rounding operation method for making the calculation accuracy the same in encoding / decoding. (VRT) and other information can be included,
Here, the former VW, VH, VHMSR, VVMSR,
CCRD and VSCT shall be incorporated. FIG. 12 shows the format of the packet header extension at that time. Here, the upper numeral represents the number of bits, and one horizontal row represents 32 bits. In the case of MPEG-4,
VW, VH, VHMSR and VVMSR are represented by 13 bits each, and the remaining CCED and VSCT are each 1 bit.

Here, as an example, in order to align to 32 bits, finally, the bit of the reserve “Reserve” (R
V) is inserted. If there is a possibility that the same bit string as the synchronization signal may appear due to the continuation of VW, VH, etc., for example, as shown in FIG.
Can be inserted between the values so as not to match a bit string that must never appear elsewhere, such as a synchronization signal. The position of the marker M does not need to be between the pieces of information, and may be embedded anywhere as long as the same rules are set on the transmitting side / receiving side.

Finally, it is necessary to embed a flag indicating that an extension header is present in ordinary header information. Therefore, information of whether or not the extension header exists in the normal header is embedded with 1-bit information. These formats are examples, and it is also possible to form header information only with a part of this data or to use it in combination with other information.

As described above, in the present embodiment, when a moving image is encoded and packetized, an extended header can be added to a packet header in which important information related to normal images is embedded, and an arbitrary-shaped image is encoded. If you send
The arbitrarily-shaped image-related important information is embedded in an extension header, added to data as a packet header, and packetized. Therefore, an image of an arbitrary shape can be reproduced for each packet, and even in the case of encoding an arbitrary shape image, the same error resistance as that of the conventional rectangular image encoding can be provided. Can be decoded even if the VP is broken.

Next, an example of a decoding unit for decoding such a packet will be described.

An example of the configuration of the decoding unit will be described below. According to the decoding unit shown in FIG. 14, the output of the separation unit 702 that receives the coded bit stream is connected to the decoding unit 703 and the important information configuration unit 705. Decoding unit 70
The output of No. 3 is connected to the important information forming unit 705 via the error checking unit 704. The output of the important information forming unit 705 is connected to the decoding unit 703.

The separating section 702 separates the bit string 731 input from the transmission path / storage medium 106 into a bit string 732 for an image, a packet header 735, and other data.

The decoding section 703 decodes the separated image bit string 732 using the important information from the important information forming section 705 to obtain the original image data. The error checking section 704 Is a unit for checking whether or not an error has occurred during decoding from the decoding information 733 obtained by the decoding unit 703.

The important information forming unit 705 includes the separating unit 7
The important information is configured from the information of the packet header 735 that has been subjected to the separation processing in step 02 and is supplied to the decoding unit 703.

In such a configuration, the bit string 731 input from the transmission path / storage medium 106 is
2, the image bit string 732 and the packet header 73
5, and separated into other data. Other data is sent to the corresponding decoding units. The separated image bit string 732 is input to the decoding unit 703, and is decoded. The decoding process in the decoding unit 703 is performed on the separated image bit string 732 using the important information from the important information forming unit 705.

The error checking section 704 is
It is checked from the decoding information 733 from 3 whether an error has occurred during decoding. As a result of the check, when it is determined that there is an error, the important information forming unit 705 forms the important information 736 existing in the packet header 735,
The decoding unit 703 starts decoding the coded bit sequence using the important information 736.

In this system, when a moving image is encoded and packetized, an extension header can be added to a packet header in which important information related to normal images is embedded. When an arbitrary-shaped image is encoded and transmitted, a packet header in which the arbitrary-shaped image-related important information is embedded in an extension header is added to the data. For this purpose, since the moving image is packetized, important information relating to the arbitrary shape image can be obtained from the extension header, and the arbitrary shape image can be decoded.

The important information constituting unit 705, which is an important element in the present embodiment, will be described in detail with reference to FIG.

As shown in FIG. 15, the important information forming unit 70
5 is a switching unit 2301 and an extension header insertion determination unit 230
2 and an arbitrary shape image related important information decoding unit 2303.

The extension header insertion determination section 2302 determines whether or not the extension header has been added to the packet header. The information bit string 732 for the image from the packet header 735 input from the separation section 702 has an arbitrary shape. Judge whether or not the image encoding is performed, determine that the extension header is added to the packet header when the image encoding of an arbitrary shape is performed, and output a control signal corresponding to the judgment. It is configured to be.

The switching section 2301 is a section open / close switch. When the extended header insertion determining section 2302 determines that the extended header is added to the packet header, the control output from the extended header insertion determining section 2302 is performed. The switching unit 2301 is closed by a signal, and the information of the packet header 735 from the separation unit 702 is given to the arbitrary-shaped image-related important information decoding unit 2303.

Arbitrary shape image related important information decoding section 2303
Is the packet header 7 input via the switching unit 2301
35 is received as input information 2333, and information related to arbitrary shape encoding is restored from this information.
6 is output.

The operation of the important information forming unit 705 having such a configuration will be described. In the input packet header 735, the extension header insertion determination unit 2302 determines whether or not an extension header is added to the packet header information by decoding the packet header information. As a result, when it is determined that there is an extension header, the extension header insertion determination unit 2302 controls the switching unit 2301 to close so as to pass through the packet header 735 and use the packet header information 2333 as important information relating to the arbitrary shape image. The data is input to the decoding unit 2303.

Arbitrary Shape Image-Related Important Information Decoding Unit 2303
Decodes important information relating to encoding of an arbitrary shape based on the packet header information 2333, and converts it into important information 736.
And outputs the result to the decoding unit 703.

In this way, it is possible to decode important information relating to an arbitrary shape image from information in an extension header provided so that important information relating to an arbitrary shape image can be embedded.

As described above, by using the technique of the second embodiment, even in the case of encoding an arbitrary-shaped image, similar to the first embodiment, it is possible to have the same error resistance as the encoding of the conventional rectangular image. It becomes possible. Further, by using the extension header of the protocol of the transmission path, the image encoding can be implemented without changing the bit sequence. This is effective when using an existing standard method or the like.

As in the modification of the first embodiment, VW, V
By using H, VHMSR, and VVMSR as variable-length coding, the code amount can be reduced.

Although this embodiment describes MPEG-4, it is possible to improve error resilience by adding the same information to the transmission of image coding of any shape other than MPEG-4. is there.

Next, as an application example of the present invention, an embodiment of a moving image transmission system to which the encoding device / decoding device of the present invention is applied will be described with reference to FIG.

A moving image signal input from a camera (not shown) provided in the personal computer 3001 is subjected to moving image encoding by an encoding device (or encoding software) incorporated in the personal computer 3001. The moving image signal output from the encoding device is wirelessly transmitted by the wireless device 3003 together with other audio and data information, and received by the other wireless device 3004. For example, a mobile phone, a PHS, a wireless LAN device, or the like may be used as the wireless device. The signal received by radio 3004 is
It is decomposed into moving picture signals and information on sound and data. Of these, the moving image signal is supplied to a decoding device (or decoding software) incorporated in the notebook computer 3005.
And displayed on the display of the notebook PC 3005.

On the other hand, a moving image signal input from a camera (not shown) provided in the notebook personal computer 3005 is obtained by using an encoding device (or encoding software) incorporated in the notebook personal computer 3005 in the same manner as described above. Decrypted. The generated moving image signal is multiplexed with other audio and data information, transmitted wirelessly by the wireless device 3004, and received by the wireless device 3003. A signal received by the wireless device 3003 is decomposed into a moving image signal and audio and data information. Of these, the moving image signal is decoded by a decoding device (or decoding software) incorporated in the personal computer 3001 and displayed on the display of the personal computer 3001.

Personal computer 3001 or notebook computer 3
The encoding / decoding device according to the present invention can be applied to moving image communication between the mobile phone 005 and the mobile videophone 3006. PC 3001 or notebook PC 3005
Generated by the encoding device incorporated in the
A moving image signal wirelessly transmitted from 003 or 3004 is received by a wireless device incorporated in portable videophone 3006. The signal received by the wireless device is decomposed into a moving image signal and audio and data information. Of these, the moving image signal is decoded by a decoding device (or decoding software) incorporated in the mobile videophone 3006 and displayed on the display of the mobile videophone 3006.

On the other hand, the moving image signal input from the camera 1007 incorporated in the portable videophone 3006 is transmitted to the personal computer 300 by using an encoding device (or encoding software) incorporated in the portable videophone 3006.
1 and the notebook computer 3005. The generated moving image signal is multiplexed with other audio and data information, transmitted wirelessly by a wireless device incorporated in the portable videophone 3006, and received by the wireless device 3003 or 3004. Radio 3003 or 3
The signal received at 004 is decomposed into a moving image signal and audio and data information. Of these, the moving image signal is decoded by a decoding device (or decoding software) incorporated in the personal computer 3001 or the notebook personal computer 3005, and is displayed on the display of the personal computer 3001 or the notebook personal computer 3005.

FIG. 26 shows an encoding apparatus according to the fourth embodiment corresponding to the encoding apparatus according to the first embodiment shown in FIG. According to the fourth embodiment, the multiplexed bit string output from the multiplexing unit 105 is stored in the recording medium 107. This recording medium 107 is formatted according to the present invention. That is, the recording medium 107 stores a shape information header and a plurality of VOPs following the shape information header. The shape information header is a field containing information commonly handled in the encoded data, and stores information higher in level than the VOP header. For example, the image size of a rectangular image is stored. Each VOP includes a plurality of macroblocks, and the first macroblock includes a VOP header and this VOP.
It consists of MB data provided after the OP header, and the subsequent macroblock is composed of a VP header and MB data following this VP header. The VP header is four-matched according to FIG.

FIG. 27 shows a decoding apparatus according to the fourth embodiment corresponding to the decoding apparatus according to the first embodiment shown in FIG. This decoding device reads and decodes the multiplexed bit string stored in the recording medium 107 by the encoding device of the fourth embodiment.

FIG. 28 shows a decoding apparatus according to the fifth embodiment corresponding to the encoding apparatus according to the second embodiment shown in FIG. This encoding device stores the multiplexed data output from the packet configuration unit 605 in the recording medium 107 according to the format of the present invention. That is, the format includes a shape information header and a plurality of VOPs, and each of the plurality of macroblocks of each VOP is provided with a VP header.

FIG. 29 shows a decoding apparatus according to the fifth embodiment corresponding to the decoding apparatus according to the second embodiment shown in FIG. This decoding device reads and decodes the multiplexed bit string stored in the recording medium 107 by the encoding device of the fifth embodiment.

In the following, the contents of the processing in the decoding device 303 of FIG. 27 will be described with reference to FIG.

An image code string is sequentially read from the storage medium 107, and a synchronization code is first detected (step S11). If the detected synchronization code is VOP start code
If it is e (YES in step S12), a process of outputting the VOP (frame) decoded immediately before to the image information output device is performed (step S13). Then, a VOP header (VOP header in FIG. 29) subsequent to the VOP start code in the image code string is decoded (step S14). If the VOP header can be correctly decoded (YES in step S15), the information recorded in the temporary storage circuit in the decoding device is replaced with the decoded VOP header information (time information, VOP prediction mode, etc.) (step S15). S16). Then, the macroblock data (MB data in FIG. 29) following the VOP header is decoded, and the video packet is decoded (step S1).
7).

If the detected synchronization code is resync
If it is a marker (YES in step S18), r
The video packet header (macro block number (MBA), video packet quantization parameter (SQ), header extension code HE following the sync marker
C)) is decrypted (step S19). If the header extension code HEC in the video packet header is "0" (NO in step S20), the video packet is decoded (step S17). If the header extension code HEC = “1” (step S
20 (YES), the subsequent duplication information (DUPH in FIG. 29) is decoded (step S21). At this time, it is determined whether or not the shape is arbitrary (step S21-1),
If YES, the arbitrarily-shaped image-related important information is decoded (step S21-2). If the determination is no, the process jumps to step 22. If the duplicated information has been correctly decoded (YES in step S22), the duplicated information is compared with the information stored in the temporary storage circuit (step S23). If the comparison results are equal (NO in step S23), the macroblock data (MBdata in FIG. 29) following the video packet header is decoded, and the video packet is decoded (step S1).
7). If the comparison results are not equal (step S2
3), it is determined that this video packet belongs to a VOP different from the VOP decoded immediately before, and the processing for outputting the VOP decoded immediately before to the image information output device is performed (step S24), and the temporary storage device Replace the recorded information with the decrypted duplicated information (step S2
5). Further, the video packet is decoded (step S17).

As described above, a series of processes starting from the detection of the synchronization code shown in FIG. 30 is repeated while sequentially reading the image code sequence recorded on the storage medium 810, and the moving image signal is reproduced.

It is to be noted that an image code string is not recorded on a storage medium as it is, but a code string obtained by multiplexing a code string obtained by encoding an audio signal or an audio signal with data, control information, and the like is recorded on the storage medium. You may do it. In this case, before the information recorded on the storage medium is encoded by the image encoding device 820, a process of demultiplexing the image code sequence and the audio / audio code sequence, data, and control information by the demultiplexing device is performed. The multiplexed image code sequence is decoded by the coding device 820.

FIG. 29 shows an example in which the information recorded on the storage medium 810 is transmitted to the decoding device 820 via the signal line 80. The information may be transmitted via the transmission path.

As described above, according to the present invention, important information is duplicated and recorded in the code string recorded on the storage medium. In the case where an error occurs in a signal line or a transmission path for sending information recorded in a storage medium to a reproduced image,
A reproduced image with little deterioration can be reproduced.

[0185]

As described above, according to the present invention, even when an image having an arbitrary shape is encoded, it is possible to provide an error resilience capability equivalent to that of encoding a conventional rectangular image. Further, even when data is transmitted in packets by using an RTP extension header as a protocol used for transmitting moving image / audio data, encoding is performed in accordance with an existing standard method such as MPEG-4. It can be transmitted, and can have the same error resilience capability as conventional rectangular image encoding.

[Brief description of the drawings]

FIG. 1 is a diagram showing a basic configuration of an encoding unit according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating a detailed basic configuration of an important information configuration unit and a bit string reconfiguration unit of an encoding unit according to the first embodiment of the present invention.

FIG. 3 is a view showing a basic flow of a bit string reconstructing unit according to the first embodiment of the present invention.

FIG. 4 is a diagram showing an extension header format of a VP according to the first embodiment of the present invention.

FIG. 5 is a diagram showing a basic configuration of a decoding unit according to the first embodiment of the present invention.

FIG. 6 is a diagram showing a detailed basic configuration of an important information forming unit in the decoding unit according to the first embodiment of the present invention.

FIG. 7 is a diagram for explaining a codeword configuration used in the present invention.

FIG. 8 is a diagram showing an example illustrating the effect of variable-length coding.

FIG. 9 is a view showing that variable-length coding is performed on important information.

FIG. 10 is a basic configuration diagram of an encoding unit according to a second embodiment of the present invention.

FIG. 11 is a detailed configuration diagram of an important information configuration unit of an encoding unit according to the second embodiment of the present invention.

FIG. 12 is a diagram showing an example of an extended packet header according to the second embodiment of the present invention.

FIG. 13 is a diagram showing an example of an extended packet header according to the second embodiment of the present invention (with a marker).

FIG. 14 is a diagram showing a basic configuration of a decoding unit according to the second embodiment of the present invention.

FIG. 15 is a diagram illustrating a detailed configuration of an important information configuration unit of a decoding unit according to the second embodiment of the present invention.

FIG. 16 is a view for explaining encoding of an arbitrary-shaped image.

FIG. 17 is a view for explaining decoding of an arbitrary-shaped image.

FIG. 18 is a diagram showing a VOP structure of MPEG-4.

FIG. 19 is a diagram showing a VP structure of MPEG-4.

FIG. 20 is a diagram showing a VP header format of MPEG-4.

FIG. 21 is a diagram showing a problem of a normal VP.

FIG. 22 is a view showing the effect of a normal VP.

FIG. 23 is a diagram showing the effect of VP when HEC is used.

FIG. 24 is a view showing necessary information when an image is synthesized and reproduced at the time of decoding an arbitrary-shaped image.

FIG. 25 is a diagram illustrating an example of a wireless moving image transmission system to which the encoding / decoding device according to the third embodiment of the present invention is applied.

FIG. 26 shows a fourth example corresponding to the encoding device of the first embodiment.
The figure which shows the encoding device of embodiment.

FIG. 27 shows a fourth example corresponding to the decoding device of the first embodiment.
The figure which shows the decoding apparatus of embodiment.

FIG. 28 shows a fifth example corresponding to the encoding device of the second embodiment.
The figure which shows the decoding apparatus of embodiment.

FIG. 29 is a fifth embodiment corresponding to the decoding device of the second embodiment;
The figure which shows the decoding apparatus of embodiment.

FIG. 30 is a view showing a flowchart of the decoding device of the third embodiment.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 101 ... Coding part 102 ... Important information formation part 103 ... Synchronization signal generation part 104 ... Bit string reconstruction part 105 ... Multiplexing part 106 ... Transmission line 201 ... MB boundary determination part 202 ... Counter 203 ... Synchronization signal insertion determination part 205 ... Header information insertion unit 206 ... Addition unit 206 ... Normal image related important information forming unit 207 ... Arbitrary shape image related important information forming unit 208 ... Arbitrary shape coding determination unit 210 ... Multiplexing unit 232 ... MB boundary determination signal 233 ... Code amount Data 241, important information relating to arbitrary shape image encoding 302, separation unit 303, decoding unit 304, synchronization detection unit 305, error check unit 306, important information configuration unit 307, normal image related important information configuration unit 308, arbitrary shape code Conversion determination unit 309, 311 switching unit 310 arbitrary image-related important information configuration unit 333 synchronization signal Detection signal 336,337,341,335 ... Header information including important information 338,339 ... Switching unit control signal 340,342 ... Important information 344,737 ... Decoded image signal 601 ... Encoding unit 602 ... Bit string division unit 603 ... Important Information configuration unit 604 Packet header generation unit 605 Packet configuration unit 734 Error detection signal 1001 Variable length coding unit 1002 Variable length code generation unit 2201 Switching unit 2202 Extended header insertion determination unit 2203 Arbitrary shape image related Important information component 3001 PC 3003 3004 Radio 3005 Laptop 3006 Mobile videophone

Claims (14)

[Claims] An encoding means for encoding an input moving image to obtain a bit string, and a set of predetermined bit strings is encoded based on encoding information by the encoding means under any rules. Important information reconstructing means for retrieving and reconstructing important information as information indicating whether the information is the same, synchronizing signal generating means for generating a synchronizing signal, and the synchronizing generating means for synchronizing the bit string encoded by the encoding means A moving image coding apparatus comprising: a bit string reconstructing means for reconstructing a bit string by adding a synchronization signal output from the above and important information reconstructed by the important information reconstructing means. 2. The image coding apparatus according to claim 1, wherein said important information reconstructing means is important information in a normal coding mode for coding a frame from the coded information in a rectangular area unit. A normal image-related important information reconstructing unit for reconstructing image-related information; and an arbitrary shape image-related important information in an arbitrary shape encoding mode for encoding an image in a frame in units of an arbitrary shape image from the encoded information. An arbitrary shape image-related important information reconstructing means for reconstructing important information; an arbitrary shape encoding judging means for judging whether or not an image encoded from the encoded information is an arbitrary shape image; and the arbitrary shape. A switching unit that outputs the arbitrary shape image-related important information when the encoding determination unit determines that the image is an arbitrary shape image; and multiplexes the normal image-related information and the output of the switching unit. Video encoding apparatus characterized by comprising a multiplexing means, the that. 3. A method of encoding a moving image to obtain a bit string including synchronization information, and as information indicating a rule of a set of certain bit strings in the encoding. As a decoding device in a moving image encoding that forms a bit string in a form in which important information is added as header information, a separating unit that separates an image bit string from an input bit string; a decoding unit that decodes the image bit string; A synchronizing signal detecting unit that detects a synchronizing signal from an image bit string and notifies the decoding unit of the synchronizing signal, an error checking unit that determines whether there is no error from decoding information of the decoding unit, and an error checking unit that there is no error. When it is determined, important information to be reconstructed from the header information output from the decoding means and to be notified to the decoding means Video decoding apparatus characterized by comprising a structure means. 4. The moving picture decoding apparatus according to claim 3, wherein the important information reconstructing means reconstructs normal image related important information from the header information; An arbitrary shape encoding determining unit that determines whether the image being decoded from the header information is an arbitrary shape image; an arbitrary shape image related important information reconstructing unit that reconstructs the arbitrary shape image related important information from the header information; Switching means for inputting header information to the arbitrary shape image related information reconstructing means, when determining that the image is an arbitrary shape image, A moving image encoding apparatus comprising: a switching unit that outputs the arbitrarily-shaped image-related important information. 5. An encoding means for encoding an input moving image to obtain a bit string, and a set of predetermined bit strings is encoded based on encoding information by the encoding means under any rules. Important information reconstructing means for retrieving and reconstructing important information as information indicating whether the information is a bit sequence, bit string dividing means for dividing the bit string encoded by the encoding means, Packet header creation means for creating a packet header from the configured important information; packet construction means for constituting a packet using the bit string divided by the bit string division means and the packet header generated by the packet header creation means; ,
A moving picture coding apparatus comprising: 6. The moving picture encoding apparatus according to claim 5, wherein said important information reconstructing means reconstructs arbitrary shape image related important information from said encoded information. Extended header insertion determining means for determining whether to add an extended header holding arbitrary shape image-related important information to the packet header from the encoded information, and when the extended header is inserted, the extended header insertion determining means Switching means for inputting the header information to the arbitrarily-shaped image-related important information when the determination is made, a moving image encoding apparatus comprising: 7. A moving picture is coded to obtain a bit string, and important information as information indicating under what rule a set of certain bit strings in this coding is coded. A decoding device for coded data in a moving image coding in which a bit string is added as packet header information in a moving image coding, a separating unit for separating an image bit string and packet header information from an input bit string, and a decoding for decoding the image bit string An error checking unit that determines whether or not there is an error from the decoding information of the decoding unit; and a rule in which a set of a certain bit string is encoded from the packet header information under any rules. Important information as information indicating whether the error check is performed, the important information is reconstructed, and the error check procedure is performed. Important information reconstructing means for notifying the decoding means of the reconstructed important information when important information is required for decoding when it is determined that there is an error in the stage. . 8. The moving picture decoding apparatus according to claim 7, wherein the important information reconstructing means reconstructs arbitrary shape picture related important information from the packet header information. An extension header insertion determining means for determining whether or not an extension header is added to the packet header from the packet header information; and the packet when the extension header insertion determining means determines that the extension header has been inserted. Switching means for inputting header information to the arbitrary-shaped image-related important information. 9. A server computer including a moving image encoding device that encodes a moving image to generate an encoded bit sequence, a transmitter that transmits the encoded bit sequence of the server computer, and an encoded bit sequence from the server computer And a client computer including a video decoding device that decodes a coded bit string from the receiver, wherein the video coding device encodes an input video. An encoding unit that generates encoded information as the bit sequence, and from the encoded information of the encoding unit, important information as information indicating under what rules a certain group of bit sequences is encoded. An important information reconstructing unit that extracts and reconstructs information; a synchronizing signal generating unit that generates a synchronizing signal; and bits encoded by the encoding unit. A bit sequence reconstructing unit that reconstructs a bit sequence by adding a synchronization signal output from the synchronization generating unit and important information reconstructed by the important information reconstructing unit, and the video decoding device includes: A separating unit that separates an image bit sequence corresponding to the moving image information from the received encoded bit sequence, a decoding unit that decodes the image bit sequence, and a synchronization that detects a synchronization signal from the bit sequence and notifies the decoding unit. A signal detection unit, an error check unit that determines whether there is no error from the decoding information of the decoding unit, and the header information output from the decoding unit when the error check unit determines that there is no error. An important information reconstructing unit that reconstructs important information from and notifies the decoding unit of the important information. 10. A server computer including a moving image encoding device that encodes a moving image to generate an encoded bit sequence, a transmitter that transmits the encoded bit sequence of the server computer, and an encoded bit sequence from the server computer And a client computer including a video decoding device that decodes a coded bit string from the receiver, wherein the video coding device encodes an input video. An encoding unit that generates the encoded information as an encoded bit sequence, and important information indicating under what rules a group of certain bit sequences is encoded from the encoded information in the encoding unit. An important information reconstructing unit that extracts and reconstructs, a bit that divides the bit string encoded by the encoding unit and generates a plurality of divided bit strings A column dividing unit, a packet header creating unit that creates a packet header from the important information reconstructed by the important information reconstructing unit, a packet constituting unit that constitutes a packet using the divided bit string and the packet header, The video decoding device, comprising: a separating unit configured to separate an image pit sequence and packet header information of the video information from the received bit sequence; a decoding unit configured to decode the image bit sequence; An error checking unit that determines whether there is no error from the decoding information of the decoding unit; and extracting the important information from the packet header information to reconstruct the important information. The error checking unit determines that there is an error. And an important information reconstructing unit that notifies the decoding unit of the reconstructed important information when important information is needed for decoding. Image transmission system. 11. A shape information header for storing arbitrary shape information of an image to be recorded, each of which includes a plurality of macroblocks, each macroblock being a VP header and M provided after the VP header.
B data, and the VP header includes a plurality of VOPs (Video Object Planes) including important information indicating under what rule a fixed group of encoded bit strings is encoded. , A recording medium on which is recorded. 12. The important information of the VP header is an image width,
12. The recording medium according to claim 11, including a height of the image, an X coordinate and a Y coordinate for pasting the image, a flag indicating whether or not the shape information is reduced and converted, and information on an encoding type of the shape information. 13. Separation means for separating an image sequence from an input bit sequence, decoding means for decoding the image bit sequence, and a synchronization signal for detecting a synchronization signal from the image bit sequence and notifying the decoding means. A moving picture decoding apparatus comprising: a detecting unit; and important information reconstructing unit that reconstructs important information from the header information output from the decoding unit and notifies the decoding unit of the important information. 14. An important information reconstructing means, comprising: a normal image related important information reconstructing means from the header information; and an arbitrary shape encoding determination for judging whether an image decoded from the header information is an arbitrary shape image. Means, arbitrary shape image-related important information reconstructing means for reconstructing the arbitrary shape image-related important information from the header information, and when the arbitrary shape coding determining means determines that the header information is an arbitrary shape, the header information is converted to the arbitrary shape image. 14. Switching means for inputting to the related information reconstructing means, and switching means for outputting the arbitrary shape image related important information when the arbitrary shape encoding determining means determines that the image is an arbitrary shape image. Video encoding device.