JPH10243393A - Image decoding method, image decoding device, image multiplexing method, image multiplexing device, and recording medium - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To efficiently operate a work area in a decoding process of a composite image and to separate a multiplexed signal and decode an image by utilizing a free area even when the capacity of the work area is insufficient. Do. A demultiplexing unit receives a multiplexed signal obtained by multiplexing coded signals of a plurality of images and separates the multiplexed signals into coded signals of each image. The work area amount detecting means 108 detects the work area amount for the encoded signal 103 and notifies the CPU 109 of the work area amount. Work area storage means 106
Stores a reference image and the like necessary for decoding in a work area. The work area information storage unit 110 stores information on the work area, and stores a work area necessary for another image processing as a CP.
Notify U109. The CPU 109 manages the work area and controls the operation of the decoding means 104 to efficiently operate the work area storage means 106 for decoding the composite image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method of recording and transmitting an image signal in order to reduce a data amount of an image, a sound, and a digital data in order to efficiently use a recording capacity and a transmission path capacity. The present invention relates to an image multiplexing method and an image multiplexing apparatus for multiplexing an encoded signal, an image decoding method and an image decoding apparatus for correctly decoding the multiplexed signal, and a recording medium for recording the multiplexed signal.

[0002]

2. Description of the Related Art An image compression technique is indispensable for efficiently recording and transmitting image data having an enormous data amount. Therefore, JPEG, MPEG, and the like have been developed as techniques for efficiently encoding images. MPEG, J
The PEG method targets a single image or a single moving image sequence. On the other hand, an image includes a composite image generated from a plurality of images.

FIG. 11 is an explanatory view showing the concept of a composite image. The background image 1101 is an image showing the background, and the images 1102 and 1104 are images of the objects that make up the composite image. First, the image 1102 of the spherical object A is synthesized with the background image 1101, and the synthesized image 1
106 is obtained. At this time, the transmittance signal 1103 is used to indicate the ratio of the combination of the pixel values of the background and the object A. The transmittance signal 1103 is a signal indicating the rate of synthesis of the pixel values of the background image 1101 and the image 1102 of the object.

Further, an image 1104 of a columnar object B and a transparency signal 1105 are synthesized on the synthesized image 1106 in a similar manner, and a final synthesized image 1107 is generated. In a composite image of a moving image, the background and the object can each be regarded as a moving image sequence. By combining images at a certain point in each moving image sequence, a moving image of a combined image can be generated.

FIG. 12 is a block diagram showing an example of the configuration of a conventional image multiplexing apparatus for encoding and multiplexing an image signal of a composite image and an image decoding apparatus for decoding a multiplexed signal of the composite image. An image multiplexing device 1201 is shown in the upper part, and an image decoding device 1202 is shown in the lower part. Image multiplexing device 1
The multiplexed signal 201 is transmitted to the image decoding device 1202 via the transmission path 1203.

[0006] The image multiplexing device 1201 has a plurality of encoding means 904, 905, 906 and a multiplexing means 910. When image signals 901, 902, and 903 of each object are input, encoding means 904, 905, and 906 encode the respective image signals, and encode signals 907, 908,
909 is generated and given to the multiplexing means 910. The multiplexing unit 910 multiplexes the coded signals 907, 908, and 909 to generate a multiplexed signal 911, and outputs the multiplexed signal 911 to the transmission path 1203.

[0007] The image decoding apparatus 1202 comprises a demultiplexing means 102, a decoding means 104, a work area storage means 106 and a synthesizing means 1207. Demultiplexing means 102
Receives the multiplexed signal 911 transmitted from the transmission line 1203, and encodes the encoded signals 1204, 1205, and 120 of each image.
6 is a means for separating and outputting. The decoding unit 104 is a unit that decodes each of the encoded signals 1204 to 1206 and converts it into an image signal of each image. The work area storage means 106 stores information necessary for decoding by the decoding means 104.
The reference image or the like is stored with priority. The synthesizing unit 1207 synthesizes the image of each frame using each image signal decoded by the decoding unit 104 and the reference image of the work area storage unit 106,
This is a means for outputting as a composite image 1208.

In the case of a moving image, the image multiplexing device 120
1 inputs a moving image sequence for each object and performs encoding and multiplexing. The image decoding device 1202 separates the multiplexed signal into encoded signals of a moving image sequence for each object, decodes each as a moving image, and combines them. The image multiplexing device 1
Reference numeral 201 simultaneously multiplexes and transmits information required at the time of combining, such as the order of combining images of the objects, the size of the image, the combining position, and the like.

[0009] The image decoding apparatus 1202
3 is received, and the demultiplexing unit 102 receives the multiplexed signal 911 of each image.
6 and output. The decoding means 104 decodes each of the encoded signals 1204 to 1206 and converts them into an image signal of each image. A decoded image (image signal) output by the decoding unit 104
Are stored in the work area storage means 106 as needed.

[0010] The work area storage means 106 is a decoding means 104
Has the function of holding the information required at the time of decoding, and in this example, holds at least the reference image necessary for predictive coding. The storage area in the work area storage means 106 must be allocated for each decoding processing of each encoded signal in the multiplexed signal. The synthesizing unit 1207 is a decoding unit 104
Synthesizes the decoded images and outputs a synthesized image 1208.

[0011]

However, in the decoding processing of a composite image, efficient management of the work area of the image decoding apparatus and consideration of a case where the capacity of the work area is not enough has not been taken until now. There is a problem that management of the work area storage means becomes difficult.

SUMMARY OF THE INVENTION The present invention has been made in view of such conventional problems, and has been made in consideration of the above-described problems. It is an object to realize a method, and an image multiplexing device and an image decoding device that embody these methods.
It is another object of the present invention to provide a recording medium on which these multiplexed signals are recorded.

[0013]

In order to solve such a problem, the invention according to claim 1 of the present application inputs a multiplexed signal obtained by multiplexing data including a plurality of encoded image signals, An image decoding method for decoding multiplexed data, wherein the multiplexed signal is separated into the respective encoded signals,
Detecting the capacity of the work area of the work area storage means required for the decoding processing of each of the encoded signals as a work area amount, assigning the detected work area amount to the work area storage means, The encoded signal is decoded using the assigned work area.

According to the invention described in claim 2 of the present application,
The position of the area in the work area storage means allocated to the decoding processing of each of the encoded signals is stored as work area information.

According to the invention of claim 3 of the present application,
A position of a free area in the work area storage means which is not assigned to the decoding processing of each of the encoded signals is stored as free area information.

According to the invention described in claim 4 of the present application,
Work area information, which is the position information of the area in the work area storage means allocated to the decoding processing of each of the encoded signals, and free space, which is the position information of the free area in the work area storage means not allocated to the decoding processing It is characterized by storing area information.

According to the invention of claim 5 of the present application,
An image decoding apparatus for receiving a multiplexed signal obtained by multiplexing data including a plurality of encoded image signals and decoding the multiplexed data, wherein the apparatus separates the multiplexed signal into the respective encoded signals. Demultiplexing means, a work area storage means for holding information used for decoding an encoded signal, and a work area capacity required for decoding each encoded signal output from the demultiplexing means. A work area amount detecting means for detecting,
Control means for allocating the work area amount detected by the work area amount detection means to the work area storage means; and the code separated by the demultiplexing means using the work area allocated to the work area storage means. Decoding means for decoding the coded signal.

According to the invention of claim 6 of the present application,
An image decoding method for inputting a multiplexed signal obtained by multiplexing data including a plurality of coded image signals and decoding the multiplexed data, wherein the multiplexed signal is separated into the respective coded signals. Then, the capacity of the work area required for the decoding process of each of the encoded signals is detected as a work area amount, and in the decoding process of each of the encoded signals, the priority of the assignment of the work area of the work area storage unit is determined. Detecting and assigning the detected work area amount to the work area storage means in order from the decoding processing with the higher priority, and decoding the encoded signal using the work area assigned to the work area storage means. It is a feature.

According to the invention described in claim 7 of the present application,
A work area of the work area storage means is not assigned to the decoding processing of the coded signal having the priority lower than a certain level.

According to the invention described in claim 8 of the present application,
When requesting allocation of a predetermined amount of work area for the decoding process, if there is no free work area in the work area storage means, a work area assigned to a decoding process with a lower priority than the decoding process. Is released, and the released work area is allocated to the decoding process.

According to the invention of claim 9 of the present application,
An image decoding apparatus for inputting a multiplexed signal obtained by multiplexing data including a plurality of images and decoding the multiplexed data, wherein the multiplexing includes multiplexing respective coded signals of data including a plurality of images. Demultiplexing means for inputting a signal and separating it into each of the coded signals; work area storage means for holding information used for decoding the coded signal; and each coded signal output from the demultiplexing means A work area amount detecting means for detecting a capacity of a work area necessary for decoding processing as a work area amount, and detecting a priority of assignment of a work area in a work area storage means at the time of decoding each of the encoded signals. Priority detection means, control means for allocating the work area amount detected by the work area amount detection means to the work area storage means according to the priority detected by the priority detection means, and the demultiplexing means Separated by It is characterized in that it comprises a decoding means for decoding using the work area allocated to the working area storage means the coded signals.

According to the invention of claim 10 of the present application, a work area information storage means for storing position information of a work area in the work area storage means as work area information, and a free space in the work area storage means. And free space information storage means for storing the position information of the area as free space information.

According to the eleventh aspect of the present invention, there is provided an image decoding method for inputting a multiplexed signal obtained by multiplexing data including a plurality of encoded image signals, and decoding the multiplexed data. In separating the multiplexed signal into the respective coded signals, detecting the buffer capacity of buffer means necessary for temporarily holding the respective coded signals, when decoding the respective coded signals, A buffer area allocation priority of the buffer means is detected, the detected buffer capacity is allocated to the buffer means in order from a coded signal having a high priority, and each coded signal held in the buffer means is decoded. It is characterized by the following.

According to the twelfth aspect of the present invention, there is provided an image decoding method for inputting a multiplexed signal obtained by multiplexing data including a plurality of encoded image signals and decoding the multiplexed data. In separating the multiplexed signal into the respective coded signals, detecting the buffer capacity of buffer means necessary for temporarily holding the respective coded signals, when decoding the respective coded signals, The priority of the allocation of the buffer area of the buffer means is detected, and the detected buffer capacity is allocated to the buffer means in order from the encoded signal having the higher priority, and the buffer area is requested to be allocated for holding the encoded signal. At this time, if there is no free space in the buffer means, the buffer use area allocated for the low-priority coded signal is released, and the buffer area is required. For the retention of the coded signal, assign the freed the buffer region, is characterized in that for decoding the coded signal stored in the buffer means.

According to the thirteenth aspect of the present invention, there is provided an image decoding apparatus which receives a multiplexed signal obtained by multiplexing data including a plurality of encoded image signals and decodes the multiplexed data. A demultiplexing unit that separates the multiplexed signal into the coded signals, a buffer unit that temporarily holds the coded signals, and a buffer for temporarily storing the coded signals output from the demultiplexing unit. Buffer capacity detection means for detecting a buffer capacity required for holding; priority detection means for detecting the priority of allocation of a buffer area of the buffer means in decoding each of the encoded signals; The buffer capacity detected by the means,
Control means for allocating to the buffer means in accordance with the priority detected by the priority detection means, and decoding means for decoding the coded signal held by the buffer means. is there.

[0026] According to the invention of claim 14 of the present application, an image multiplexing method for generating a coded signal obtained by coding data including a plurality of images and outputting a multiplexed signal obtained by multiplexing the coded signals. The encoding method is characterized in that the priority of assignment in a work area storage means used as a work area at the time of decoding is multiplexed on the coded signal and output by the image decoding apparatus.

According to the fifteenth aspect of the present invention, there is provided an image multiplexing apparatus for generating a coded signal obtained by coding data including a plurality of images, and outputting a multiplexed signal obtained by multiplexing the coded signals. A plurality of encoding means for encoding data including a plurality of images to generate an encoded signal, and an assignment in a work area storage means used as a work area at the time of decoding on the image decoding apparatus side. Priority determining means for determining the priority, and multiplexing means for multiplexing and outputting the encoded signal output from each of the encoding means and the priority signal output from the priority determining means. It is a feature.

[0028] According to the invention of claim 16 of the present application, an image multiplexing method for generating a coded signal obtained by coding data including a plurality of images and outputting a multiplexed signal obtained by multiplexing the coded signals. A plurality of encoding means for encoding data including a plurality of images to generate an encoded signal, and a buffer means for holding an encoded signal used at the time of decoding. Priority determining means for determining the degree, and multiplexing means for multiplexing and outputting a coded signal output from each of the coding means and a priority signal output from the priority determining means. It is assumed that.

According to the seventeenth aspect of the present invention, a multiplexed signal obtained by generating a coded signal obtained by coding a plurality of images and other data and multiplexing the coded signals, The present invention is characterized in that priorities of assignment to work area storage means for holding coded signals to be used are accumulated.

According to the invention of claim 18 of the present application, a multiplexed signal obtained by generating a coded signal obtained by coding a plurality of images and other data and multiplexing the coded signals, The present invention is characterized in that priorities of allocation to buffer means for holding coded signals to be used are accumulated.

[0031]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. In each of the embodiments, multiplexing of a coded signal of an image will be described. However, not only an image but also digital data including voice and text data is handled in the same manner.

(Embodiment 1) An image decoding method and an image decoding apparatus according to Embodiment 1 of the present invention are shown in FIG.
This will be described with reference to FIGS. FIG. 1 is a block diagram showing a basic configuration of an image decoding apparatus according to the present embodiment, and the same parts as those of the conventional example are denoted by the same reference numerals. In FIG. 1, the image decoding apparatus includes a demultiplexing unit 102, a decoding unit 104, a work area storage unit 106, a work area amount detection unit 108, a CPU 109, a work area information storage unit 110,
It is configured to include a free area information storage unit 111.

The demultiplexing means 102 is a means for inputting the multiplexed signal 101, separating the multiplexed signal into coded signals of respective images, and sequentially switching them to output a coded signal 103. The decoding unit 104 is a unit that decodes the sequentially switched encoded signals and outputs a decoded image signal 105.

The work area storage means 106 is a means for storing information necessary for decoding. The work area is
For example, it refers to a memory area for storing a reference image for predictive encoding, or a working memory area necessary for decoding such as an output buffer for a decoded image. This work area is assigned to each decoding process of the coded signal of each image in the multiplexed signal.

The work area amount detecting means 108 outputs the encoded signal 1
03 is a means for detecting the required capacity of the work area by inputting 03. The CPU 109 is a control unit that manages the work area storage unit 106 and controls the decoding unit 104. The work area information storage means 110 is a means for storing information on the work area in the work area storage means 106. The free area information storage means 111 is a means for storing information on a free area in the work area storage means 106.

The operation of the image decoding apparatus thus configured will be described. The demultiplexing unit 102 separates the multiplexed signal 101 input to the image decoding device into encoded signals of each image. The work area amount detection means 108 detects a work area amount required at the time of decoding from each of the currently input coded signals 103 and notifies the CPU 109 of the work area amount.

The work area storage means 106 of the present embodiment preferentially holds a reference image for predictive coding. As a method of detecting the work area amount in the work area amount detection means 108,
For example, the image multiplexing device side multiplexes the information of the work area in advance and transmits the information, and the image decoding device side extracts the information, or the image decoding device stores the reference image from the decoded image size information and the like. For example, there is a method of calculating a work area amount required for use.

FIG. 2 is an explanatory diagram showing the concept of work area management information. The map 202 shown in the center shows the internal state of the work area storage means 106 in FIG. The information shown on the left side indicates the contents relating to the work area stored in the work area information storage means 110. The information 203 shown on the right side indicates the contents related to the free space stored in the free space information storage unit 111. In the map 202, the work area 1 and the work area 2 are allocated for the work of the decoding process. Information 20
The first work area information is information on which place in the work area storage means 106 is used for each work area, for example, the start position and size of the work area. Information 203
The free area information is information on which location in the work area storage means 106 is a free area, for example, the start position and size of the free area.

FIG. 3 is a flowchart showing the work area allocation process of the CPU 109. First step S3
In step 1, the work area amount required for decoding each encoded signal is input from the work area amount detection means 108. In the next step S32, a free area having a size equal to or larger than the requested work area amount is searched from the free area information storage unit 111. In step S33, it is checked whether there is a free area having a size equal to or larger than the requested work area amount. If there is a free area, the process proceeds to step S34, and the requested work area among the detected free areas is allocated as a free area.

In the next step S35, information on the assigned work area is stored in the work area information storage means 110.
The process further proceeds to step S36, and the free area information storage unit 1
From 11, the item of the allocated area is deleted or the information is changed to the information of the empty portion excluding the allocated portion. In addition, the order of the processes of step S35 and step S36 may be interchanged.

In step S33, if the area of the requested work area amount is not found in the free area storage means 111, the process moves to step S37, and the work area is rearranged.

The rearrangement of the work area in step S37 will be described in detail with reference to FIG. Map 5 shown on the left
01 indicates the internal state of the work area storage unit 106 before the rearrangement, and the map 502 on the right indicates the internal state of the work area storage unit 106 after the rearrangement. As the work area is allocated, the free area is subdivided, and a free area of a sufficient size may not be obtained. In that case, each work area is rearranged so as to be continuous in the work area storage means 106,
By combining the empty areas into one, a large empty area can be obtained. For example, as shown by the map 502, by rearranging the work area 2 so as to be continuous with the work area 1, the empty area 1 and the empty area 2 can be combined into one. At this time, the work area information and the free area information are also updated.

By repeating this operation for each work area, a large empty area can be obtained. It should be noted that the repetition may be performed until a necessary free area is obtained,
The process may be performed until all the work areas are rearranged. As a result of relocation, if free space that satisfies the request can be obtained,
The work area is allocated in step S34 in FIG. If it is not possible to obtain a free area satisfying the request in step S38, the process proceeds to step S39, and it is determined that allocation is impossible. If it is determined in step S33 that the work area cannot be allocated without performing such rearrangement of the work area, the work area may be determined to be unallocable, and the process may be terminated.

Next, the CPU shown in FIG.
The operation of the work area release processing at 109 will be described. Unnecessary work areas must be released for use in other decoding processes. In the first step S41 in FIG. 4, an identifier that can uniquely identify an item of unnecessary work area information, for example, the start position of the work area is stored in the CPU 109.
Give to. In the next step S42, a work area item to be released is searched from the work area information storage unit 110. In step S43, the information on the work area to be released is deleted, and the flow advances to step S44 to add the information on the released work area to the free area information. With the above-described release processing, a work area that has become unnecessary can be released.

The decoding means 104 shown in FIG. 1 decodes the coded signal 103 and outputs a decoded image signal (also referred to as a decoded image) 1.
05 is output. Then, combining means (not shown) combines and outputs the decoded images of the respective images. The decoding means 104 decodes a specific decoded image 105 to perform predictive coding.
Is stored in the assigned work area in the work area storage means 106 as a reference image. When the reference image becomes unnecessary, the CPU 109 requests the release of the work area, and releases the work area.

As described above, in the present embodiment, the work area storage means for decoding a composite image can be efficiently operated.

If it is not possible to obtain a work area satisfying the request, a method of waiting until a free area is created,
It is also possible to use a method in which the securing of the work area is abandoned and the decoding process of the image is not performed.

It is to be noted that the request for the assignment of the work area can be made before the decoding of the image sequence starts or during the decoding of each frame of the image sequence. Similarly, the work area release request can be made during the decoding of each frame of the image sequence.

Although one work area is assigned to the decoding process of each encoded signal, it may be divided into a plurality of work areas and assigned.

In this embodiment, the work area information and the free area information are stored. However, without storing the free area information, it is possible to search for an unused area from the work area information to obtain a free area. Good.

In the present embodiment, the work area storage means has been described as the storage means for the reference image. However, other work information necessary for decoding, an output buffer for storing a decoded image for the synthesis processing, etc. This method can also be used to manage the data.

Although the present embodiment has been described with reference to a case where there is one work area storage means, there may be a plurality of work area storage means.

Although the present embodiment has been described with reference to a case where there is one decoding means, a plurality of decoding means may be provided.

In the present embodiment, the case of multiplexing the image coded signal has been described. However, not only the image coded signal but also other data such as a coded signal of audio and digital data such as text information. May be multiplexed.

(Embodiment 2) An image decoding apparatus according to Embodiment 2 of the present invention will be described. FIG. 6 is a block diagram showing a basic configuration of the image decoding apparatus according to the present embodiment. The same parts as those in FIG. This image decoding apparatus includes a demultiplexer 10
2. A priority detecting means 601 is provided in addition to the decoding means 104, the working area storing means 106, the working area amount detecting means 108, the CPU 109, the working area information storing means 110, and the free area information storing means 111. It is. The priority detection means 601 is a means for detecting the priority of assignment of a work area with reference to the encoded signal 103 output from the demultiplexing means 102, and the detection result is determined by the CPU.
109.

The operation of the image decoding apparatus having such a configuration will be described. Note that the means and signals having the same reference numerals as those in FIG. 1 perform the same operations as in the first embodiment, and therefore, the description of the operations will be simplified. The demultiplexing means 102 separates the multiplexed signal 101 into a coded signal 103 of each image. The work area amount detection means 108 detects the work area amount required at the time of decoding from each encoded signal 103 and notifies the CPU 109 of the work area amount.

The priority detection means 601 detects each encoded signal 10
3 or using the transmitted priority signal to detect the priority of the work area allocation. The priority can be determined by the image multiplexing device side, multiplexed in a multiplexed signal or transmitted using another communication path, and the image decoding device side can extract the priority. In addition, the image decoding device can independently determine the priority.

When the capacity of the work area storage means 106 is insufficient, the work area storage means 10 is used, for example, by a method described below using the priority information of the work area allocation.
6 are to be managed.

First, the first method will be described with reference to the flowchart of FIG. Since the work area of the work area storage unit 106 of the image decoding apparatus is limited, it is more likely that a request for the work area is issued earlier than a request issued later. Therefore, if a request for a work area is issued in order from the decoding processing of the coded signal of the image with the highest priority, the work area can be allocated according to the priority.

In FIG. 7, the priority order is represented by a priority order n. However, n starts from 1, and the smaller the value of n, the higher the priority. First, in step S71, the value of n is initialized to 1. Next step S72
Then, it is determined whether or not a work area can be assigned in response to a request for a work area of an image having the n-th priority. If the assignment is possible, the process proceeds to step S73. If the assignment is not possible, the process proceeds to step S74 without assigning a work area.

In step S73, a required work area is allocated. Note that the method described in Embodiment 1 can be used to determine the possibility of assignment and to execute assignment. In the next step S74, it is determined whether the process of determining the assignment of the coded signals of all the images has been completed. If not completed, the process proceeds to step S75, where the priority value n is incremented, the process returns to step S72, and the same processing is continued. If it is determined in step S74 that the process is completed, the process ends.

This method is particularly useful in allocating working areas for several images at the same time. Note that an image for which the work area could not be allocated may be requested again later.

Next, the second method will be described below with reference to the flowchart of FIG. First step S
At 81, it is determined whether or not the requested work area can be allocated. If the work area can be allocated, the process moves to step S88, and the requested work area is allocated. If it is determined in step S81 that the work area cannot be allocated, the flow advances to step S82 to calculate the total amount of the work area used in the decoding processing of the image having a lower priority than the decoding processing of the image requesting the work area. If the total value is not equal to or more than the requested work area amount, the process moves to step S84, and it is determined that allocation is impossible. If it is determined in step S83 that the total value is equal to or greater than the requested work area amount, it is determined that work areas can be allocated, and the flow advances to next step S85.

In step S85, the work area for the decoding processing of the image having a lower priority than the decoding processing of the image for which the work area is requested is released until a free area corresponding to the requested work area amount is secured. After the release, it is determined whether or not the work area needs to be relocated as described in the first embodiment (step 8).
6) If necessary, the process proceeds to step S87 to relocate the work area. If it is determined in step S86 that the rearrangement is not necessary, and if the rearrangement is completed in step S87, the process proceeds to step S88. In this step S88, the requested work area is allocated. In step S82, the total value of the work areas with low priorities is calculated. However, the total value is not calculated and compared with the area amount of each work area to determine whether the requested work area can be secured. May be determined.

The work areas can be allocated according to the priorities by the above two methods. Note that the first method and the second method can be used in combination.

The decoding means 104 of FIG. 6 decodes the coded signal 103 of each image and outputs it as a decoded signal (decoded image) 105. Then, combining means (not shown) combines and displays the decoded images. The reference image of the decoded image 105 is stored in the assigned work area in the work area storage unit 106. If this reference image is no longer needed, C
The PU 109 requests release of the work area, and releases the work area.

As described above, in this embodiment, the work area storage means for decoding the composite image can be efficiently operated according to the priority.

If a work area satisfying the request cannot be obtained, a method of waiting until a free area is formed,
It is also possible to use a method in which the securing of the work area is abandoned and the decoding process of the image is not performed.

The request for assigning a work area can be made before the decoding of the image sequence starts or during the decoding of each frame of the image sequence. Similarly, the work area release request can be made during the decoding of each frame of the image sequence.

In the present embodiment, one work area is assigned to the decoding process of each encoded signal, but it may be divided into a plurality of work areas and assigned.

When releasing the decoding processing of an image with a low priority, in the case of bidirectional prediction encoding such as MPEG,
Only one of the two reference images can be released.

In the present embodiment, the assignment of the work area is performed by the method shown in the first embodiment, but the function of allocating and releasing the work area by another method may be used. For example, for a decoding process of an encoded signal having a certain priority or less, a work area may not be assigned and decoding may not be performed.

In this embodiment, both the work area information and the free area information are stored. However, the free area information is not stored, and the unused area is searched for from the work area information to find the free area. Is also good.

In the present embodiment, the work area storage means is used as the storage means for the reference image. However, in order to perform other work areas required for decoding and to perform synthesis processing, the decoded image is stored. This method can also be applied to an output buffer or the like for storing.

In this embodiment, the case where there is one work area storage means has been described, but there may be a plurality of work area storage means.

Although the present embodiment has been described with reference to a case where there is one decoding unit, a plurality of decoding units may be provided.

In the present embodiment, the case of multiplexing the coded image signal has been described, but not only the coded image signal but also other data including digital data such as a coded audio signal and text information. May be multiplexed.

(Embodiment 3) Next, an image encoding method and an image encoding apparatus according to Embodiment 3 of the present invention will be described with reference to the drawings. FIG. 9 is a block diagram illustrating a basic configuration of the image encoding device according to the present embodiment. The image coding apparatus according to the present embodiment is characterized in that the priority of work area allocation is set and output to the image decoding apparatus. This image encoding device includes encoding means 9
04, 905, 906, priority order determining means 912, and multiplexing means 910.

The image signals 901 to 903 are image signals of each object and background in the composite image. Encoding means 904-9
Reference numeral 06 denotes a unit that encodes the image signals 901 to 903 of each object and outputs encoded signals 907 to 909. The priority determining unit 912 is a unit that receives the encoded signals 907 to 909 and determines the priority of decoding. The multiplexing means 910 multiplexes each of the encoded signals 907 to 909 and the priority signal output from the priority determining means 912, and outputs a multiplexed signal 911. This multiplexed signal 9
11 is recorded on a recording medium 913 or transmitted on a transmission medium 914.
Is transmitted to the image decoding apparatus via the.

The operation of the thus configured image multiplexing apparatus will be described. When the image signals 901 to 903 of the respective objects in the composite image are input, the encoding units 904 to 906 encode these image signals and encode the encoded signals 907 to 909.
Is output. The priority determining means 912 outputs the encoded signal 907
909, and outputs a priority signal to the multiplexing means 910. The multiplexing unit 910 multiplexes the priority signal output from the priority determining unit 912 with the coded signals 907 to 909 of each image, and outputs a multiplexed signal 911.

As described above, in the present embodiment, the image decoding device on the receiving side adds the priority of work area allocation for use in managing the allocation of work areas to the multiplexed signal on the transmitting side. Therefore, the load of signal processing of the image decoding device is reduced.

Although a plurality of encoding means are provided in this embodiment, the encoding may be performed by one encoding means by switching the input of the encoding means.

In the present embodiment, the case of multiplexing the coded image signal is shown. However, not only the image but also other data including digital data such as coded audio signal and text information is multiplexed. It may be.

(Embodiment 4) An image decoding apparatus according to Embodiment 4 of the present invention will be described with reference to FIG. FIG.
0 is a block diagram showing a basic configuration of an image decoding apparatus according to the present embodiment. The same parts as those in the second embodiment are denoted by the same reference numerals, and detailed description is omitted. This image decoding apparatus is characterized in that it has a buffer for holding an encoded signal required for decoding until decoding. The method described so far can be used for managing the buffer.

As shown in FIG. 10, the image decoding apparatus according to the present embodiment includes a work area amount detecting means 108, a work area information storage means 110, and a free area information storage means 11 shown in FIG.
1, a buffer capacity detecting means 1001, a buffer used area information storing means 1003, and a buffer free area information storing means 1004 are provided. The coded signal 103 output from the demultiplexing means 102 is supplied to a buffer 1005
Is temporarily stored in the decoding unit 10
4 is given.

The buffer 1005 is a buffer for holding the encoded signal 103 for each image. The buffer capacity detecting means 1001 is a detecting means for detecting a buffer amount necessary for buffering the coded signal 103 of each image. The CPU 1002 manages the buffer 1005 based on the detection result of the priority detection unit 601 and the detection result of the buffer capacity detection unit 1001, and sends the used area information to the buffer used area information storage unit 1003 and the buffer free area information storage unit 1004. And functions to record or update information on the free space.

The operation of the image decoding apparatus thus configured will be described. Note that the same means and signals as those in FIGS. 1 and 6 perform the same operations as in the first and second embodiments, and therefore, the description thereof will be simplified.

The demultiplexing means 102 in FIG. 10 separates the input multiplexed signal 101 into a coded signal 103 for each image. The buffer capacity detecting means 1001 detects each encoded signal 10
3, the required buffer capacity is detected, and the CPU 100
Notify 2. The priority detection means 601 detects the priority of the buffer area allocation.

The priority is determined on the image multiplexing device side and multiplexed in a multiplexed signal, or transmitted by using another communication channel, and the priority is extracted on the image decoding device side. can do. The priority can be multiplexed on the image multiplexing device side in the same manner as in the above-described embodiment. Also,
The priority can be determined independently on the image decoding device side.

The buffer use area information storage means 1003 manages information on the buffer use area for each coded signal 103 in the multiplexed signal. In addition, information on the free area is managed by the buffer free area information storage unit 1004. These management methods are the same as in the first embodiment. Furthermore, using the priority information detected by the priority detection unit 601, the allocation of the buffer area can be managed in the same manner as in the second embodiment.

The decoding means 104 takes out a necessary coded signal from the buffer 1005 and decodes it into an image signal. For the buffer area that is no longer necessary after decoding, the CP
U1002 requests release.

As described above, in the present embodiment, it is possible to efficiently manage the buffer for decoding the composite image according to the priority.

In the present embodiment, one buffer is used for each coded signal, but it may be divided into a plurality of buffers and used.

[0094] Although the case where the priority is used has been described in the present embodiment, the priority need not be used.

The method of the present embodiment is applied to the second embodiment.
May be combined with the first and second methods.

Although the present embodiment has been described with reference to a case where there is one buffer, a plurality of buffers may be provided.

Although the present embodiment has been described with reference to a case where there is one decoding unit, a plurality of decoding units may be provided.

In this embodiment, the case of multiplexing the coded image signal has been described, but not only the coded signal of the image but also other signals such as the coded signal of audio and digital data such as text information. Data multiplexing may be used.

(Embodiment 5) A recording medium according to Embodiment 5 of the present invention will be described. The recording medium according to the present embodiment is a recording medium 913 that stores or records a multiplexed signal 911 output from the image multiplexing apparatus in units of programs or sequences, as shown in FIG. An optical disk, a magnetic disk, a flash memory, or the like is used as the recording medium 913.

When transmitting the multiplexed signal 911 to the image decoding apparatus, a transmission medium 914 is used. The transmission medium 914 may be a wireless medium such as a radio wave or a wired medium such as an electric wire. . Embodiments 1, 2, and
4 decodes the multiplexed signal transmitted by the transmission medium 914.

In this embodiment, the case of multiplexing the coded image signal has been described. However, not only the coded signal of the image but also the multiplexed signal including the coded signal of audio and digital data such as text information are included. It may be.

[0102]

According to the first to fifth aspects of the present invention, the work area storage means for decoding an encoded signal can be operated efficiently. Therefore, in the object encoding method, information necessary for decoding can be accumulated while referring to the work area amount and the free area amount, and data including an image can be efficiently decoded.

According to the sixth to ninth aspects of the present invention, the work area storage means for decoding the coded signal can be operated efficiently. For this reason, in the object encoding method, the capacity of the work area storage unit does not break down,
Data including an image can be decoded from the highest priority to the lowest priority.

Further, according to the present invention, the buffer means for holding the encoded signal can be operated efficiently. For this reason, in the object encoding method, data including an image can be decoded from a higher priority to a lower priority without causing a failure in the capacity of the buffer means.

According to the present invention, priority information is generated so that the work area storage means or buffer means on the decoding device side can be operated efficiently, and the priority information is multiplexed with the coded signal. Output. For this reason, the load on the decoding process of the image decoding device is greatly reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a basic configuration of an image decoding device according to Embodiment 1 of the present invention.

FIG. 2 is an explanatory diagram of work area management information of the image decoding apparatus according to the first embodiment.

FIG. 3 is a flowchart showing an operation of allocating a work area by a CPU according to the first embodiment;

FIG. 4 is a flowchart showing processing for releasing a work area of a CPU according to the first embodiment;

FIG. 5 is an explanatory diagram showing a method for rearranging a work area in a work area storage unit according to the first embodiment.

FIG. 6 is a block diagram illustrating a basic configuration of an image decoding device according to Embodiment 2 of the present invention.

FIG. 7 is a flowchart illustrating an operation (first method) of allocating a work area by a CPU according to the second embodiment;

FIG. 8 is a flowchart showing a work area allocation operation (second method) of a CPU according to the second embodiment.

FIG. 9 is a block diagram illustrating a basic configuration of an image multiplexing apparatus according to Embodiment 3 of the present invention.

FIG. 10 is a block diagram showing a basic configuration of an image decoding device according to Embodiment 4 of the present invention.

FIG. 11 is an explanatory diagram illustrating the concept of a composite image.

FIG. 12 is a block diagram illustrating a configuration of a conventional image multiplexing device and a conventional image decoding device.

[Explanation of symbols]

 101, 911 multiplexed signal 102 demultiplexing means 103, 907 to 909 coded signal 104 decoding means 105 decoded image signal 106 work area storage means 107 reference image signal 108 work area amount detection means 109, 1002 CPU 110 work area information storage Means 111 Free area information storage means 601 Priority detection means 901 to 903 Input image signals 904 to 906 Encoding means 910 Multiplexing means 912 Priority determination means 913 Recording medium 914 Transmission medium 1001 Buffer capacity detection means 1003 Buffer use area information storage Means 1004 Free space information storage means 1005 Buffer 1101 Background image 1102, 1104 Object image 1103, 1105 Transparency information 1106, 1107 Composite image

──────────────────────────────────────────────────の Continued on front page (51) Int.Cl. ⁶ Identification code FI H04N 7/081

Claims (18)

[Claims] 1. An image decoding method for inputting a multiplexed signal obtained by multiplexing data including a plurality of encoded image signals and decoding the multiplexed data, wherein the multiplexed signal is It separates into coded signals, detects the capacity of the work area of the work area storage means necessary for the decoding processing of each of the coded signals as a work area amount, and allocates the detected work area amount to the work area storage means. An image decoding method for decoding the coded signal using a work area assigned to the work area storage means. 2. The image decoding method according to claim 1, wherein the position of the area in said work area storage means allocated to the decoding processing of each of said encoded signals is stored as work area information. 3. The position of a free area in the work area storage means, which is not allocated to the decoding processing of each of the encoded signals,
2. The method according to claim 1, wherein the information is stored as free space information.
The image decoding method as described in the above. 4. A work area information, which is position information of an area in the work area storage means allocated to the decoding processing of each of the encoded signals, and a free area in the work area storage means not allocated to the decoding processing. 2. The image decoding method according to claim 1, further comprising storing free area information as position information. 5. An image decoding apparatus for inputting a multiplexed signal obtained by multiplexing data including a plurality of encoded image signals and decoding the multiplexed data, wherein the multiplexed signal is Demultiplexing means for separating into coded signals, work area storage means for holding information used for decoding coded signals, and decoding processing of each coded signal output from the demultiplexing means A work area amount detecting means for detecting a capacity of a work area; a control means for allocating the work area amount detected by the work area amount detecting means to the work area storage means; a work area allocated to the work area storage means Decoding means for decoding the coded signal separated by the demultiplexing means by using the decoding means. 6. An image decoding method for inputting a multiplexed signal obtained by multiplexing data including a plurality of encoded image signals and decoding the multiplexed data, wherein the multiplexed signal is Separate into coded signals, detect the capacity of the work area required for the decoding processing of each coded signal as a work area amount, and perform the decoding processing of each coded signal. Detecting the priority of allocation, allocating the detected work area amount to the work area storage means in order from the decoding processing having the higher priority, and using the work area allocated to the work area storage means, An image decoding method for decoding an image. 7. The image decoding method according to claim 6, wherein a work area of said work area storage unit is not assigned to a decoding process of the coded signal whose priority is equal to or less than a predetermined value. 8. When requesting allocation of a predetermined amount of work area for the decoding processing, if there is no free work area in the work area storage means, the work area is allocated to a decoding processing having a lower priority than the decoding processing. 7. The image decoding method according to claim 6, wherein said work area is released and said released work area is allocated to said decoding process. 9. An image decoding apparatus for inputting a multiplexed signal obtained by multiplexing data including a plurality of images and decoding the multiplexed data, wherein each of the coded signals of the data including a plurality of images is decoded. Demultiplexing means for inputting the multiplexed multiplexed signal and separating the coded signals into the coded signals; work area storage means for storing information used for decoding the coded signal; and output from the demultiplexing means. A work area amount detection means for detecting a capacity of a work area required for decoding processing of each encoded signal as a work area amount, and a work area storage means for assigning a work area in the decoding processing of each encoded signal. Priority detection means for detecting priority, the work area amount detected by the work area amount detection means,
Control means for allocating to the work area storage means according to the priority detected by the priority detection means; and a work area allocated to the work area storage means for the coded signal separated by the demultiplexing means. An image decoding device, comprising: decoding means for decoding using the image data. 10. A work area information storage means for storing position information of a work area in the work area storage means as work area information, and a position information of a free area in the work area storage means is stored as free area information. 10. The image decoding apparatus according to claim 5, further comprising: a free area information storage unit. 11. An image decoding method for inputting a multiplexed signal obtained by multiplexing data including a plurality of encoded image signals, and decoding the multiplexed data, wherein the multiplexed signal is Separate into coded signals, detect the buffer capacity of the buffer means necessary for temporarily holding each of the coded signals, and at the time of decoding each of the coded signals, assign the priority of the buffer area allocation of the buffer means. An image decoding method, comprising: detecting, allocating the detected buffer capacity to the buffer means in order from a coded signal having a high priority, and decoding each coded signal held in the buffer means. 12. An image decoding method for inputting a multiplexed signal obtained by multiplexing data including a plurality of encoded image signals and decoding the multiplexed data, wherein the multiplexed signal is Separate into coded signals, detect the buffer capacity of the buffer means necessary for temporarily holding each of the coded signals, and at the time of decoding each of the coded signals, assign the priority of the buffer area allocation of the buffer means. Detecting and allocating the detected buffer capacity to the buffer means in order from a coded signal having a high priority.When requesting allocation of a buffer area for holding the coded signal, if there is no free area in the buffer means, Releasing the buffer use area allocated for the low priority coded signal, and holding the coded signal requesting the buffer area. Allocating the released buffer area, and decoding each encoded signal held in the buffer means. 13. An image decoding apparatus for inputting a multiplexed signal obtained by multiplexing data including a plurality of encoded image signals and decoding the multiplexed data, wherein the multiplexed signal is Demultiplexing means for separating into coded signals, buffer means for temporarily holding each coded signal, and buffer for detecting the buffer capacity required for temporarily holding each coded signal output from the demultiplexing means Capacity detection means, priority detection means for detecting the priority of the allocation of the buffer area of the buffer means in the decoding processing of each of the encoded signals, and the buffer capacity detected by the buffer capacity detection means, Control means for allocating to the buffer means according to the priority detected by the degree detecting means, and decoding means for decoding the coded signal held by the buffer means. An image decoding apparatus, comprising: 14. An image multiplexing method for generating a coded signal obtained by coding data including a plurality of images, and outputting a multiplexed signal obtained by multiplexing the coded signals, comprising: Wherein the priority of assignment in a work area storage means used as a work area at the time of decoding is multiplexed with the coded signal and output. 15. An image multiplexing apparatus for generating a coded signal obtained by coding data including a plurality of images and outputting a multiplexed signal obtained by multiplexing the coded signals, wherein the multiplexed signal includes a plurality of images. A plurality of encoding means for encoding data to generate an encoded signal; a priority determining means for determining a priority of assignment in a work area storage means used as a work area at the time of decoding on the image decoding device side; An image multiplexing apparatus, comprising: a multiplexing unit that multiplexes and outputs a coded signal output from each of the coding units and a priority signal output from the priority determination unit. 16. An image multiplexing apparatus for generating a coded signal obtained by coding data including a plurality of images and outputting a multiplexed signal obtained by multiplexing the coded signals, wherein the multiplexed signal includes a plurality of images. A plurality of encoding means for encoding data to generate an encoded signal; and a priority determining means for determining a priority of allocation to a buffer means for holding an encoded signal used at the time of decoding; An image multiplexing apparatus comprising: multiplexing means for multiplexing and outputting a coded signal output from each coding means and a priority signal output from the priority determination means. 17. An operation for generating a coded signal obtained by coding a plurality of images and other data and multiplexing the coded signals, and a coded signal used for decoding. A storage medium storing priority of assignment to an area storage means. 18. A buffer for generating a coded signal obtained by coding a plurality of images and other data and multiplexing the coded signals, and a coded signal used for decoding. A storage medium storing priority of assignment to means.