JPH10155149A - Frame rate conversion method and frame rate conversion device - Google Patents

Abstract

(57) Abstract: To perform frame rate conversion in accordance with the processing capability of a terminal, and to perform high-speed, low-load frame rate conversion of hybrid-coded image data including blocks with motion compensation. . SOLUTION: A plurality of frames are synthesized into one frame without performing inverse orthogonal transform, motion detection, and orthogonal transform on input hybrid-encoded image data including a block with motion compensation, and The image data is converted into hybrid-encoded image data of a block without motion compensation of the number of frames corresponding to the processing capacity and a block which has been intra-coded, and outputs the image data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION The present invention relates to a video conference system and a method for transmitting a moving image. 261 or the like, when transmitting hybrid-coded image data between terminals having different processing capabilities, or when transmitting hybrid-coded image data by relaying communication lines having different transmission speeds, or when the processing capability is A frame rate conversion method for transmitting hybrid-coded image data by relaying communication lines with different transmission speeds between different terminals, and a hybrid coding method for performing orthogonal transformation on inter-frame prediction errors involving motion compensation. Processing rate using a frame rate conversion method for image data containing compressed data, and a frame rate conversion method for image data containing hybrid encoded data that performs orthogonal transformation on inter-frame prediction errors involving motion compensation When transmitting hybrid-coded image data between different terminals, When transmitting hybrid-coded image data by relaying communication lines with different transmission speeds, or when transmitting hybrid-coded image data by relaying communication lines with different transmission speeds between terminals with different processing capabilities And a frame rate conversion apparatus.

[0002]

2. Description of the Related Art A conventional frame rate converter will be described with reference to the drawings. FIG. 20 is a block diagram showing a configuration of a conventional frame rate conversion method (apparatus). 20, 821 and 825 are terminals, 822 is connected to the terminal 821, a communication line with a high transmission speed, 824 is connected to the terminal 825, a communication line with a low transmission speed, and 823 is the transmission speed This is a frame rate conversion unit which is a conventional frame rate conversion device provided between the communication line 822 having a high transmission speed and the communication line 824 having a low transmission speed.

Next, the operation will be described. In FIG. 20, the frame rate conversion unit 823 transmits the hybrid-coded image data having a high frame rate output from the terminal 821 to the terminal 825 in real time. upper,
The frame rate of the hybrid-coded image data with a high frame rate is converted and output as hybrid-coded image data with a low frame rate over a slow communication line 824.

[0004] As a frame rate conversion method for hybrid-coded image data in which orthogonal transformation is performed on inter-frame prediction errors accompanied by motion compensation, for example, a rate conversion image disclosed in Japanese Patent Application Laid-Open No. 7-50834 is disclosed. A frame rate conversion method using an apparatus having a configuration in which a variable length decoder and a variable length encoder are added to an encoding apparatus will be described below with reference to the drawings.

FIG. 10 shows a rate-converted image coding apparatus having a configuration in which a variable-length decoder and a variable-length encoder are added to the rate-converted image coding apparatus disclosed in JP-A-7-50834.

In FIG. 10, reference numeral 901 denotes an input of image data; 902, a variable length decoding means for decoding the input image data; 903, an inverse quantization means for inversely quantizing the decoded data; Addition means for adding the output of the inverse quantization means 903 to the data output from the image memory 920; selectors 921 and 922; inverse orthogonal transformation means 923; addition means 924; image memory 925;
Reference numeral 6 denotes an in-loop filter unit that receives an image memory output from the image memory 925 as an input, 927 a selector, 92
8 is a subtraction means, 930 is an orthogonal transformation means, 931 is a selector, 914 is a quantization means, 932 is an inverse quantization means, 93
3 is an inverse orthogonal transforming means, 934 is an adding means, 935 is an image memory, 936 is a filter in a loop, 937 is a selector, 938 is a motion detecting means, 939 is an encoding control means, 917 is a variable length encoding means, and 918. Is the output.

Next, the operation will be described. In FIG. 10, in order to perform rate conversion of a frame accompanied by motion compensation having a high frame rate and input from an input 901, first, hybrid-encoded image data is subjected to variable-length decoding by a variable-length decoding unit 902. Then, the data is converted into quantized prediction error orthogonal transform data with motion compensation, and then inversely quantized by the inverse quantization means 903 to be converted into prediction error orthogonal transform data with motion compensation.

[0008] The prediction error orthogonal transform data is
The inverse orthogonal transform means 923 selected by the selector 922
, And is converted to prediction error data accompanied by motion compensation.
Is added to the image memory 925 that has passed through the in-loop filter means 926 selected in step (1), and is converted into coefficients (pixel data).

Using the coefficient (pixel data) output from the adding means 924 and the coefficient (pixel data) on the image memory 935, the motion detecting means 938 performs motion detection to obtain a motion vector, and subtracts the motion vector. By means 928, from the coefficient (pixel data) output from the adding means 924,
After passing through the image memory 935 considering the motion vector through the in-loop filter means 936, the image data is subtracted and converted into prediction error data with motion compensation.

[0010] The obtained prediction error data with motion compensation is orthogonally transformed by orthogonal transform means 930, converted into prediction error orthogonal transformation data with motion compensation, selected by selector 931, and selected at the subsequent stage. Image data that has been quantized by the converting means 914 and converted into quantized prediction error orthogonal transform data with motion compensation, and subsequently, is subjected to variable-length coding by the variable-length coding means 917 and hybrid-coded. It is converted and output as a frame with motion compensation at a lower frame rate than output 918.

If the hybrid-coded image data having a high frame rate is only intra-frame coded image data, it is possible to omit processing such as inverse orthogonal transformation, motion detection, and orthogonal transformation. If at least one block with motion compensation is included, it is necessary to detect a motion vector, and it is necessary to perform all of the above processing.

In the frame rate converter configured as described above, when converting the frame rate, the conversion is performed based on the transmission speed of the communication line. That is,
In FIG. 20, for example, terminal 821 has an encoding capability of 30 fps, terminal 825 has a decoding capability of 5 fps, communication line 822 has a transmission capability of 30 fps, and communication line 824 has a transmission capability of 15 fps. Then At this time, the frame rate converter (converter 82
In 3), a 30 fps frame is converted to 15 fps.

For this reason, 15 fp is placed on the communication line 824.
Although the hybrid-coded image data of s is transmitted, the terminal 825 can decode only at a speed of 5 fps, and thus can display an image only at a speed of 5 fps. From this, it can be seen that unnecessary hybrid-coded image data that has not been decoded has been transmitted on the communication line 824. Therefore, the use efficiency of the communication line 824 at the subsequent stage of the frame rate conversion unit 823 is deteriorated.

Further, as can be seen from the above, in order to perform rate conversion of a frame including motion compensation and a frame including a block including motion compensation, decoding of hybrid-coded data and coefficients (pixel data) are performed. ), The entire process had to be performed, and the conversion was time-consuming and burdensome.

[0015]

The conventional frame rate conversion apparatus is configured as described above. Between communication lines having different transmission speeds, the frame rate conversion device is simply adapted to the communication line having the lower transmission speed. Since the conversion is performed so as to have an appropriate frame rate, there is a problem that the use efficiency of a communication line downstream of the frame rate conversion device may be deteriorated. In addition, there is a problem that a processing time for processing a frame with motion compensation is long and a heavy load is imposed on the system.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and is intended for converting hybrid-coded image data having a high frame rate into hybrid-coded image data having a low frame rate. A frame rate conversion method capable of improving the efficiency of use of a communication line and generating hybrid-encoded image data having a frame rate corresponding to the processing capability of the terminal by considering the processing capability of the terminal; It is an object to provide a rate conversion device.

In addition, even when performing rate conversion of a frame including motion compensation and a frame including a block including motion compensation, a frame rate capable of performing a high-speed and low-load frame rate conversion by greatly reducing the amount of calculation. It is an object to provide a conversion method and a frame rate conversion device.

[0018]

According to the frame rate conversion method according to the first aspect of the present invention, when hybrid-coded image data is communicated between terminals, one terminal is a1 and the other terminal is a2. When the transmission speed of the communication line to which the terminal a1 is connected is b1, the transmission speed of the communication line to which the terminal a2 is connected is b2, the processing capability of a1 is c1, and the processing capability of the terminal a2 is c2 , Input image data with a high frame rate from the terminal a1 side,
The number of frames to be output to the terminal a2 is determined by the transmission speed b1 ≧
When b2 and the processing capacity of the terminal is c1 = c2,
Set the number of frames that can be transmitted in b2, and set the transmission speed to b1
If ≧ b2 and the processing capability of the terminal is c1> c2, the number of frames that can be transmitted in b2 or the number of frames that can be processed in c2 is set to the smaller number, the transmission speed is b1 <b2, and The processing capacity of the terminal is c1> c
2, the number of frames that can be transmitted by b1 or the number of frames that can be processed by c2 is set to the smaller number. If the transmission speed is b1> b2 and the processing capability of the terminal is c1 <c2, b2 Is set to the smaller of the number of frames that can be transmitted by c1 and the number of frames that can be processed by c1, and the frame rate of the high frame rate image data input from the terminal a1 is converted to a frame to the terminal a2. The image data is output as low-rate image data.

In the frame rate conversion method according to the second aspect of the present invention, hybrid-coded image data for performing orthogonal transformation on an inter-frame prediction error accompanied by high frame rate motion compensation is input, A plurality of hybrid-encoded image data frames that perform orthogonal transformation on inter-frame prediction errors with compensation
The frame rate is converted by synthesizing a frame of hybrid-coded image data that performs orthogonal transform on inter-frame prediction errors without motion compensation
This is output as hybrid-coded image data that performs orthogonal transformation on an inter-frame prediction error without motion compensation at a low frame rate.

Further, the frame rate conversion method according to the third aspect of the present invention is a method of performing a hybrid coded image data for performing an orthogonal transformation on an inter-frame prediction error accompanied by a high frame rate motion compensation, and a motion compensation. Hybrid coding that performs orthogonal transform on inter-frame prediction error without image data and image data that includes both encoded image data and performs orthogonal transform on inter-frame prediction error with motion compensation Image data and
Performs orthogonal transformation on inter-frame prediction error without motion compensation. A plurality of frames of image data including both the hybrid-coded image data and the hybrid-coded image data are grouped together, and one inter-frame prediction error without motion compensation is included. Hybrid encoding that converts the frame rate by combining it with a frame of hybrid-coded image data that performs orthogonal transformation on the frame, and performs orthogonal transformation on inter-frame prediction errors without motion compensation at low frame rates This is output as the processed image data.

According to a fourth aspect of the present invention, there is provided a frame rate conversion system, comprising: hybrid-encoded image data for performing orthogonal transformation on an inter-frame prediction error accompanied by motion compensation having a high frame rate; Image data that includes both the image data and the encoded image data, hybrid-encoded image data that performs orthogonal transformation on inter-frame prediction errors involving motion compensation, and intra-frame encoded image data. , A plurality of frames of image data including both of the above, hybrid encoding image data that performs orthogonal transformation on an inter-frame prediction error without motion compensation, and intra-frame encoded image data. The frame rate is converted by synthesizing it into a frame of image data that contains both And image data hybrid coding performing orthogonal transformation on the inter-frame prediction error without amortization is to that output as image data that contains both the intra-frame coded image data.

Further, the frame rate conversion method according to claim 5 of the present invention is characterized in that hybrid-coded image data for performing orthogonal transformation on an inter-frame prediction error accompanied by motion compensation with a high frame rate, and motion compensation are performed. Image data including three pieces of image data, that is, hybrid-coded image data that performs orthogonal transformation on an inter-frame prediction error that is not accompanied by intra-frame coding, and inter-frame prediction error that involves motion compensation Hybrid-encoded image data that performs orthogonal transformation on, and hybrid-encoded image data that performs orthogonal transformation on inter-frame prediction errors without motion compensation, and intra-frame encoded image data A plurality of frames of image data including The frame rate is converted by combining it with a frame of image data that includes both hybrid-coded image data that performs orthogonal transform and intra-frame coded image data, which involves motion compensation with a low frame rate. Hybrid-encoded image data that performs orthogonal transformation on no inter-frame prediction error;
The image data is output as image data including both the intra-frame encoded image data.

According to a sixth aspect of the present invention, there is provided a frame rate conversion system according to any one of the second to fifth aspects, wherein the frame rate conversion is performed in the frame rate conversion system according to the first aspect. The frame rate conversion is performed using the described frame rate conversion method.

According to a seventh aspect of the present invention, there is provided a frame rate conversion apparatus for converting a frame rate using the frame rate conversion method according to any one of the first to sixth aspects. .

[0025]

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiment 1 FIG. Hereinafter, Embodiment 1 corresponding to claim 1 of the present invention. About the frame rate conversion method according to
This will be described with reference to the configuration of the frame rate conversion method shown in FIG.

FIG. 1 is a block diagram showing the configuration of the frame rate conversion system.

In FIG. 1, reference numeral 121 denotes a terminal, and its processing capability is assumed to be c1. Reference numeral 122 denotes a communication line connected to the subsequent stage of the terminal 121, and its transmission speed is assumed to be b1. Reference numeral 123 denotes a frame rate conversion unit connected to the subsequent stage of the communication line 122. The frame rate conversion information is input from the terminal 125 and the transmission rate b of the communication line 122.
1, the transmission rate b2 of the communication line 124, and the processing capability c2 of the terminal 125, are configured to perform frame rate conversion. A communication line 124 has a transmission speed b2. A terminal 125 has a processing capability of c2.

Hereinafter, image data having a high frame rate is input from the terminal 121, and the transmission speed of the communication line is set to b1.
≧ b2 and the processing capability of the terminal is c1 =
The frame rate conversion method for c2 will be described with reference to FIG.

Here, since c1 = c2, the processing capabilities of the terminals are equal, and it is not necessary to consider the conversion of the frame rate according to this. Therefore, attention is paid to the relationship between the transmission speeds b1 and b2 between the communication lines.

Since the transmission speed b2 of the communication line 124 is equal to or lower than the transmission speed b1 of the communication line 122, the frame rate converter 123 outputs the transmission speed b2 of the communication line 124.
In this case, the frame rate can be converted to a frame rate that can be transmitted. Therefore, the hybrid-encoded image data with a high frame rate input from the terminal 121 is converted into a hybrid-encoded image data with a low frame rate, which has the performance of the transmission rate b2, and is subjected to frame rate conversion. Is output to At the terminal 125, image data having a frame rate equal to the transmission speed b2 of the communication line 124 is sent, and is completely reproduced and displayed within the processing capability range of the terminal 125.

Next, image data having a high frame rate is input from the terminal 121, and the transmission speed of the communication line is set to b1.
≧ b2 and the processing capability of the terminal is c1>
The frame rate conversion method at the time of c2 will be described with reference to FIG.

The processing capacities c1 and c2 of the terminals 121 and 125
Paying attention to the relationship, since the processing capability c2 of the terminal 125 is lower than the processing capability c1 of the terminal 121, the frame rate can be converted to a frame rate that can be processed by the processing capability c2 of the terminal 125. Focusing on the relationship between the transmission speeds b1 and b2 of the communication lines 122 and 124, since the transmission speed b2 of the communication line 124 is equal to or lower than the transmission speed b1 of the communication line 122, transmission is possible at the transmission speed b2 of the communication line 124. What is necessary is to convert it to a frame rate.

Here, the transmission rate b of the communication line 124 is larger than the number of frames that can be processed by the processing capacity c2 of the terminal 125.
If the number of frames that can be transmitted by the terminal 2 is larger, the terminal 125
The terminal 125 cannot decode even if the number of frames that can be processed with the processing capability c2 of
The use efficiency of the communication line becomes worse. In order to avoid this, the hybrid-coded image data with a high frame rate input from the terminal 121 side is a hybrid-coded image data with a low frame rate that is in the performance of the processing capability c2 of the terminal 125. And then output to the terminal 125 side.

The transmission speed b2 of the communication line 124 is larger than the number of frames that can be processed by the processing capability c2 of the terminal 125.
If the number of frames that can be transmitted by the
Since only the number of frames that can be transmitted at the transmission rate b2 of 24 can be transmitted, the hybrid-coded image data having a high frame rate input from the terminal 121 side does not meet the performance of the transmission rate b2 of the communication line 124. Was
The frame rate is converted to hybrid-coded image data having a low frame rate and output to the terminal 125 side.

Further, the number of frames that can be processed by the processing capability c2 of the terminal 125 and the transmission speed b2 of the communication line 124
When the number of frames that can be transmitted by the terminal 121 is equal, the hybrid-encoded image data having a high frame rate input from the terminal 121 is transmitted at the transmission speed b of the communication line 124.
2 or the frame rate is converted to hybrid-coded image data with a low frame rate that matches the processing capability c2 of the terminal 125 and is output to the terminal 125 side.

Next, image data having a high frame rate is input from the terminal 121, and the transmission speed of the communication line is set to b.
1 <b2 and the processing capability of the terminal is c1
The frame rate conversion method in the case of> c2 will be described with reference to FIG.

Processing capacities c1 and c2 of terminals 121 and 125
Paying attention to the relationship, since the processing capability c2 of the terminal 125 is lower than the processing capability c1 of the terminal 121, the frame rate can be converted to a frame rate that can be processed by the processing capability c2 of the terminal 125. Focusing on the relationship between the transmission speeds b1 and b2 of the communication lines 122 and 124, since the transmission speed b1 of the communication line 122 is lower than the transmission speed b2 of the communication line 124, it is converted to a frame rate that can be transmitted at the transmission speed b1. That's all I need to do.

Here, the transmission speed b1 of the communication line 122 is larger than the number of frames that can be processed by the processing capability c2 of the terminal 125.
When the number of frames that can be transmitted by the terminal 125 is larger, even if the number of frames that can be processed by the processing capability c2 of the terminal 125 is transmitted, the terminal 125 cannot decode the frame, so that the use efficiency of the communication line deteriorates. . In order to avoid this, the hybrid-coded image data with a high frame rate input from the terminal 121 is
The frame rate is converted to hybrid-coded image data having a low frame rate and having a processing capability c2 of 25 and output to the terminal 125 side.

The transmission speed b1 of the communication line 122 is larger than the number of frames that can be processed by the processing capability c2 of the terminal 125.
If the number of frames that can be transmitted by the
Since only the number of frames that can be transmitted at the transmission rate b1 of 22 can be transmitted, the hybrid-coded image data with a high frame rate input from the terminal 121 does not meet the performance of the transmission rate b1 of the communication line 122. Was
The frame rate is converted to hybrid-coded image data having a low frame rate and output to the terminal 125 side.

Further, the processing capability c2 of the terminal 125
Number of frames that can be processed by the communication and the transmission speed b of the communication line 122
1 and the number of frames that can be transmitted is equal, the terminal 121
The hybrid-encoded image data with a high frame rate input from the side
1 or the frame rate is converted to hybrid-encoded image data having a low frame rate that matches the processing capability c2 of the terminal 125, and is output to the terminal 125 side.

Next, image data with a high frame rate is input from the terminal 121 side, and the transmission speed of the communication line is b1>
b2 and the processing capability of the terminal is c1 <c
The frame rate conversion method in the case of the relationship 2 will be described with reference to FIG.

Processing capacities c1 and c2 of terminals 121 and 125
Paying attention to the relationship, since the processing capability c1 of the terminal 121 is lower than the processing capability c2 of the terminal 125, the frame rate may be converted to a frame rate that can be processed by the processing capability c1 of the terminal 121. Focusing on the relationship between the transmission speeds b1 and b2 of the communication lines 122 and 124, since the transmission speed b2 of the communication line 124 is lower than the transmission speed b1 of the communication line 122, transmission is possible at the transmission speed b2 of the communication line 124. It is only necessary to convert the frame rate to an appropriate frame rate.

Here, the transmission rate b of the communication line 124 is larger than the number of frames that can be processed by the processing capacity c1 of the terminal 121.
If the number of frames that can be transmitted by the terminal 2 is larger, the terminal 121
A frame exceeding the number of frames that can be processed by the processing capability c1 cannot be transmitted even if the transmission speed b2 of the communication line 124 is used. Therefore, the hybrid-encoded image data input from the terminal 121 is output to the terminal 125 without any frame rate conversion.

The transmission speed b2 of the communication line 124 is larger than the number of frames that can be processed by the processing capability c1 of the terminal 121.
If the number of frames that can be transmitted by the
Since only the number of frames that can be transmitted at the transmission rate b2 of 24 can be transmitted, the hybrid-coded image data having a high frame rate input from the terminal 121 side does not meet the performance of the transmission rate b2 of the communication line 124. Was
The frame rate is converted to hybrid-coded image data having a low frame rate and output to the terminal 125 side.

When the number of frames that can be processed by the processing capability c1 of the terminal 121 is equal to the number of frames that can be transmitted at the transmission speed b2 of the communication line 124, the hybrid-encoded image data input from the terminal 121 Is output to the terminal 125 as it is without frame rate conversion.

The details of the frame rate conversion processing will be described below. Assuming that the input image data has no frame rate conversion from the 0th to the i-th frame, (i +
It is assumed that the frame rate conversion is performed by combining the frames from (1) to the (i + n) -th frame, and the frame after the (i + n + 1) -th frame has no frame rate conversion. The case where the input image data having a high frame rate is intra-frame encoded data will be described as an example with reference to the drawings.

FIG. 2 shows a case where (i + 1) to (i + n) -th frames of the input image data having a high frame rate are intra-coded data. It shows a frame rate conversion method at the time of processing.

In FIG. 2, reference numeral 101 denotes an input to which image data having a high frame rate is input.

Reference numeral 102 denotes a variable length decoding means.
By performing variable-length decoding of the hybrid coded data, the prediction error orthogonal transform data with quantized motion compensation, the prediction error orthogonal transform data without quantized motion compensation, and the quantized orthogonal transform coefficients are obtained. Convert. Also,
At the same time, the intra-frame / inter-frame identification set by the encoding device of the input terminal is detected, the intra-frame / inter-frame (in) is set, and the setting is made by the encoding device of the input terminal. , The presence / absence of motion compensation is detected, and the presence / absence of motion compensation (in) is set, and the quantization coefficient set in the encoding device of the input terminal is detected, and the quantization coefficient (in ) Is set, and the motion vector set by the coding device of the input terminal is detected, and the motion vector (in) is set.

Reference numeral 103 denotes an inverse quantization means, which uses the set quantization coefficient (in) to quantize the prediction error orthogonal transform data with motion compensation and the quantized prediction error orthogonal transform data without motion compensation. Transform data, the quantized orthogonal transform coefficients are transformed into prediction error orthogonal transform data with motion compensation,
The data is converted into prediction error orthogonal transform data without motion compensation and orthogonal transform coefficients.

Numeral 104 denotes a prediction error orthogonal transform data adding means, which is an orthogonal transform coefficient calculated in consideration of the motion vector by the motion vector / orthogonal transform coefficient calculating means 108, or an orthogonal transform coefficient stored in the image memory 106. The coefficient and prediction error orthogonal transform data with motion compensation or prediction error orthogonal transform data without motion compensation are added.

Reference numeral 105 denotes an orthogonal transform coefficient selecting means, which is controlled by the set intra-frame / inter-frame (in). That is, within the frame / between frames (i
When n) is “intra-frame”, the orthogonal transform coefficient inversely quantized by the inverse quantization means 103 is selected. When the intra-frame / inter-frame (in) is “inter-frame”, the prediction error orthogonal The orthogonal transform coefficients calculated by the transform data adding means 104 are selected and output.

Reference numeral 106 denotes an image memory for storing the orthogonal transform coefficients selected by the orthogonal transform coefficient selecting means 105.

Numeral 107 is a means for judging whether motion compensation is performed or not (in).
6 is output to the motion vector / orthogonal transform coefficient calculating means 108 described later. When the motion compensation is “none”, the orthogonal transformation coefficient stored in the image memory 106 is added to the prediction error orthogonal transformation data adding means 104.
Output to

Numeral 108 denotes a motion vector / orthogonal transform coefficient calculating means, which uses the motion vector (in) to determine whether or not there is motion compensation (in). By performing a matrix operation on the orthogonal transform coefficients stored in the matrix, the orthogonal transform coefficients calculated in consideration of the motion vector are calculated. For the matrix operation, a method described later is used.

Reference numeral 109 denotes an initial image capturing / selecting unit, which is an encoding control / intra-frame / inter-frame determining unit 1 described later.
In response to an image capture instruction set by 13, the orthogonal transform coefficients stored in the image memory 106 are captured, and the orthogonal transform coefficients are output to an image memory 110 described later.

Reference numeral 110 denotes an image memory for storing the orthogonal transformation coefficients selected by the initial image acquisition selecting means 109.

Reference numeral 111 denotes within a frame / between frames (in).
It is a judgment means.
06 orthogonal transformation coefficient is set at the subsequent stage of encoding control / in-frame /
It outputs to the inter-frame determination means 113, and in the case of "inter-frame", outputs the orthogonal transform coefficient of the image memory 106 to the orthogonal transform coefficient subtraction means 112 described later.

Numeral 112 denotes an orthogonal transform coefficient subtracting means, which calculates an image based on the orthogonal transform coefficient stored in the image memory 106 when the intra-frame / inter-frame (in) judging means 111 judges "inter-frame". The orthogonal transform coefficient of the memory 110 is subtracted.

Numeral 113 denotes encoding control / intra-frame / inter-frame judging means. When frame rate conversion is not performed, motion compensation / no motion (in) is output as motion compensation / no motion (out). When the frame rate conversion is being performed, the motion compensation with / without (out) is set to “none” and output. Also, the frame immediately before the frame rate conversion is stored in the image memory 1.
10 to output an image capture instruction. Also, using the intra-frame / inter-frame (in), the orthogonal transform coefficient from the intra-frame / inter-frame (in) determining means 111 and the prediction error orthogonal transform data without motion compensation from the orthogonal transform coefficient subtracting means 112 are used. And outputs the result to a quantization means 114 described later. Further, the quantization coefficient (in) is reset, and this is output to the quantization means 114 described later as a quantization coefficient (out).

Numeral 114 denotes a quantizing means for transforming the orthogonal transform data and the orthogonal transform coefficient without motion compensation by the output quantized coefficient (out) into the quantized prediction error without motion compensation. The orthogonal transform data and the quantized orthogonal transform coefficient are respectively converted.

Reference numeral 115 denotes frame rate conversion control means for determining continuation / end of the frame rate conversion based on the frame rate conversion information. Then, after the frame rate conversion is completed, quantized prediction error orthogonal transform data without motion compensation and quantized orthogonal transform coefficients are output to the variable length coding means 117, and the frame rate conversion is not performed. Outputs the quantized prediction error orthogonal transform data with motion compensation, the quantized prediction error orthogonal transform data without motion compensation, and the quantized orthogonal transform coefficients from the variable length decoding means 102 as they are. .

Reference numeral 116 denotes a motion vector selection means, which is the above-described coding control / intra-frame / inter-frame determination means 113
With and without motion compensation (out) output from
Select a motion vector. That is, with motion compensation /
When “out” is “present”, the motion vector (i)
n) is output as it is to the variable length encoding means 117 described later, and when "none", the motion vector (in) is not output to the variable length encoding means 117.

Numeral 117 denotes a variable-length encoding means which converts the quantized prediction error orthogonal transform data with motion compensation, the quantized prediction error orthogonal transform data without motion compensation, and the quantized orthogonal transform coefficient. By performing variable length coding, the data is converted into hybrid coded data. At the same time, the intra-frame / inter-frame discrimination set for the decoding device of the output terminal is set according to the intra-frame / inter-frame (in), and the motion compensation (with / without (out)) and the motion vector (in) are performed. ), A motion vector with / without motion compensation is set for the decoding device of the output terminal, and a quantization coefficient is set for the decoding device of the output terminal by the quantization coefficient (in). .

Reference numeral 118 denotes an output from which image data having a low frame rate is output.

Here, Emv (i) is prediction error orthogonal transform data with motion compensation, Q (Emv (i)) is quantized prediction error orthogonal transform data with motion compensation, and V (Q
(Emv (i))) is quantized and variable-length coded prediction error orthogonal transform data with motion compensation, S (i) is an orthogonal transform coefficient, and Q (S (i)) is a quantized orthogonal transform. Coefficients, V (Q (S (i))) are quantized and variable-length coded orthogonal transform coefficients, S ′ (i) is an orthogonal transform coefficient calculated in consideration of a motion vector, and E (i) is The prediction error orthogonal transform data without motion compensation, Q (E (i)) is the quantized prediction error orthogonal transform data without motion compensation, V
(Q (E (i))) is quantized and variable-length coded prediction error orthogonal transform data without motion compensation, Emv_n
onmv (i) is prediction error orthogonal transform data with motion compensation, or prediction error orthogonal transform data without motion compensation, Q (Emv_nonmv (i)) is quantized prediction error orthogonal transform data with motion compensation, or Prediction error orthogonal transformation data without quantized motion compensation, V
(Q (Emv_nonmv (i))) is quantized and variable-length coded prediction error orthogonal transform data with motion compensation, or quantized and variable-length coded prediction error orthogonal transform data without motion compensation; SEmv (i) is prediction error orthogonal transform data with motion compensation or orthogonal transform coefficient, and Q (SEmv (i)) is quantized prediction error orthogonal transform data with motion compensation or quantized orthogonal transform coefficient. , V (Q (SEmv (i))) are quantized and variable-length coded prediction error orthogonal transform data with motion compensation, or quantized and variable-length coded orthogonal transform coefficients, SEmv_nonmv (i) is Prediction error orthogonal transformation data with motion compensation, prediction error orthogonal transformation data without motion compensation, or orthogonal transformation coefficient, Q (SE
mv_nonmv (i) is quantized prediction error orthogonal transform data with motion compensation, quantized prediction error orthogonal transform data without motion compensation, or quantized orthogonal transform coefficient, V (Q (SEmv_nonmv
(I))) is quantized and variable-length encoded prediction error orthogonal transform data with motion compensation, or is quantized and variable-length encoded prediction error orthogonal transform data without motion compensation, or is quantized and The variable-length-coded orthogonal transform coefficients are used.

Variable-length decoding means 102, inverse quantization means 10
3. Prediction error orthogonal transform data adding means 104, orthogonal transform coefficient selecting means 105, motion compensation with / without (in) determining means 107, motion vector / orthogonal transform coefficient calculating means 10
8 and stored in the image memory 106 as S (i). The data is also stored as S (i) in the image memory 110 via the initial image capture selecting means 109. Also,
The variable length decoded Q by the variable length decoding means 102
(SEmv_nonmv (0)) to Q (SEmv_n
onmv (i)) is the frame rate conversion control means 11
5 and is variable-length coded by the variable-length coding means 117, and V (Q (SEmv_nonmv
(0))) to V (Q (SEmv_nonmv
(I)) is output as).

On the other hand, frame rate conversion is performed (i +
1) The frame is subjected to variable-length decoding by the variable-length decoding unit 102 and quantized orthogonal transform coefficients Q (S (i +
1)).

Then, Q (S (i + 1)) is inversely quantized by the inverse quantization means 103, and the orthogonal transform coefficient S (i +
Is converted to 1). This orthogonal transformation coefficient selection means 105 determines that S (i + 1) is “in a frame”, and stores it in the image memory 106 as S (i + 1).

The processes from the (i + 2) th frame to the (i + n) th frame are processed by repeating the same process as the (i + 1) th frame.

When the processing of the (i + n) frame is completed, the orthogonal transform coefficient S is stored in the image memory 106.
(I + n) are stored, and the image memory 110 stores the orthogonal transform coefficient S (i).

The orthogonal transformation coefficient S (i + n) is determined to be “in the frame” by the intra-frame / inter-frame (in) determination means 111, and S (i + n) is output as it is.
Here, the orthogonal transform coefficient S (i + n) is replaced with a new (i + 1)
It is assumed that the frame is the first frame and S (i + 1) = S (i + n).

The above-mentioned orthogonal transform coefficient S (i + 1) is obtained by using the intra-frame / inter-frame (in) by the encoding control / intra-frame / inter-frame judging means 113.
It is determined to be "in a frame" and is selected. During frame rate conversion, motion compensation with / without (out) is “none”
Is set to The image capture instruction is sent to the quantization means 114 as S
After (i + 1) is output, it is output.

S (i + 1) is quantized by the quantization means 114, and the quantized orthogonal transform coefficient Q (S (i +
1)).

The quantized orthogonal transform coefficients Q (S (i
+1)) is subjected to variable-length encoding by the variable-length encoding means 117, and is output as a quantized and variable-length encoded orthogonal transform coefficient V (Q (S (i + 1))).

On the other hand, no frame rate conversion is performed (i + n
The (+1) th frame includes a variable length decoding unit 102, an inverse quantization unit 103, a prediction error orthogonal transform data addition unit 1
04, orthogonal transform coefficient selecting means 105, motion compensation presence / absence (in) judging means 107, and motion vector / orthogonal transform coefficient calculating means 108, and are stored in image memory 106 as orthogonal transform coefficients S (i + n + 1). Further, the orthogonal transformation coefficient S (i + n + 1) is also stored in the image memory 110 via the initial image acquisition selection means 109. Q (SEmv_nonmv (i + n + 1)) obtained by variable-length decoding by the variable-length decoding unit 102
Is selected by the frame rate conversion control means 115, becomes Q (SEmv_nonmv (i + 2)), is further variable-length coded by the variable-length coding means 117, and is output as V (Q (SEmv_nonmv (i + 2))). . Then, the same processing as that of the (i + n + 1) -th frame is performed on the frame after (i + n + 1).

As described above, when the frame rate conversion section 123 converts the hybrid-coded image data with a high frame rate to the hybrid-coded image data with a low frame rate, In consideration of the speed and the processing capability of the terminal, the frame rate is converted in accordance with the processing speed that is the most limiting, so that the efficiency of use of the communication line does not deteriorate.

Further, by controlling an arbitrary start frame and an arbitrary number of combined frames in the hybrid-encoded image data, the hybrid encoding for orthogonally transforming the inter-frame prediction error accompanied by the motion compensation is performed. Data and intra-frame coded data,
It is possible to selectively perform frame rate conversion on only the data, greatly reduce the amount of calculation, and perform high-speed and low-load frame rate conversion. For reference, Recommendation H. Assuming that the total amount of computation in the H.261 decoder and the encoder is 100%, it is necessary to calculate the inverse orthogonal transform: about 10%, the orthogonal transform: about 10%, and the motion detection: about 45%. (Interface January 1996 issue), and it can be expected that the amount of calculation will be greatly reduced according to the present embodiment.

Embodiment 2 Hereinafter, Embodiment 2 corresponding to claim 2 of the present invention. Will be described with reference to the drawings.

FIG. 3 shows input image data having a high frame rate. From the (i + 1) th frame to the (i + n) th frame, hybrid coding is performed to perform orthogonal transformation on an inter-frame prediction error accompanied by motion compensation. This shows a frame rate conversion method when processing the (i + n) -th frame in the case of data.

In FIG. 3, reference numeral 201 denotes an input to which image data having a high frame rate is input.

Reference numeral 202 denotes a variable-length decoding unit which performs variable-length decoding of the hybrid encoded data to thereby perform prediction error orthogonal transformation data with quantized motion compensation and quantized prediction without motion compensation. The error orthogonal transform data is converted into quantized orthogonal transform coefficients. At the same time, the intra-frame / inter-frame identification set by the encoding device of the input terminal is detected, and the intra-frame / inter-frame (in) is set. Detected motion compensation presence / absence discrimination, set motion compensation presence / absence (in), and detect a quantization coefficient set by the encoding device of the input side terminal to obtain a quantization coefficient ( in), and further, the motion vector set by the encoding device of the input terminal is detected, and the motion vector (i
Set n).

Numeral 203 denotes an inverse quantization means, which uses the set quantization coefficient (in) to quantize the prediction error orthogonal transform data with motion compensation and the quantized prediction error orthogonal transform data without motion compensation. Transform data, the quantized orthogonal transform coefficients are transformed into prediction error orthogonal transform data with motion compensation,
The data is converted into prediction error orthogonal transform data without motion compensation and orthogonal transform coefficients.

Reference numeral 204 denotes a prediction error orthogonal transform data adding means, which is an orthogonal transform coefficient calculated in consideration of the motion vector by the motion vector / orthogonal transform coefficient calculating means 208, or an orthogonal transform coefficient stored in the image memory 206. The coefficient and prediction error orthogonal transform data with motion compensation or prediction error orthogonal transform data without motion compensation are added.

Reference numeral 205 denotes an orthogonal transform coefficient selecting means, which is controlled by the set intra-frame / inter-frame (in). That is, within the frame / between frames (i
When n) is “intra-frame”, the orthogonal transform coefficient inversely quantized by the inverse quantization means 203 is selected. When the intra-frame / inter-frame (in) is “inter-frame”, the prediction error orthogonal transform is performed. The orthogonal transform coefficients calculated by the data adding means 204 are selected and output.

Reference numeral 206 denotes an image memory for storing the orthogonal transform coefficients selected by the orthogonal transform coefficient selecting means 205.

Reference numeral 207 denotes a means for judging the presence / absence (in) of motion compensation.
6 is output to the motion vector / orthogonal transform coefficient calculating means 208. When the motion compensation is “none”, the orthogonal transform coefficient stored in the image memory 206 is output to the prediction error orthogonal transform data adding means 204.

Reference numeral 208 denotes a motion vector / orthogonal transform coefficient calculating means, which is stored in the image memory 206 which has been judged as "present" by the motion compensation with / without (in) judging means 207 using the motion vector (in). By performing a matrix operation on the orthogonal transform coefficients that have been set, an orthogonal transform coefficient calculated in consideration of the motion vector is calculated. For the matrix operation, a method described later is used.

Reference numeral 209 denotes an initial image capturing / selecting unit, which is an encoding control / intra-frame / inter-frame determining unit 2 described later.
In response to an image capture instruction set by 13, the orthogonal transform coefficients stored in the image memory 206 are captured, and the orthogonal transform coefficients are output to an image memory 210 described later.

Reference numeral 210 denotes an image memory for storing the orthogonal transformation coefficients selected by the initial image capture selecting means 209.

Reference numeral 211 denotes within a frame / between frames (in).
It is a judgment means.
06 orthogonal transformation coefficient is set at the subsequent stage of encoding control / in-frame /
It outputs to the inter-frame determination means 213, and in the case of “inter-frame”, outputs the orthogonal transform coefficient of the image memory 206 to the orthogonal transform coefficient subtraction means 212 described later.

Numeral 212 denotes an orthogonal transform coefficient subtracting means. The orthogonal / transform coefficient stored in the image memory 206 when the "intra-frame / inter-frame (in)" judging means 211 judges "inter-frame". The orthogonal transform coefficient in the memory 210 is subtracted.

Reference numeral 213 denotes an encoding control / intra-frame / inter-frame judging means. When frame rate conversion is not performed, motion compensation / no motion (in) is output as motion compensation / no motion (out) as it is. When frame rate conversion is being performed, motion compensation with / without (out) is set to “none” and output. Further, the frame immediately before the frame rate conversion is stored in the image memory 2.
10 to output an image capture instruction. Also, using the intra-frame / inter-frame (in), the orthogonal transform coefficient from the intra-frame / inter-frame (in) determining means 211 and the prediction error orthogonal transform data without motion compensation from the orthogonal transform coefficient subtracting means 212 are used. And outputs the result to the quantization means 214 described later. Further, the quantization coefficient (in) is reset and output to the quantization means 214 described later as the quantization coefficient (out).

Numeral 214 denotes a quantizing means which converts the orthogonal transform data and the orthogonal transform coefficient without motion compensation by the output quantized coefficient (out) into the quantized prediction error without motion compensation. The orthogonal transform data and the quantized orthogonal transform coefficient are respectively converted.

Reference numeral 215 denotes frame rate conversion control means for determining continuation / end of frame rate conversion based on the frame rate conversion information. Then, after the frame rate conversion is completed, quantized prediction error orthogonal transform data without motion compensation and quantized orthogonal transform coefficients are output to the variable length coding means 217, and the frame rate conversion is not performed. Outputs the quantized prediction error orthogonal transform data with motion compensation, the quantized prediction error orthogonal transform data without motion compensation, and the quantized orthogonal transform coefficients from the variable length decoding means 202 as they are. .

Reference numeral 216 denotes a motion vector selecting means, which is the above-mentioned coding control / intra-frame / inter-frame determining means 213.
With and without motion compensation (out) output from
Select a motion vector. That is, with motion compensation /
When “out” is “present”, the motion vector (i)
n) is output to the variable length coding means 217 as it is, and when "none", the motion vector (in) is not output to the variable length coding means 217.

Reference numeral 217 denotes a variable-length encoding unit that converts quantized prediction error orthogonal transform data with motion compensation, quantized prediction error orthogonal transform data without motion compensation, and quantized orthogonal transform coefficients. By performing variable length coding, the data is converted into hybrid coded data. At the same time, the intra-frame / inter-frame discrimination set for the decoding device of the output terminal is set according to the intra-frame / inter-frame (in), and the motion compensation (with / without (out)) and the motion vector (in) are performed. ), A motion vector with / without motion compensation is set for the decoding device of the output terminal, and a quantization coefficient is set for the decoding device of the output terminal by the quantization coefficient (in). I do.

Reference numeral 218 denotes an output from which image data having a low frame rate is output.

Hereinafter, details of the frame rate conversion processing will be described. Assuming that the input image data has no frame rate conversion from the 0th to the i-th frame, (i +
It is assumed that frames from (1) to the (i + n) -th frame are synthesized and subjected to frame rate conversion, and that frames subsequent to the (i + n + 1) -th frame have no frame rate conversion.

Variable frame decoding means 202, inverse quantization means 20
3. Prediction error orthogonal transform data adding means 204, orthogonal transform coefficient selecting means 205, motion compensation presence / absence (in) judging means 207, motion vector / orthogonal transform coefficient calculating means 20
8 and stored in the image memory 206 as S (i). Further, the data is also stored as S (i) in the image memory 210 via the initial image capture selecting means 209. Also,
The variable length decoded Q by the variable length decoding means 202
(Emv (0)) to Q (Emv (i)) are selected by the frame rate conversion control unit 215, are variable-length coded by the variable-length coding unit 217, and V (Q (E (E)
mv (0))) to V (Q (Emv (i))).

On the other hand, frame rate conversion is performed (i +
1) The frame is variable-length decoded by the variable-length decoding unit 202, and is converted into quantized prediction error orthogonal transform data Q (Emv (i + 1)) with motion compensation.

Then, Q (Emv (i + 1)) is inversely quantized by the inverse quantization means 203, and is converted into prediction error orthogonal transform data Emv (i + 1) with motion compensation.

This Emv (i + 1) is stored in the image memory 206 where the prediction error orthogonal transform data adding means 204 determines that motion compensation is present / absent (in). ) Is added to S ′ (i) processed by the motion vector / orthogonal transform coefficient calculating means 208, determined to be “between frames” by the orthogonal transform coefficient selecting means 205, and stored as S (i + 1) in the image memory 206. It is memorized.

The processes from the (i + 2) th frame to the (i + n) th frame are processed by repeating the same processing as the (i + 1) th frame.

When the processing of the (i + n) frame is completed, the orthogonal transform coefficient S is stored in the image memory 206.
(I + n) are stored, and the orthogonal transform coefficient S (i) is stored in the image memory 210.

The orthogonal transform coefficient S (i + n) is determined to be “between frames” by the intra-frame / inter-frame (in) determining means 211, and S (i + n) −S (i) by the orthogonal transform coefficient subtracting means 212. ) Is performed, and S (n)
Is calculated. Here, S (n) is prediction error orthogonal transform data without motion compensation, and assuming that data of the (i + 1) th frame is E (i + 1) = S (n).

The prediction error orthogonal transform data E (i + 1) without the motion compensation is encoded / intra-frame / inter-frame (i.
By using n), it is determined that “between frames”
Selected. With motion compensation during frame rate conversion /
None is set to "none". The image capture instruction is output after E (i + 1) is output to the quantization means 214.

E (i + 1) is quantized by the quantization means 214, and is converted into quantized prediction error orthogonal transform data Q (E (i + 1)) without motion compensation.

The quantized prediction error orthogonal transform data Q (E (i + 1)) without motion compensation is subjected to variable length encoding by the variable length encoding means 217, and the quantized and variable length encoded motion It is output as prediction error orthogonal transform data V (Q (E (i + 1))) without compensation.

On the other hand, no frame rate conversion is performed (i + n
The (+1) th frame includes a variable length decoding unit 202, an inverse quantization unit 203, a prediction error orthogonal transform data adding unit 2
04, orthogonal transform coefficient selecting means 205, motion compensation presence / absence (in) judging means 207, and motion vector / orthogonal transform coefficient calculating means 208, and are stored in image memory 206 as orthogonal transform coefficients S (i + n + 1). . Also, the image memory 210 via the initial image capture selection means 209
Is also stored as S (i + n + 1). The Q obtained by variable-length decoding by the variable-length decoding means 202 is
(Emv (i + n + 1)) is selected by the frame rate conversion control means 215, and Q (Emv (i + 2))
Then, the data is further subjected to variable-length encoding by the variable-length encoding means 217 and output as V (Q (Emv (i + 2))). Then, the same processing as that of the (i + n + 1) -th frame is performed on the frame after (i + n + 1).

As described above, according to the present embodiment, by controlling an arbitrary start frame and an arbitrary number of combined frames in hybrid-coded image data, the inter-frame prediction error with motion compensation is controlled. On the other hand, the frame rate conversion can be selectively performed only on the data, the amount of calculation can be significantly reduced, and the frame rate conversion with high speed and low load can be performed.

Embodiment 3 Hereinafter, Embodiment 3 corresponding to claim 3 of the present invention. Will be described with reference to the drawings.

FIG. 4 shows input image data having a high frame rate. The (i + 1) th to (i + n) th frames are subjected to hybrid encoding in which orthogonal transformation is performed on an inter-frame prediction error accompanied by motion compensation. The data includes both data and hybrid-coded data that performs orthogonal transform on an inter-frame prediction error without motion compensation, and the (i + 1) -th and (i + n) -th frames are:
It shows a frame rate conversion method at the time of processing the (i + n) -th frame in the case of hybrid-coded data that performs orthogonal transformation on inter-frame prediction errors without motion compensation.

In FIG. 4, reference numeral 301 denotes an input to which image data having a high frame rate is input.

Numeral 302 denotes a variable-length decoding unit which performs variable-length decoding of the hybrid coded data so as to obtain prediction error orthogonal transform data with quantized motion compensation and quantized prediction without motion compensation. The error orthogonal transform data is converted into quantized orthogonal transform coefficients. At the same time, within the frame set by the coding device of the input side terminal /
Detecting inter-frame discrimination, setting intra-frame / inter-frame (in), detecting motion-compensated / non-compensated discrimination set by the coding device of the input terminal, and performing / without motion compensation ( in), and also detects the quantization coefficient set by the encoding device of the input terminal, sets the quantization coefficient (in), and further sets the quantization coefficient (in) by the encoding device of the input terminal. The motion vector is detected and the motion vector (i
Set n).

Numeral 303 denotes an inverse quantization means, which uses the set quantization coefficient (in) to quantize the prediction error orthogonal transform data with motion compensation and the quantized prediction error orthogonal transform data without motion compensation. Transform data, the quantized orthogonal transform coefficients are transformed into prediction error orthogonal transform data with motion compensation,
The data is converted into prediction error orthogonal transform data without motion compensation and orthogonal transform coefficients.

Numeral 304 denotes a prediction error orthogonal transform data adding means, which is an orthogonal transform coefficient calculated in consideration of a motion vector by a motion vector / orthogonal transform coefficient calculating means 308, or an orthogonal transform coefficient stored in an image memory 306. The coefficient and prediction error orthogonal transform data with motion compensation or prediction error orthogonal transform data without motion compensation are added.

Reference numeral 305 denotes an orthogonal transform coefficient selection means, which is controlled by the set intra-frame / inter-frame (in). That is, within the frame / between frames (i
When n) is “intra-frame”, the orthogonal transform coefficient inversely quantized by the inverse quantization means 303 is selected. When the intra-frame / inter-frame (in) is “inter-frame”, the prediction error orthogonal The orthogonal transform coefficients calculated by the transform data adding means 304 are selected and output.

Reference numeral 306 denotes an image memory for storing the orthogonal transform coefficients selected by the orthogonal transform coefficient selecting means 305.

Reference numeral 307 denotes a means for judging whether motion compensation is performed or not (in).
6 is output to the motion vector / orthogonal transform coefficient calculating means 308. When the motion compensation is “none”, the orthogonal transform coefficient stored in the image memory 306 is output to the prediction error orthogonal transform data adding means 304.

Numeral 308 denotes a motion vector / orthogonal transform coefficient calculating means, which is stored in the image memory 306 which has been judged as "present" by the motion compensation with / without (in) judging means 307 using the motion vector (in). By performing a matrix operation on the orthogonal transform coefficients that have been set, an orthogonal transform coefficient calculated in consideration of the motion vector is calculated. For the matrix operation, a method described later is used.

Numeral 309 denotes an initial image capture selecting means, which is an encoding control / intra-frame / inter-frame determining means 3 which will be described later.
In response to an image capture instruction set by 13, the orthogonal transform coefficients stored in the image memory 306 are fetched, and the orthogonal transform coefficients are output to an image memory 310 described later.

Reference numeral 310 denotes an image memory for storing the orthogonal transform coefficients selected by the initial image capture selecting means 309.

Reference numeral 311 denotes within / between frames (in).
It is a judgment means.
06 orthogonal transformation coefficient is set at the subsequent stage of encoding control / in-frame /
It outputs to the inter-frame determination means 313, and in the case of "inter-frame", outputs the orthogonal transform coefficient of the image memory 306 to the orthogonal transform coefficient subtraction means 312 described later.

Numeral 312 denotes an orthogonal transform coefficient subtracting means. The intra-frame / inter-frame (in) judging means 311 calculates an image based on the orthogonal transform coefficient stored in the image memory 306 when it is judged "inter-frame". The orthogonal transform coefficient of the memory 310 is subtracted.

Reference numeral 313 denotes encoding control / intra-frame / inter-frame judging means. When frame rate conversion is not performed, motion compensation with / without (in) is output as it is as motion compensation with / without (out). When frame rate conversion is being performed, motion compensation with / without (out) is set to “none” and output. Further, the frame immediately before the frame rate conversion is stored in the image memory 3.
10 to output an image capture instruction. Also, using the intra-frame / inter-frame (in), the orthogonal transform coefficient from the intra-frame / inter-frame (in) determining means 311 and the prediction error orthogonal transform data without motion compensation from the orthogonal transform coefficient subtracting means 312 are used. And outputs the result to the quantization means 314 described later. Further, the quantization coefficient (in) is reset and output to the quantization means 314 described later as the quantization coefficient (out).

Numeral 314 denotes a quantizing means for converting the orthogonal transform data and the orthogonal transform coefficient without motion compensation by the output quantized coefficient (out) into the quantized prediction error without motion compensation. The orthogonal transform data and the quantized orthogonal transform coefficient are respectively converted.

Reference numeral 315 denotes a frame rate conversion control unit which determines the continuation / end of the frame rate conversion based on the frame rate conversion information. Then, after the frame rate conversion is completed, quantized prediction error orthogonal transform data without motion compensation and quantized orthogonal transform coefficients are output to the variable length coding means 317, and frame rate conversion is not performed. Outputs the quantized prediction error orthogonal transform data with motion compensation, the quantized prediction error orthogonal transform data without motion compensation, and the quantized orthogonal transform coefficients from the variable length decoding means 302 as they are. .

Reference numeral 316 denotes a motion vector selection means, which is the above-described coding control / intra-frame / inter-frame determination means 313.
With and without motion compensation (out) output from
Select a motion vector. That is, with motion compensation /
When “out” is “present”, the motion vector (i)
n) is output to the variable-length coding unit 317 as it is, and when "none", the motion vector (in) is not output to the variable-length coding unit 317.

Reference numeral 317 denotes a variable-length encoding unit which converts quantized prediction error orthogonal transform data with motion compensation, quantized prediction error orthogonal transform data without motion compensation, and quantized orthogonal transform coefficients. By performing variable length coding, the data is converted into hybrid coded data. At the same time, the intra-frame / inter-frame discrimination set for the decoding device of the output terminal is set according to the intra-frame / inter-frame (in), and the motion compensation (with / without (out)) and the motion vector (in) are performed. ), A motion vector with / without motion compensation is set for the decoding device of the output terminal, and a quantization coefficient is set for the decoding device of the output terminal by the quantization coefficient (in). .

Reference numeral 318 denotes an output from which image data having a low frame rate is output.

The details of the frame rate conversion process will be described below. Assuming that the input image data has no frame rate conversion from 0 to the i-th frame, (i +
It is assumed that frames from (1) to the (i + n) -th frame are synthesized and subjected to frame rate conversion, and that frames subsequent to the (i + n + 1) -th frame have no frame rate conversion.

The variable length decoding means 302 and the inverse quantization means 30
3. Prediction error orthogonal transform data adding means 304, orthogonal transform coefficient selecting means 305, motion compensation with / without (in) determining means 307, motion vector / orthogonal transform coefficient calculating means 30
8 and stored in the image memory 306 as S (i). Further, the image data is also stored as S (i) in the image memory 310 via the initial image capture selecting unit 309. Also,
The variable-length-decoded Q
(Emv_nonmv (0)) to Q (Emv_nonmv (0))
mv (i)) is selected by the frame rate conversion control means 315, is variable-length coded by the variable-length coding means 317, and V (Q (Emv_nonmv (0)))
Is output as V (Q (Emv_nonmv (i))).

On the other hand, frame rate conversion is performed (i +
1) The frame is subjected to variable-length decoding by the variable-length decoding unit 302, and is converted into quantized prediction error orthogonal transform data Q (E (i + 1)) without motion compensation.

Then, Q (E (i + 1)) is inversely quantized by the inverse quantization means 303 and converted into E (i + 1).

This E (i + 1) is calculated by the prediction error orthogonal transform data adding means 304 with / without motion compensation (i.
n) S ′ (i) stored in the image memory 306 determined to be “present” by the determining unit 307 is processed by the motion vector / orthogonal transform coefficient calculating unit 308, or S ′ (i) with motion compensation / S stored in the image memory 306 determined to be “none” by the absence (in) determination unit 307
(I), and the orthogonal transform coefficient selection means 305 adds
It is determined that “between frames”, and S (i
+1).

When no motion compensation is performed for the (i + 2) to (i + n) th frames, the above (i + 2)
1) The first frame when the motion compensation is performed. The processing is performed by repeating the same processing as described above.

When the processing of the (i + n) frame is completed, the orthogonal transform coefficient S is stored in the image memory 306.
(I + n) is stored, and the orthogonal transform coefficient S (i) is stored in the image memory 310.

The orthogonal transform coefficient S (i + n) is determined as “between frames” by the intra-frame / inter-frame (in) determining means 311, and the orthogonal transform coefficient subtracting means 312
The processing of S (i + n) -S (i) is performed, and S (n) is calculated. Here, S (n) is prediction error orthogonal transform data without motion compensation, and assuming that data of the (i + 1) th frame is E (i + 1) = S (n).

The prediction error orthogonal transform data E (i + 1) without the motion compensation is subjected to encoding control / intra-frame / inter-frame determination means 313 to execute the intra-frame / inter-frame (i
By using n), it is determined that “between frames”
Selected. With motion compensation during frame rate conversion /
None is set to "none". The image capture instruction is output after E (i + 1) is output to the quantization means 314.

E (i + 1) is calculated by the quantization means 314.
It is quantized and converted to quantized prediction error orthogonal transform data Q (E (i + 1)) without motion compensation.

The quantized prediction error orthogonal transform data Q (E (i + 1)) without motion compensation is subjected to variable-length coding by the variable-length coding means 317, and the quantized and variable-length coded motion It is output as prediction error orthogonal transform data V (Q (E (i + 1))) without compensation.

On the other hand, no frame rate conversion is performed (i + n
The (+1) -th frame includes a variable length decoding unit 302, an inverse quantization unit 303, and a prediction error orthogonal transformation data adding unit 3
04, orthogonal transform coefficient selection means 305, motion compensation presence / absence (in) decision means 307, and motion vector / orthogonal transform coefficient calculation means 308, and are stored in the image memory 306 as orthogonal transform coefficients S (i + n + 1). Further, the image data is also stored as S (i + n + 1) in the image memory 310 via the initial image capture selecting unit 309. Also, Q obtained by variable-length decoding by the variable-length decoding means 302 is obtained.
(Emv_nonmv (i + n + 1)) is selected by the frame rate conversion control means 315, and Q (Emv
_Nonmv (i + 2)), and further subjected to variable-length coding by the variable-length coding unit 317 to obtain V (Q (Emv
_Nonmv (i + 2))). Then, the same processing as that of the (i + n + 1) -th frame is performed on the frame after (i + n + 1).

As described above, according to the present embodiment, by controlling an arbitrary start frame and an arbitrary number of combined frames in hybrid-encoded image data, an inter-frame prediction error with motion compensation is controlled. Even in the case where both data of hybrid-coded data performing orthogonal transform on, and data of hybrid-coded data performing orthogonal transform on an inter-frame prediction error without motion compensation are included, It is possible to selectively perform frame rate conversion on only the data, greatly reduce the amount of calculation, and perform high-speed and low-load frame rate conversion.

Embodiment 4 Hereinafter, a fourth embodiment corresponding to claim 4 of the present invention. Will be described with reference to the drawings.

FIG. 5 shows the input image data having a high frame rate, in which the (i + 1) to (i + n) th frames are hybrid-coded data in which orthogonal transformation is performed on an inter-frame prediction error accompanied by motion compensation. And both intra-frame encoded data and
It shows a frame rate conversion method at the time of processing the (i + n) -th frame when the (i + n) -th frame is intra-frame encoded data.

In FIG. 5, reference numeral 401 denotes an input to which image data having a high frame rate is input.

Reference numeral 402 denotes a variable-length decoding unit which performs variable-length decoding of the hybrid coded data to obtain orthogonally transformed prediction error data with quantized motion compensation and prediction without quantized motion compensation. The error orthogonal transform data is converted into quantized orthogonal transform coefficients. At the same time, within the frame set by the coding device of the input side terminal /
Detecting inter-frame discrimination, setting intra-frame / inter-frame (in), detecting motion-compensated / non-compensated discrimination set by the coding device of the input terminal, and performing / without motion compensation ( in), and also detects the quantization coefficient set by the encoding device of the input terminal, sets the quantization coefficient (in), and further sets the quantization coefficient (in) by the encoding device of the input terminal. The motion vector is detected and the motion vector (i
Set n).

Numeral 403 denotes an inverse quantization means, which uses the set quantization coefficient (in) to quantize the prediction error orthogonal transform data with motion compensation and the quantized prediction error orthogonal transform data without motion compensation. Transform data, the quantized orthogonal transform coefficients are transformed into prediction error orthogonal transform data with motion compensation,
The data is converted into prediction error orthogonal transform data without motion compensation and orthogonal transform coefficients.

Numeral 404 denotes a prediction error orthogonal transform data adding means. The orthogonal transform coefficient calculated in consideration of the motion vector by the motion vector / orthogonal transform coefficient calculating means 408 or the orthogonal transform coefficient stored in the image memory 406. The coefficient and prediction error orthogonal transform data with motion compensation or prediction error orthogonal transform data without motion compensation are added.

Reference numeral 405 denotes an orthogonal transform coefficient selection means, which is controlled by the set intra-frame / inter-frame (in). That is, within the frame / between frames (i
When n) is “intra-frame”, the orthogonal transform coefficient inversely quantized by the inverse quantization means 403 is selected. When the intra-frame / inter-frame (in) is “inter-frame”, the prediction error orthogonal The orthogonal transform coefficients calculated by the transform data adding means 404 are selected and output.

Reference numeral 406 denotes an image memory for storing the orthogonal transform coefficients selected by the orthogonal transform coefficient selecting means 405.

Reference numeral 407 denotes a means for judging whether motion compensation is performed or not (in).
6 is output to the motion vector / orthogonal transform coefficient calculating means 408. When the motion compensation is “none”, the orthogonal transform coefficient stored in the image memory 406 is output to the prediction error orthogonal transform data adding means 404.

Reference numeral 408 denotes a motion vector / orthogonal transformation coefficient calculating means, which is stored in the image memory 406 which has been judged as "present" by the motion compensation with / without (in) judging means 407 using the motion vector (in). By performing a matrix operation on the orthogonal transform coefficients that have been set, an orthogonal transform coefficient calculated in consideration of the motion vector is calculated. For the matrix operation, a method described later is used.

Reference numeral 409 denotes an initial image capturing / selecting means, which is an encoding control / intra-frame / inter-frame determining means 4 described later.
In response to an image capture instruction set by the CPU 13, the orthogonal transform coefficients stored in the image memory 406 are fetched, and the orthogonal transform coefficients are output to an image memory 410 described later.

Reference numeral 410 denotes an image memory for storing the orthogonal transformation coefficients selected by the initial image capture selecting means 409.

Reference numeral 411 denotes within / between frames (in).
It is a judgment means.
06 orthogonal transformation coefficient is set at the subsequent stage of encoding control / in-frame /
It outputs to the inter-frame determination means 413, and in the case of “inter-frame”, outputs the orthogonal transform coefficient of the image memory 406 to the orthogonal transform coefficient subtraction means 412 described later.

Numeral 412 denotes an orthogonal transform coefficient subtracting means. The intra-frame / inter-frame (in) judging means 411 calculates an image based on the orthogonal transform coefficient stored in the image memory 406 when it is judged “inter-frame”. The orthogonal transform coefficient of the memory 410 is subtracted.

Reference numeral 413 denotes an encoding control / intra-frame / inter-frame judging means. When frame rate conversion is not performed, motion compensation with / without (in) is output as it is as motion compensation with / without (out). When frame rate conversion is being performed, motion compensation with / without (out) is set to “none” and output. Also, the frame immediately before the frame rate conversion is stored in the image memory 4.
10 to output an image capture instruction. Also, using the intra-frame / inter-frame (in), the orthogonal transform coefficient from the intra-frame / inter-frame (in) determining means 411 and the prediction error orthogonal transform data without motion compensation from the orthogonal transform coefficient subtracting means 412 are used. And outputs the result to the quantization means 414. Also, the quantization coefficient (in)
Is reset, and is output as a quantization coefficient (out) to a quantization means 414 described later.

Numeral 414 denotes a quantizing means which converts the orthogonal transform data and the orthogonal transform coefficient without motion compensation by the output quantized coefficient (out) into the quantized prediction error without motion compensation. The orthogonal transform data and the quantized orthogonal transform coefficient are respectively converted.

Reference numeral 415 denotes frame rate conversion control means for determining continuation / end of frame rate conversion based on the frame rate conversion information. Then, after the frame rate conversion is completed, quantized prediction error orthogonal transform data without motion compensation and quantized orthogonal transform coefficients are output to the variable length coding means 417, and the frame rate conversion is not performed. Outputs the quantized prediction error orthogonal transform data with motion compensation, the quantized prediction error orthogonal transform data without motion compensation, and the quantized orthogonal transform coefficients from the variable length decoding means 402 as they are. .

Reference numeral 416 denotes a motion vector selecting means, which is the above-described coding control / intra-frame / inter-frame determining means 413.
With and without motion compensation (out) output from
Select a motion vector. That is, with motion compensation /
When “out” is “present”, the motion vector (i)
n) is output to the variable length coding unit 417 as it is, and when “none”, the motion vector (in) is not output to the variable length coding unit 417.

Reference numeral 417 denotes a variable-length encoding unit that converts quantized prediction error orthogonal transform data with motion compensation, quantized prediction error orthogonal transform data without motion compensation, and quantized orthogonal transform coefficients. By performing variable length coding, the data is converted into hybrid coded data. At the same time, the intra-frame / inter-frame discrimination set for the decoding device of the output terminal is set according to the intra-frame / inter-frame (in), and the motion compensation (with / without (out)) and the motion vector (in) are performed. ), A motion vector with / without motion compensation is set for the decoding device of the output terminal, and a quantization coefficient is set for the decoding device of the output terminal by the quantization coefficient (in). .

Reference numeral 418 denotes an output from which image data having a low frame rate is output.

Hereinafter, details of the frame rate conversion processing will be described. Assuming that the input image data has no frame rate conversion from 0 to the i-th frame, (i +
It is assumed that frames from (1) to the (i + n) -th frame are synthesized and subjected to frame rate conversion, and that frames subsequent to the (i + n + 1) -th frame have no frame rate conversion.

The variable length decoding means 402 and the inverse quantization means 40
3, prediction error orthogonal transform data adding means 404, orthogonal transform coefficient selecting means 405, motion compensation presence / absence (in) judging means 407, motion vector / orthogonal transform coefficient calculating means 40
8 and stored in the image memory 406 as S (i). Further, the image data is also stored as S (i) in the image memory 410 via the initial image capture selecting unit 409. Also,
The variable length decoded Q by the variable length decoding means 402
(SEmv (0)) to Q (SEmv (i)) are selected by the frame rate conversion control means 415, are variable-length coded by the variable-length coding means 417, and V (Q
(SEmv (0))) to V (Q (SEmv (i)))
Is output as

On the other hand, frame rate conversion is performed (i +
1) The frame is subjected to variable-length decoding by the variable-length decoding unit 402, and the quantized orthogonal transform coefficient Q (S (i +
1)).

Then, Q (S (i + 1)) is inversely quantized by the inverse quantization means 403 and converted to S (i + 1).

This orthogonal transform coefficient selection means 405 determines that S (i + 1) is "in a frame", and stores it as S (i + 1) in the image memory 406.

From the (i + 2) th frame to the (i + n) th frame, the (i + 1) th frame in the case of a frame, and the second embodiment in the case of involving motion compensation.
The processing is performed by repeating the same processing as described above.

When the processing of the (i + n) frame is completed, the orthogonal transform coefficient S is stored in the image memory 406.
(I + n) are stored, and the orthogonal transform coefficient S (i) is stored in the image memory 410.

The orthogonal transformation coefficient S (i + n) is determined to be “in frame” by the intra-frame / inter-frame (in) determination means 411, and S (i + n) is output. here,
Let S (i + n) be the new (i + 1) th frame,
S (i + 1) = S (i + n).

The above-mentioned orthogonal transform coefficient S (i + 1) is obtained by using the intra-frame / inter-frame (in) in the encoding control / intra-frame / inter-frame determining means 413.
It is determined to be "in a frame" and is selected. During frame rate conversion, motion compensation with / without (out) is “none”
Is set to The image capture instruction is sent to the quantization means 414 as S.
After (i + 1) is output, it is output.

S (i + 1) is calculated by the quantization means 414.
The quantized orthogonal transform coefficient Q (S (i +
1)).

The quantized orthogonal transform coefficient Q (S
(I + 1)) is subjected to variable-length encoding by the variable-length encoding unit 417, and is output as V (Q (S (i + 1))).

On the other hand, no frame rate conversion is performed (i + n
The (+1) th frame includes a variable length decoding unit 402, an inverse quantization unit 403, and a prediction error orthogonal transform data addition unit 4
04, orthogonal transform coefficient selection means 405, motion compensation presence / absence (in) determination means 407, and motion vector / orthogonal transform coefficient calculation means 408, and are stored in the image memory 406 as orthogonal transform coefficients S (i + n + 1). Further, it is also stored as S (i + n + 1) in the image memory 410 via the initial image capture selecting means 409. The Q obtained by variable-length decoding by the variable-length decoding unit 402
(SEmv (i + n + 1)) is selected by the frame rate conversion control unit 415, and Q (SEmv (i + n + 1))
2)), and further subjected to variable-length encoding by the variable-length encoding unit 417, and output as V (Q (SEmv (i + 2))). Then, the same processing as that of the (i + n + 1) -th frame is performed on the frame after (i + n + 1).

As described above, according to the present embodiment, by controlling an arbitrary start frame and an arbitrary number of combined frames in hybrid-coded image data, the inter-frame prediction error with motion compensation is controlled. Even when both the hybrid-coded data and the intra-frame coded data that perform orthogonal transform are included, it is possible to selectively perform frame rate conversion on only the data. The amount is greatly reduced, and a high-speed and low-load frame rate conversion can be performed.

Embodiment 5 FIG. Hereinafter, a fifth embodiment corresponding to claim 5 of the present invention. Will be described with reference to the drawings.

FIG. 6 shows input image data having a high frame rate, in which the (i + 1) to (i + n) th frames are hybrid-coded data in which orthogonal transformation is performed on inter-frame prediction errors with motion compensation. And three types of data, that is, hybrid-coded data that performs orthogonal transform on an inter-frame prediction error without motion compensation, and further, intra-frame coded data,
The (i + 1) -th frame is intra-coded data, and the (i + n-1) -th frame is a hybrid-coded data intra code that performs orthogonal transformation on an inter-frame prediction error accompanied by motion compensation. (I + n) -th frame when the (i + n) -th frame is hybrid-coded data that performs orthogonal transform on an inter-frame prediction error without motion compensation. It shows a frame rate conversion method.

In FIG. 6, reference numeral 501 denotes an input to which image data having a high frame rate is input.

Reference numeral 502 denotes a variable-length decoding unit that performs variable-length decoding of the hybrid encoded data to thereby perform prediction error orthogonal transformation data with quantized motion compensation and quantized prediction without motion compensation. The error orthogonal transform data is converted into quantized orthogonal transform coefficients. At the same time, within the frame set by the coding device of the input side terminal /
Detecting inter-frame discrimination, setting intra-frame / inter-frame (in), detecting motion-compensated / non-compensated discrimination set by the coding device of the input terminal, and performing / without motion compensation ( in), and also detects the quantization coefficient set by the encoding device of the input terminal, sets the quantization coefficient (in), and further sets the quantization coefficient (in) by the encoding device of the input terminal. The motion vector is detected and the motion vector (i
Set n).

Reference numeral 503 denotes an inverse quantization means, which uses the set quantization coefficient (in) to quantize the prediction error orthogonal transform data with motion compensation and the quantized prediction error orthogonal transform data without motion compensation. Transform data, the quantized orthogonal transform coefficients are transformed into prediction error orthogonal transform data with motion compensation,
The data is converted into prediction error orthogonal transform data without motion compensation and orthogonal transform coefficients.

Reference numeral 504 denotes a prediction error orthogonal transform data adding means, which is an orthogonal transform coefficient calculated in consideration of the motion vector by the motion vector / orthogonal transform coefficient calculating means 508, or an orthogonal transform coefficient stored in the image memory 506. The coefficient and prediction error orthogonal transform data with motion compensation or prediction error orthogonal transform data without motion compensation are added.

Reference numeral 505 denotes orthogonal transform coefficient selection means, which is controlled by the set intra-frame / inter-frame (in). That is, within the frame / between frames (i
When n) is “intra-frame”, the orthogonal transform coefficient inversely quantized by the inverse quantization means 503 is selected. When the intra-frame / inter-frame (in) is “inter-frame”, the prediction error orthogonal The orthogonal transform coefficients calculated by the transform data adding means 504 are selected and output.

Reference numeral 506 denotes an image memory for storing the orthogonal transform coefficients selected by the orthogonal transform coefficient selecting means 505.

Reference numeral 507 denotes a means for judging the presence / absence (in) of motion compensation.
6 is output to the motion vector / orthogonal transform coefficient calculating means 508. When the motion compensation is “none”, the orthogonal transform coefficient stored in the image memory 506 is output to the prediction error orthogonal transform data adding means 504.

Reference numeral 508 denotes a motion vector / orthogonal transform coefficient calculating means, which is stored in the image memory 506 determined as "present" by the motion compensation with / without (in) determining means 507 using the motion vector (in). By performing a matrix operation on the orthogonal transform coefficients that have been set, an orthogonal transform coefficient calculated in consideration of the motion vector is calculated. For the matrix operation, a method described later is used.

Reference numeral 509 denotes an initial image capturing / selecting means, which is an encoding control / intra-frame / inter-frame determining means 5 described later.
In response to an image capture instruction set by 13, the orthogonal transform coefficients stored in the image memory 506 are fetched, and the orthogonal transform coefficients are output to an image memory 510 described later.

Reference numeral 510 denotes an image memory for storing the orthogonal transformation coefficient selected by the initial image capture selecting means 509.

Reference numeral 511 denotes within / between frames (in).
It is a judgment means.
06 orthogonal transformation coefficient is set at the subsequent stage of encoding control / in-frame /
It outputs to the inter-frame determination means 513, and in the case of "inter-frame", outputs the orthogonal transform coefficient of the image memory 506 to the orthogonal transform coefficient subtraction means 512 described later.

Numeral 512 denotes an orthogonal transform coefficient subtracting means. The intra-frame / inter-frame (in) judging means 511 calculates an image based on the orthogonal transform coefficient stored in the image memory 506 when "inter-frame" is judged. The orthogonal transform coefficient of the memory 510 is subtracted.

Reference numeral 513 denotes encoding control / intra-frame / inter-frame determination means. When frame rate conversion is not performed, motion compensation with / without (in) is output as it is as motion compensation with / without (out). When frame rate conversion is being performed, motion compensation with / without (out) is set to “none” and output. Further, the frame immediately before the frame rate conversion is stored in the image memory 5.
10 to output an image capture instruction. Also, using the intra-frame / inter-frame (in), the orthogonal transform coefficient from the intra-frame / inter-frame (in) determining means 511 and the prediction error orthogonal transform data without motion compensation from the orthogonal transform coefficient subtracting means 512 are used. And outputs the result to the quantization means 514. Also, the quantization coefficient (in)
Is reset, and the result is output to a quantization unit 514 described later as a quantization coefficient (out).

Numeral 514 denotes a quantizing means which converts the orthogonal transformation data and the orthogonal transformation coefficient without motion compensation by the output quantized coefficient (out) into the quantized prediction error without motion compensation. The orthogonal transform data and the quantized orthogonal transform coefficient are respectively converted.

Reference numeral 515 denotes frame rate conversion control means for determining continuation / end of the frame rate conversion based on the frame rate conversion information. Then, after the frame rate conversion is completed, quantized prediction error orthogonal transform data without motion compensation and quantized orthogonal transform coefficients are output to the variable length coding means 517, and the frame rate conversion is not performed. Outputs the quantized prediction error orthogonal transform data with motion compensation, the quantized prediction error orthogonal transform data without motion compensation, and the quantized orthogonal transform coefficients from the variable length decoding means 502 as they are. .

Reference numeral 516 denotes a motion vector selection means, which is the above-described coding control / intra-frame / inter-frame determination means 513.
With and without motion compensation (out) output from
Select a motion vector. That is, with motion compensation /
When “out” is “present”, the motion vector (i)
n) is output to the variable length coding unit 517 as it is, and when "none", the motion vector (in) is not output to the variable length coding unit 517.

Reference numeral 517 denotes a variable-length encoding unit that converts the quantized prediction error orthogonal transform data with motion compensation, the quantized prediction error orthogonal transform data without motion compensation, and the quantized orthogonal transform coefficient. By performing variable length coding, the data is converted into hybrid coded data. At the same time, the intra-frame / inter-frame discrimination set for the decoding device of the output terminal is set based on the intra-frame / inter-frame (in), with / without motion compensation (out), and with the motion vector (in). ), A motion vector with / without motion compensation is set for the decoding device of the output terminal, and a motion vector is set, and a quantization coefficient is set for the decoding device of the output terminal by the quantization coefficient (in). .

Reference numeral 518 denotes an output from which image data having a low frame rate is output.

Hereinafter, details of the frame rate conversion processing will be described. Assuming that the input image data has no frame rate conversion from 0 to the i-th frame, (i +
It is assumed that frames from (1) to the (i + n) -th frame are synthesized and subjected to frame rate conversion, and that frames subsequent to the (i + n + 1) -th frame have no frame rate conversion.

The variable length decoding means 502 and the inverse quantization means 50
3, prediction error orthogonal transform data adding means 504, orthogonal transform coefficient selecting means 505, with / without motion compensation (in) determining means 507, motion vector / orthogonal transform coefficient calculating means 50
8 and stored in the image memory 506 as S (i). The image data is also stored as S (i) in the image memory 510 via the initial image capture selecting unit 509. Also,
The variable length decoded Q by the variable length decoding means 502
(SEmv_nonmv (0)) to Q (SEmv_n
onmv (i)) is the frame rate conversion control means 51
5 and is variable-length coded by the variable-length coding unit 517, and V (Q (SEmv_nonmv
(0))) to V (Q (SEmv_nonmv
(I)) is output as).

On the other hand, frame rate conversion is performed (i +
1) The frame is subjected to variable-length decoding by the variable-length decoding unit 502 and the quantized orthogonal transform coefficient Q (S (i +
1)).

Then, Q (S (i + 1)) is inversely quantized by the inverse quantization means 503 and is converted into S (i + 1).

This S (i + 1) is judged to be “within a frame” by the orthogonal transform coefficient selection means 505 and stored in the image memory 506 as S (i + 1).

From the (i + 2) th frame to the (i + n) th frame, the (i + 1) th frame is within the frame, and the motion compensation is performed or the motion compensation is not performed. The processing is performed by repeating the same processing as described above.

When the processing of the (i + n) frame is completed, the orthogonal transform coefficient S is stored in the image memory 506.
(I + n) are stored, and the image memory 510 stores the orthogonal transform coefficient S (i).

The orthogonal transform coefficient S (i + n) is determined to be “between frames” by the intra-frame / inter-frame (in) determining means 511, and the orthogonal transform coefficient subtracting means 512 determines
The processing of S (i + n) -S (i) is performed, and S (n) is calculated. Here, S (n) is prediction error orthogonal transform data without motion compensation, and assuming that data of the (i + 1) th frame is E (i + 1) = S (n).

The prediction error orthogonal transform data E (i + 1) without the motion compensation is subjected to encoding control / intra-frame / inter-frame determination means 513 to generate the intra-frame / inter-frame (i
By using n), it is determined that “between frames”
Selected. With motion compensation during frame rate conversion /
None is set to "none". The image capture instruction is output after E (i + 1) is output to the quantization means 514.

E (i + 1) is calculated by the quantization means 514.
It is quantized and converted to quantized prediction error orthogonal transform data Q (E (i + 1)) without motion compensation.

The quantized prediction error orthogonal transform data Q (E (i + 1)) without motion compensation is variable-length coded by variable-length coding means 517, and V (Q (E (i (i
+1))).

On the other hand, no frame rate conversion is performed (i + n
The (+1) th frame includes a variable-length decoding unit 502, an inverse quantization unit 503, and a prediction error orthogonal transformation data addition unit 5
04, orthogonal transformation coefficient selection means 505, motion compensation presence / absence (in) judgment means 507, and motion vector / orthogonal transformation coefficient calculation means 508, and are stored in the image memory 506 as orthogonal transformation coefficients S (i + n + 1). Further, the image data is stored in the image memory 510 via the initial image capture selecting unit 509.
Is also stored as S (i + n + 1). Also, Q obtained by variable length decoding by the variable length decoding
(SEmv_nonmv (i + n + 1)) is selected by the frame rate conversion control unit 515, and Q (SE
mv_nonmv (i + 2)), and is further subjected to variable-length coding by the variable-length coding unit 517 to obtain V (Q (S
Emv_nonmv (i + 2))).
Then, the same processing as that of the (i + n + 1) -th frame is performed on the frame after (i + n + 1).

As described above, according to the present embodiment, by controlling an arbitrary start frame and an arbitrary number of combined frames in hybrid-encoded image data, an inter-frame prediction error accompanied with motion compensation is controlled. , Hybrid-coded data that performs orthogonal transformation on inter-frame prediction errors without motion compensation, and intra-coded data. Even when data is included, it is possible to selectively perform frame rate conversion on only the data, greatly reduce the amount of calculation, and perform high-speed and low-load frame rate conversion.

Embodiment 6 FIG. Hereinafter, a sixth embodiment corresponding to claim 6 of the present invention. With regard to the frame rate conversion method according to the above, the configuration of the frame rate conversion method will be described with reference to the drawings.

FIG. 7 shows the structure of the frame rate conversion method according to the first embodiment.

In FIG. 7, reference numeral 621 denotes a terminal, and its processing capability is c1. Reference numeral 622 denotes a communication line connected downstream of the terminal 621, and its transmission speed is assumed to be b1. Reference numeral 623 denotes a frame rate conversion unit connected downstream of the communication line 622. The frame rate conversion information is input from the terminal 625, and the transmission rate b of the communication line 622.
1 and the frame rate conversion taking into account the transmission speed b2 of the communication line 624 and the processing capability c2 of the terminal 625, respectively. 624 is a communication line,
The transmission speed is b2. A terminal 625 has a processing capability of c2.

Hereinafter, image data having a high frame rate is input from the terminal 621, and the transmission speed of the communication line is set to b1
≧ b2 and the processing capability of the terminal is c1 =
The frame rate conversion method when the relationship is c2 will be described with reference to FIG.

Here, since c1 = c2, the processing capabilities of the terminals are equal, and it is not necessary to consider the conversion of the frame rate according to this. Therefore, attention is paid to the relationship between the transmission speeds b1 and b2 between the communication lines.

Since the transmission speed b2 of the communication line 624 is equal to or lower than the transmission speed b1 of the communication line 622, the frame rate conversion unit 623 only needs to convert the data into a frame rate that can be transmitted at the transmission speed b2 of the communication line 624. become. Therefore, the hybrid-coded image data with a high frame rate input from the terminal 621 is subjected to frame rate conversion into hybrid-coded image data with a low frame rate, which has the performance of the transmission rate b2, and the terminal 625 Output to the side. At the terminal 625, image data having a frame rate equal to the transmission speed b2 of the communication line 624 is transmitted, and is completely reproduced and displayed within the processing capability range of the terminal 625.

Next, image data having a high frame rate is input from the terminal 621, and the transmission speed of the communication line is changed to b1.
≧ b2 and the processing capability of the terminal is c1>
The frame rate conversion method when the relationship is c2 will be described with reference to FIG.

Processing capacities c1 and c2 of terminals 621 and 625
Paying attention to the relationship, since the processing capability c2 of the terminal 625 is lower than the processing capability c1 of the terminal 621, the frame rate may be converted to a frame rate that can be processed by the processing capability c2 of the terminal 625. Focusing on the relationship between the transmission speeds b1 and b2 of the communication lines 622 and 624, since the transmission speed b2 of the communication line 624 is equal to or lower than the transmission speed b1 of the communication line 622, transmission is possible at the transmission speed b2 of the communication line 624. What is necessary is to convert it to a frame rate.

Here, the transmission speed b of the communication line 624 is larger than the number of frames that can be processed by the processing capability c2 of the terminal 625.
If the number of frames that can be transmitted by the terminal 2 is larger, the terminal 625
The terminal 625 cannot decode even if the number of frames that can be processed with the processing capability c2 of
The use efficiency of the communication line becomes worse. In order to avoid this, the hybrid-encoded image data having a high frame rate input from the terminal 621 side is a hybrid-encoded image data having a low frame rate which is in the performance of the processing capability c2 of the terminal 625. Is converted to a frame rate and output to the terminal 625 side.

The transmission rate b2 of the communication line 624 is larger than the number of frames that can be processed by the processing capacity c2 of the terminal 625.
If the number of frames that can be transmitted by the
Since only the number of frames that can be transmitted at the transmission rate b2 of 24 can be transmitted, the hybrid-coded image data with a high frame rate input from the terminal 621 does not meet the performance of the transmission rate b2 of the communication line 624. Was
The frame rate is converted to hybrid-coded image data having a low frame rate and output to the terminal 625 side.

Furthermore, the processing capability c2 of the terminal 625
Number of frames that can be processed by the system and the transmission speed b of the communication line 624
2 and the number of frames that can be transmitted is equal, the hybrid-coded image data with a high frame rate input from the terminal 621 side is transmitted at the transmission speed b of the communication line 624.
2, or is subjected to frame rate conversion to hybrid-encoded image data having a low frame rate that matches the performance of the processing capability c2 of the terminal 625, and is output to the terminal 625 side.

Next, image data having a high frame rate is input from the terminal 621, and the transmission speed of the communication line is set to b.
1 <b2 and the processing capability of the terminal is c1
The frame rate conversion method when the relationship of> c2 is satisfied will be described with reference to FIG.

Processing capabilities c1 and c2 of terminals 621 and 625
Paying attention to the relationship, since the processing capability c2 of the terminal 625 is lower than the processing capability c1 of the terminal 621, the frame rate may be converted to a frame rate that can be processed by the processing capability c2 of the terminal 625. Focusing on the relationship between the transmission speeds b1 and b2 of the communication lines 622 and 624, since the transmission speed b1 of the communication line 622 is lower than the transmission speed b2 of the communication line 624, a frame that can be transmitted at the transmission speed b1 of the communication line 622 is used. It just has to be converted to a rate.

Here, the transmission rate b of the communication line 622 is larger than the number of frames that can be processed by the processing capacity c2 of the terminal 625.
If the number of frames that can be transmitted by 1 is larger, the terminal 625
The terminal 625 cannot decode even if the number of frames that can be processed with the processing capability c2 of
The use efficiency of the communication line becomes worse. In order to avoid this, the hybrid-encoded image data having a high frame rate input from the terminal 621 side is a hybrid-encoded image data having a low frame rate which is in the performance of the processing capability c2 of the terminal 625. Is converted to a frame rate and output to the terminal 625 side.

The transmission rate b1 of the communication line 622 is larger than the number of frames that can be processed by the processing capacity c2 of the terminal 625.
If the number of frames that can be transmitted by the
Since only the number of frames that can be transmitted at the transmission speed b1 of 22 can be transmitted, the hybrid-coded image data with a high frame rate input from the terminal 621 does not meet the performance of the transmission speed b1 of the communication line 622. Was
The frame rate is converted to hybrid-coded image data having a low frame rate and output to the terminal 625 side.

Furthermore, the processing capability c2 of the terminal 625
Number of frames that can be processed by the transmission speed b of the communication line 622
1 and the number of transmittable frames is equal, the hybrid-coded image data with a high frame rate input from the terminal 621 side is transmitted at the transmission speed b of the communication line 622.
1 or the image data subjected to the hybrid encoding with a low frame rate, which has the performance of the processing capability c2 of the terminal 625, is converted to the frame rate and output to the terminal 625 side.

Next, image data having a high frame rate is input from the terminal 621, and the transmission speed of the communication line is b1>
b2 and the processing capability of the terminal is c1 <c
The frame rate conversion method in the case of the relationship 2 will be described with reference to FIG.

The processing capacities c1 and c2 of the terminals 621 and 625
Paying attention to the relationship, since the processing capability c1 of the terminal 621 is lower than the processing capability c2 of the terminal 625, the frame rate that can be processed by the processing capability c1 of the terminal 621 may be used as it is. In addition, communication lines 622 and 624
Focusing on the relationship between the transmission speeds b1 and b2 of the communication line 6,
Since the transmission speed b2 of the communication line 624 is lower than the transmission speed b1 of the communication line 622, the frame rate can be converted to a frame rate that can be transmitted at the transmission speed b2 of the communication line 624.

Here, the transmission speed b of the communication line 624 is larger than the number of frames that can be processed by the processing capability c1 of the terminal 621.
When the number of frames that can be transmitted by the terminal 2 is larger, the terminal 621
A frame exceeding the number of frames that can be processed by the processing capability c1 cannot be transmitted even if the transmission speed b2 of the communication line 624 is used. Therefore, the hybrid-coded image data input from the terminal 621 is directly output to the terminal 625 without being subjected to frame rate conversion.

The transmission rate b2 of the communication line 624 is larger than the number of frames that can be processed by the processing capacity c1 of the terminal 621.
If the number of frames that can be transmitted by the
Since only the number of frames that can be transmitted at the transmission rate b2 of 24 can be transmitted, the hybrid-coded image data with a high frame rate input from the terminal 621 does not meet the performance of the transmission rate b2 of the communication line 624. Was
The frame rate is converted to hybrid-coded image data having a low frame rate and output to the terminal 625 side.

Furthermore, the processing capability c1 of the terminal 621
Number of frames that can be processed by the system and the transmission speed b of the communication line 624
2 and the number of frames that can be transmitted is equal, the terminal 621
The hybrid-coded image data input from the terminal 625 is not subjected to frame rate conversion and is
Output to the side.

FIG. 8 is a diagram showing a detailed configuration of the frame rate conversion section shown in FIG. 7. In the second to fifth embodiments, (i + 1) The frame rate conversion method at the time of processing the (i + n) -th frame when the data from (i + n) to the (i + n) -th frame is hybrid-coded data that performs orthogonal transformation on an inter-frame prediction error accompanied by motion compensation. It is shown.

In FIG. 8, reference numeral 601 denotes an input to which image data having a high frame rate is input.

Reference numeral 602 denotes a variable-length decoding unit which performs variable-length decoding of the hybrid coded data to obtain quantized prediction error orthogonal transform data with motion compensation and quantized prediction without motion compensation. The error orthogonal transform data is converted into quantized orthogonal transform coefficients. At the same time, within the frame set by the coding device of the input side terminal /
Detecting inter-frame discrimination, setting intra-frame / inter-frame (in), detecting motion-compensated / non-compensated discrimination set by the coding device of the input terminal, and performing / without motion compensation ( in), and also detects the quantization coefficient set by the encoding device of the input terminal, sets the quantization coefficient (in), and further sets the quantization coefficient (in) by the encoding device of the input terminal. The motion vector is detected and the motion vector (i
Set n).

Numeral 603 denotes an inverse quantization means, which uses the set quantization coefficient (in) to quantize the prediction error orthogonal transform data with motion compensation and the quantized prediction error orthogonal transform data without motion compensation. Transform data, the quantized orthogonal transform coefficients are transformed into prediction error orthogonal transform data with motion compensation,
Prediction error orthogonal transformation data without motion compensation is transformed into orthogonal transformation coefficients.

Numeral 604 denotes a prediction error orthogonal transform data adding means. The orthogonal transform coefficient calculated by the motion vector / orthogonal transform coefficient calculating means 608 in consideration of the motion vector or the orthogonal transform coefficient stored in the image memory 606. The coefficient and the prediction error orthogonal transform data with motion compensation or the prediction error orthogonal transform data without motion compensation are added.

Reference numeral 605 denotes an orthogonal transform coefficient selection means, which is controlled by the set intra-frame / inter-frame (in). That is, within the frame / between frames (i
When n) is “intra-frame”, the orthogonal transform coefficient inversely quantized by the inverse quantization means 603 is selected, and when the intra-frame / inter-frame (in) is “inter-frame”, the prediction error orthogonal The orthogonal transform coefficients calculated by the transform data adding means 604 are selected and output.

An image memory 606 stores the orthogonal transform coefficients selected by the orthogonal transform coefficient selecting means 605.

Reference numeral 607 denotes a means for judging whether or not there is motion compensation (in).
6 is output to the motion vector / orthogonal transform coefficient calculating means 608. When the motion compensation is “none”, the orthogonal transform coefficient stored in the image memory 606 is output to the prediction error orthogonal transform data adding means 604.

Numeral 608 denotes a motion vector / orthogonal transform coefficient calculating means, which uses the motion vector (in) to determine the presence / absence (in) of the motion compensation. By performing a matrix operation on the orthogonal transform coefficients stored in the matrix, the orthogonal transform coefficients calculated in consideration of the motion vector are calculated. For the matrix operation, a method described later is used.

Reference numeral 609 denotes an initial image capturing / selecting means, which is a coding control / intra-frame / inter-frame determining means 6 described later.
In response to an image capture instruction set by 13, the orthogonal transform coefficients stored in the image memory 606 are fetched, and the orthogonal transform coefficients are output to an image memory 610 described later.

Reference numeral 610 denotes an image memory for storing the orthogonal transform coefficients selected by the initial image capture selecting means 609.

Reference numeral 611 denotes an intra-frame / inter-frame (in).
It is a judgment means.
06 orthogonal transformation coefficient is set at the subsequent stage of encoding control / in-frame /
It outputs to the inter-frame determination means 613, and in the case of “inter-frame”, outputs the orthogonal transform coefficient of the image memory 606 to the orthogonal transform coefficient subtraction means 612 described later.

Numeral 612 denotes an orthogonal transform coefficient subtracting means. The intra-frame / inter-frame (in) judging means 611 uses the orthogonal transform coefficient stored in the image memory 606 determined as “inter-frame” to store the image memory 610. Subtract the orthogonal transform coefficients.

Reference numeral 613 denotes encoding control / intra-frame / inter-frame judging means. When frame rate conversion is not performed, motion compensation with / without (in) is output as motion compensation with / without (out) as it is. When frame rate conversion is being performed, motion compensation with / without (out) is set to “none” and output. Further, the frame immediately before the frame rate conversion is stored in the image memory 6.
10 to output an image capture instruction. Also, using the intra-frame / inter-frame (in), the orthogonal transform coefficient from the intra-frame / inter-frame (in) determining means 611 and the prediction error orthogonal transform data without motion compensation from the orthogonal transform coefficient subtracting means 612 are used. And outputs the result to a quantization unit 614 described later. Also, the quantization coefficient (in) is reset and output to the quantization means 614 described later as the quantization coefficient (out).

Numeral 614 denotes a quantizing means for converting the orthogonal transform data and the orthogonal transform coefficient without motion compensation by the output quantized coefficient (out) into the quantized prediction error without motion compensation. The orthogonal transform data and the quantized orthogonal transform coefficient are respectively converted.

Reference numeral 615 denotes frame rate conversion control means, which determines the continuation / end of the frame rate conversion based on the frame rate conversion information. Then, after the frame rate conversion is completed, quantized prediction error orthogonal transform data without motion compensation and quantized orthogonal transform coefficients are output to the variable-length coding unit 617, and when frame rate conversion is not performed, , The quantized prediction error orthogonal transform data with motion compensation, the quantized prediction error orthogonal transform data without motion compensation, and the quantized orthogonal transform coefficients from the variable length decoding means 602 are output as they are.

Reference numeral 616 denotes a motion vector selection means, which is the above-described coding control / intra-frame / inter-frame determination means 613.
A motion vector is selected based on the presence / absence (out) of the motion compensation output from. That is, when “with / without motion compensation” (out) is “present”, the motion vector (in)
Is output as it is to the variable length encoding means 617 described later,
When “none”, the motion vector (in) is not output to the variable length coding unit 617.

Reference numeral 617 denotes a variable-length encoding means for transforming prediction error orthogonal transform data with quantized motion compensation, prediction error orthogonal transform data without quantized motion compensation, and quantized orthogonal transform coefficients. By performing variable length coding, the data is converted into hybrid coded data. At the same time, the intra-frame / inter-frame discrimination set for the decoding device of the output terminal is set according to the intra-frame / inter-frame (in), and the motion compensation (with / without (out)) and the motion vector (in) are performed. ), A motion vector with / without motion compensation is set for the decoding device of the output terminal, and a quantization coefficient is set for the decoding device of the output terminal by the quantization coefficient (in). .

Reference numeral 618 denotes an output from which image data having a low frame rate is output.

Hereinafter, details of the frame rate conversion processing will be described. Assuming that the input image data has no frame rate conversion from the 0th to the i-th frame, (i +
It is assumed that frames from (1) to the (i + n) -th frame are synthesized and subjected to frame rate conversion, and that frames subsequent to the (i + n + 1) -th frame have no frame rate conversion.

Variable frame decoding means 602, inverse quantization means 60
3, prediction error orthogonal transform data adding means 604, orthogonal transform coefficient selecting means 605, motion compensation presence / absence (in) determining means 607, motion vector / orthogonal transform coefficient calculating means 60
8 and stored in the image memory 606 as S (i). Further, the data is also stored as S (i) in the image memory 610 via the initial image capture selecting unit 609. Also,
The variable length decoded Q by the variable length decoding means 602
(Emv (0)) to Q (Emv (i)) are selected by the frame rate conversion control means 615, are variable-length coded by the variable-length coding means 617, and V (Q (E (E)
mv (0))) to V (Q (Emv (i))).

On the other hand, frame rate conversion is performed (i +
1) The frame is variable-length decoded by the variable-length decoding unit 602, and is converted into quantized prediction error orthogonal transform data Q (Emv (i + 1)) with motion compensation.

Then, Q (Emv (i + 1)) is inversely quantized by the inverse quantization means 603, and is transformed into prediction error orthogonal transform data Emv (i + 1) with motion compensation.

This Emv (i + 1) is stored in the image memory 606 which is judged as “present” by the motion compensation presence / absence (in) decision means 607 by the prediction error orthogonal transform data adding means 604. ) Is added to S ′ (i) processed by the motion vector / orthogonal transform coefficient calculating means 608, determined to be “between frames” by the orthogonal transform coefficient selecting means 605, and stored as S (i + 1) in the image memory 606. Is done.

The (i + 2) to (i + n) th frames are processed by repeating the same processing as the (i + 1) th frame.

When the processing of the (i + n) frame is completed, the orthogonal transform coefficient S (i + n) is stored in the image memory 606, and the orthogonal transform coefficient S (i) is stored in the image memory 610. It is remembered.

The orthogonal transform coefficient S (i + n) is determined as “between frames” by the intra-frame / inter-frame (in) determining means 611, and the orthogonal transform coefficient subtracting means 612 determines S (i + n) −S (i). ) Is performed, and S (n)
Is calculated. Here, S (n) is prediction error orthogonal transform data without motion compensation, and assuming that data of the (i + 1) th frame is E (i + 1) = S (n).

The prediction error orthogonal transform data E (i + 1) without the motion compensation is subjected to encoding control / intra-frame / inter-frame determination means 613 to determine the intra-frame / inter-frame (i
By using n), it is determined that “between frames”
Selected. With motion compensation during frame rate conversion /
None is set to "none". The image capture instruction is output after E (i + 1) is output to the quantization means 614.

E (i + 1) is quantized by the quantization means 614, and is transformed into quantized prediction error orthogonal transform data Q (E (i + 1)) without motion compensation.

The quantized prediction error orthogonal transform data Q (E (i + 1)) without motion compensation is subjected to variable length coding by the variable length coding means 617, and the quantized and variable length coded motion It is output as prediction error orthogonal transform data V (Q (E (i + 1))) without compensation.

On the other hand, no frame rate conversion is performed (i + n
The (+1) th frame is a variable length decoding means 602, an inverse quantization means 603, a prediction error orthogonal transform data adding means 6
04, orthogonal transform coefficient selection means 605, motion compensation presence / absence (in) determination means 607, motion vector / orthogonal transform coefficient calculation means 608, and are stored in the image memory 606 as orthogonal transform coefficients S (i + n + 1). You. Also,
Via the initial image capture selection means 609, the image memory 6
10 is also stored as S (i + n + 1). Further, Q (Emv (i + n + 1)) obtained by performing variable length decoding by the variable length decoding unit 602 is selected by the frame rate conversion control unit 615, and Q (Emv (i +
2)), and further subjected to variable-length encoding by the variable-length encoding means 617, and output as V (Q (Emv (i + 2))). Then, the same processing as that of the (i + n + 1) -th frame is performed on the frame after (i + n + 1).

As described above, according to the present embodiment, frame-rate converting section 623 converts hybrid-coded image data with a high frame rate into hybrid-coded image data with a low frame rate. In considering the transmission speed of the communication line and the processing capacity of the terminal,
Since the frame rate is converted in accordance with the processing rate which is the most limiting, the use efficiency of the communication line does not deteriorate.

Also, by controlling an arbitrary start frame and an arbitrary number of combined frames in the hybrid-encoded image data, the hybrid encoding that performs orthogonal transformation on inter-frame prediction errors involving motion compensation is performed. The frame rate conversion can be selectively performed only on the data, and the amount of calculation can be significantly reduced, and the frame rate conversion can be performed at a high speed with a small load.

Embodiment 7 FIG. Hereinafter, a seventh embodiment corresponding to claim 7 of the present invention. A frame rate conversion apparatus using the frame rate conversion method according to the first embodiment will be described with reference to the drawings.

FIG. 9 shows a frame rate conversion apparatus using the frame rate conversion method in the first to sixth embodiments.

In FIG. 9, reference numeral 701 denotes an input terminal.
High frame rate image data from a terminal having high processing capability or high frame rate image data from a communication line having a high transmission speed is input.

Reference numeral 702 denotes a variable-length decoding unit which performs variable-length decoding of the hybrid coded data, thereby obtaining quantized prediction error orthogonal transformation data with motion compensation, and quantized prediction without motion compensation. The error orthogonal transform data is converted into quantized orthogonal transform coefficients. At the same time, within the frame set by the coding device of the input side terminal /
Detecting inter-frame discrimination, setting intra-frame / inter-frame (in), detecting motion-compensated / non-compensated discrimination set by the coding device of the input terminal, and performing / without motion compensation ( in), and also detects the quantization coefficient set by the encoding device of the input terminal, sets the quantization coefficient (in), and further sets the quantization coefficient (in) by the encoding device of the input terminal. The motion vector is detected and the motion vector (i
Set n).

Numeral 703 denotes an inverse quantization means, which uses the set quantization coefficient (in) to quantize the prediction error orthogonal transform data with motion compensation and the quantized prediction error orthogonal transform data without motion compensation. Transform data, the quantized orthogonal transform coefficients are transformed into prediction error orthogonal transform data with motion compensation,
The data is converted into prediction error orthogonal transform data without motion compensation and orthogonal transform coefficients.

Numeral 704 denotes a prediction error orthogonal transform data adding means. The orthogonal transform coefficient calculated in consideration of the motion vector by the motion vector / orthogonal transform coefficient calculating means 708, or the orthogonal transform coefficient stored in the image memory 706. The coefficient and prediction error orthogonal transform data with motion compensation or prediction error orthogonal transform data without motion compensation are added.

Reference numeral 705 denotes an orthogonal transform coefficient selecting means, which is controlled by the set intra-frame / inter-frame (in). That is, within the frame / between frames (i
When n) is “intra-frame”, the orthogonal transform coefficient inversely quantized by the inverse quantization means 703 is selected. When the intra-frame / inter-frame (in) is “inter-frame”, the prediction error orthogonal The orthogonal transform coefficients calculated by the transform data adding means 704 are selected and output.

Reference numeral 706 denotes an image memory for storing the orthogonal transform coefficients selected by the orthogonal transform coefficient selecting means 705.

Reference numeral 707 denotes a means for judging whether or not there is motion compensation (in).
6 is output to the motion vector / orthogonal transform coefficient calculating means 708. When the motion compensation is “none”, the orthogonal transform coefficient stored in the image memory 706 is output to the prediction error orthogonal transform data adding means 704.

Reference numeral 708 denotes a motion vector / orthogonal transform coefficient calculating means, which is stored in the image memory 706 determined as “present” by the motion compensation with / without (in) judging means 707 using the motion vector (in). By performing a matrix operation on the orthogonal transform coefficients that have been set, an orthogonal transform coefficient calculated in consideration of the motion vector is calculated. For the matrix operation, a method described later is used.

Numeral 709 denotes an initial image capturing / selecting means, which will be described later.
13 fetches the orthogonal transformation coefficient stored in the image memory 706 according to the image capture instruction set by
An orthogonal transform coefficient is output to an image memory 710 described later.

[0276] Reference numeral 710 denotes an image memory for storing the orthogonal transform coefficients selected by the initial image capture selecting means 709.

Reference numeral 711 denotes within a frame / between frames (in).
It is a judgment means.
06 orthogonal transformation coefficient is set at the subsequent stage of encoding control / in-frame /
It outputs to the inter-frame determination means 713, and in the case of “inter-frame”, outputs the orthogonal transform coefficient of the image memory 706 to the orthogonal transform coefficient subtraction means 712 described later.

Numeral 712 denotes an orthogonal transform coefficient subtracting means. The intra-frame / inter-frame (in) judging means 711 calculates an image based on the orthogonal transform coefficient stored in the image memory 706 when it is judged “inter-frame”. The orthogonal transform coefficient of the memory 710 is subtracted.

Reference numeral 713 denotes coding control / intra-frame / inter-frame determination means. When frame rate conversion is not performed, motion compensation with / without (in) is output as is with / without motion compensation (out). When frame rate conversion is being performed, motion compensation with / without (out) is set to “none” and output. Further, the frame immediately before the frame rate conversion is stored in the image memory 7.
10 to output an image capture instruction. Also, using the intra-frame / inter-frame (in), the orthogonal transform coefficient from the intra-frame / inter-frame (in) determining means 711 and the prediction error orthogonal transform data without motion compensation from the orthogonal transform coefficient subtracting means 712 are used. And outputs the result to the quantization means 714. Also, the quantization coefficient (in)
Is reset, and the result is output to a quantization unit 714 described later as a quantization coefficient (out).

Numeral 714 denotes a quantizing means which converts the orthogonal transform data and the orthogonal transform coefficient without motion compensation by the output quantized coefficient (out) into the quantized prediction error without motion compensation. The orthogonal transform data and the quantized orthogonal transform coefficient are respectively converted.

Reference numeral 715 denotes a frame rate conversion control unit which determines the continuation / end of the frame rate conversion based on the frame rate conversion information. Then, after the frame rate conversion is completed, quantized prediction error orthogonal transform data without motion compensation and quantized orthogonal transform coefficients are output to the variable length encoding means 717, and the frame rate conversion is not performed. Outputs the quantized prediction error orthogonal transform data with motion compensation, the quantized prediction error orthogonal transform data without motion compensation, and the quantized orthogonal transform coefficients from the variable length decoding unit 702 as they are. .

Reference numeral 716 denotes a motion vector selection means, which is the above-described coding control / intra-frame / inter-frame determination means 713.
With and without motion compensation (out) output from
Select a motion vector. That is, with motion compensation /
When “out” is “present”, the motion vector (i)
n) is output to the variable-length encoding unit 717 as it is, and when "none", the motion vector (in) is not output to the variable-length encoding unit 717.

[0283] Reference numeral 717 denotes a variable-length coding unit that converts quantized prediction error orthogonal transform data with motion compensation, quantized prediction error orthogonal transform data without motion compensation, and quantized orthogonal transform coefficients. By performing variable length coding, the data is converted into hybrid coded data. At the same time, the intra-frame / inter-frame discrimination set for the decoding device of the output terminal is set based on the intra-frame / inter-frame (in), with / without motion compensation (out), and with the motion vector (in). ), A motion vector with / without motion compensation is set for the decoding device of the output terminal, and a motion vector is set, and a quantization coefficient is set for the decoding device of the output terminal by the quantization coefficient (in). .

Reference numeral 718 denotes an output, which outputs image data having a low frame rate to a terminal having low processing capability or image data having a low frame rate to a communication line having a low transmission speed.

The motion vector / orthogonal transform coefficient calculation means (108, 20) used in the first to seventh embodiments will be described below.
8, 308, 408, 508, 608, 708), a matrix operation for calculating an orthogonal transform coefficient calculated in consideration of a motion vector will be described.

FIG. 11 shows the motion vector / orthogonal transform coefficient calculating means (108, 208, 308, 408, 50).
8, 608, 708) is a flow chart for explaining the details, wherein 1001 denotes a block search means, 1002 denotes an orthogonal transform coefficient calculating means, and 1003 denotes an orthogonal transform coefficient adding means.

Here, the motion vector / orthogonal transform coefficient calculating means (108, 208, 308, 408, 508, 60)
8, 708), the following is defined.

The motion vector / orthogonal transform coefficient calculating means (1
08, 208, 308, 408, 508, 608, 70
8) In the case where motion compensation is performed, the image memory (106, 2)
06, 306, 406, 506, 606, and 706) and the motion vector (in).

The processing of the two-dimensional orthogonal transformation is represented by F, the transformation matrix when the two-dimensional orthogonal transformation is decomposed into the one-dimensional orthogonal transformation is represented by T, and the transposed matrix is represented by t.

An image block obtained by performing an inverse orthogonal transform on hybrid-coded data for performing an orthogonal transform on an inter-frame prediction error accompanied by motion compensation with input image data is denoted by M, and an image memory (106, 206, 306, 406). , 5
06, 606, and 706), an image block obtained by performing an inverse orthogonal transform on the orthogonal transform coefficient stored in the image block is denoted by A. A is a reference block of M. Image memory (106, 206, 30)
6, 406, 506, 606, and 706) are orthogonal transform coefficients and are therefore divided at block boundaries, and A may be divided into four at the maximum. When the block is divided into four, the upper left block is A1, the upper right block is A2, the lower left block is A3, and the lower right block is A4. In A1, A2, A3, and A4, the common part with A is moved to the position corresponding to M, and the image blocks whose other parts are set to 0 are B1, B2, and B, respectively.
3, B4. FIG. 12 shows image blocks M, A, A
1, A2, A3, A4, B1, B2, B3, B4. A1, A2, A3, A4 and B1, in FIG.
The same shaded portions of B2, B3, and B4 have the same values. Further, the symbols used for A and M only indicate the positional relationship, and their values are different.

In FIG. 11, reference numeral 1001 denotes a block search means for searching for A by using a motion vector (in).

[0292] Reference numeral 1002 in FIG. 11 denotes orthogonal transform coefficient calculating means. From the searched A, F (A1), F (A
2) Find F (A3) and F (A4). Where F
(A1), F (A2), F (A3) and F (A4) are image memories (106, 206, 306, 406, 506, 506).
606, 706). Using a matrix operation described later, F (B1), F (B2), F (B3), F (B4)
Ask for. Hereinafter, a matrix calculation method will be described. The matrices used to perform this operation are shown in FIGS. This matrix is called an S matrix. Here, a case where the block size is 8 × 8 will be described. FIGS. 15, 16, 17, and 18 show the movement of 8 × 8 blocks when multiplied by the S matrix. Assuming that m = 1 to 7, when Sdlm is multiplied from the left to an 8 × 8 block, the 8 × 8 block is moved down by m rows as shown in FIG. Converted to 0. When Sdlm is multiplied from the right to the 8 × 8 block, as shown in FIG. 16, the 8 × 8 block is moved to the left by m columns, and is all converted to 0 from the right to the mth column. When Surm is multiplied from the left to the 8 × 8 block, the 8 × 8 block is moved up by m rows and converted to 0 from the bottom to the m-th row as shown in FIG.
When Surm is multiplied from the right to the 8 × 8 block, as shown in FIG. 18, the 8 × 8 block is moved right by m columns, and is converted to 0 from the left to the mth column.

Next, a method of converting F (A1) to F (B1) will be described with reference to the drawings.

FIG. 19 illustrates the method of converting A1 to B1. A, A1, and B1 are 8 × 8 blocks, and the size of the common part of A1 and A is a row and b column. A1 may be moved upward by (8-a) rows and leftward by (8-b) columns, and may be converted to 0 from the right to the 8-ath row and from the bottom to the 8-bth column. This conversion is S1 for A1
ur (8-a) is multiplied from the left, and then Sdl (8-a)
b) from the right. When this is represented by an equation, Sur (8-a) × A1 × Sdl (8-b) = B1. To obtain F (B1), F (B1) = F (Sur (8-a) .times.A1.times.Sdl (8-b)) = TSur (8-a) .times.A1.times.Sdl (8-b) tT = TSur (8-a) Tt × TA1Tt × TSdl (8-b) tT = F (Sur (8-a)) × F (A1) × F (Sdl (8-b)) Thus, it is understood that the S matrix itself is unnecessary, and if F (S matrix) is calculated and prepared in advance, it becomes easy to obtain F (B1) from F (A1). F (B2), F (B3), F (B
4) can be similarly obtained.

[0295] Reference numeral 1003 in FIG. 11 denotes orthogonal transform coefficient adding means, and F (B1), F (B2), F (B3), and F (B
4) is added to obtain F (A). To show the details of this processing, since the orthogonal transformation is a linear operation, F (B1),
When F (B2), F (B3), and F (B4) are added, F (B1) + F (B2) + F (B3) + F (B4) = F
(B1 + B2 + B3 + B4), where the shaded portions of B1 to B4 and A1 to A4
Since the shaded portions are equal, B1 + B2 + B3 + B4 = A, so that F (B1 + B2 + B3 + B4) = F (A) Therefore, F (B1) + F (B2) + F (B3) + F (B4) = F
(A)

F (A) obtained by this operation becomes S ′ (i) by being grouped in frame units.

As described above, according to the present embodiment, the amount of calculation is reduced by omitting the processes of inverse orthogonal transform, motion detection, and orthogonal transform, thereby significantly reducing the load and increasing the speed as compared with the conventional technology. Thus, the use efficiency of the communication line can be improved, and image data having a frame rate corresponding to the processing capability of the terminal can be generated.

[0298]

As described above, according to the frame rate conversion method according to the first aspect of the present invention, when communicating hybrid-coded image data between terminals, the frame rate is determined by the transmission speed of the communication line. In addition to converting the image data, the input image data having a high frame rate is converted into a frame rate corresponding to the processing capability of the terminal by considering the processing capability of the terminal, and output as image data having a low frame rate. In the output, the number of frames that can be transmitted on the communication line is
If the number of frames that can be processed by the terminal is smaller, the frame rate is converted in consideration of the processing capacity of the terminal,
Hybrid-coded image data of the number of frames that cannot be processed by the terminal is not output to the communication line, and there is an effect that the use efficiency of the communication line can be improved.

According to the frame rate conversion method of the second aspect of the present invention, hybrid-encoded image data for performing orthogonal transformation on an inter-frame prediction error accompanied by high frame rate motion compensation is input. By omitting the processes of inverse orthogonal transform, motion detection, and orthogonal transform, the amount of calculation can be reduced, the load can be greatly reduced and the speed can be increased as compared with the conventional technology, and motion compensation with a low frame rate can be performed. There is an effect that it is possible to provide a frame rate conversion method capable of generating hybrid-encoded image data that performs orthogonal transformation on an inter-frame prediction error that is not accompanied.

Further, according to the frame rate conversion method of the third aspect of the present invention, the hybrid coded image data for performing the orthogonal transform on the inter-frame prediction error accompanied by the high frame rate motion compensation, and By inputting image data including both hybrid-coded image data that performs orthogonal transformation on inter-frame prediction errors without compensation and omitting the processes of inverse orthogonal transformation, motion detection, and orthogonal transformation It can reduce the amount of computation, greatly reduce the load and increase the speed compared to the conventional technology.
There is an effect that it is possible to provide a frame rate conversion method capable of generating hybrid-encoded image data that performs orthogonal transformation on an inter-frame prediction error without motion compensation at a low frame rate.

Further, according to the frame rate conversion method of the fourth aspect of the present invention, the hybrid encoded image data for performing the orthogonal transformation on the inter-frame prediction error accompanied by the high frame rate motion compensation, and By inputting image data that includes both the inner-coded image data and the processing of inverse orthogonal transformation, motion detection, and orthogonal transformation, the amount of computation is reduced, and the load is greatly reduced compared to the conventional technology. Between the hybrid-coded image data and the intra-coded image data that perform orthogonal transformation on inter-frame prediction errors without motion compensation at a low frame rate. There is an effect that a frame rate conversion method capable of generating image data including both of them can be provided.

[0302] According to the frame rate conversion method according to claim 5 of the present invention, the hybrid coded image data for performing orthogonal transform on the inter-frame prediction error accompanied by the high frame rate motion compensation, and Input image data including three of hybrid-coded image data that performs orthogonal transformation on inter-frame prediction errors without compensation and intra-frame coded image data;
By omitting the processes of inverse orthogonal transform, motion detection, and orthogonal transform, the amount of calculation can be reduced, the load can be greatly reduced and the speed can be increased as compared with the conventional technology, and motion compensation with a low frame rate is involved. It is possible to provide a frame rate conversion method capable of generating image data including both hybrid-coded image data that performs orthogonal transform on no inter-frame prediction error and intra-frame coded image data. This has the effect.

Further, according to the frame rate conversion method according to the sixth aspect of the present invention, when communicating hybrid-coded image data between terminals, it is only necessary to convert the frame rate according to the transmission speed of the communication line. The image data having a high frame rate is converted into a frame rate corresponding to the processing capability of the terminal by considering the processing capability of the terminal.
Conversion using the frame rate conversion method described in any one of the above, the amount of computation is reduced by omitting the processing of inverse orthogonal transformation, motion detection, and orthogonal transformation, greatly reducing the load and increasing the speed compared to the conventional technology. And output as image data having a low frame rate. If the number of frames that can be processed by the terminal is smaller than the number of frames that can be transmitted by the communication line in the output, the processing of the terminal is performed. The conversion of the frame rate is performed in consideration of the capability, so that the hybrid-coded image data of the number of frames that cannot be processed by the terminal is not output to the communication line,
There is an effect that the use efficiency of the communication line can be improved.

According to a frame rate conversion apparatus according to claim 7 of the present invention, the frame rate conversion method according to any one of claims 1 to 6 is used,
By omitting the inverse orthogonal transform, motion detection, and orthogonal transform processes, the amount of computation can be reduced, significantly reducing the load and increasing the speed compared to the conventional technology, and improving the use efficiency of communication lines. Further, there is an effect that image data having a frame rate corresponding to the processing capability of the terminal can be generated.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining a configuration of a frame rate conversion method according to claim 1 of the present invention.

FIG. 2 is a flowchart illustrating a frame rate conversion process according to the frame rate conversion method according to the first embodiment.

FIG. 3 is a diagram showing a flowchart for explaining a frame rate conversion method according to claim 2 of the present invention.

FIG. 4 is a flowchart illustrating a frame rate conversion method according to a third embodiment of the present invention.

FIG. 5 is a flowchart illustrating a frame rate conversion method according to a fourth embodiment of the present invention.

FIG. 6 is a flowchart illustrating a frame rate conversion method according to claim 5 of the present invention.

FIG. 7 is a diagram for explaining a configuration of a frame rate conversion method according to claim 6 of the present invention.

FIG. 8 is a flowchart illustrating a frame rate conversion process according to the frame rate conversion method according to claim 6;

FIG. 9 is a diagram showing a configuration of a frame rate conversion device according to claim 7 of the present invention.

FIG. 10 is a diagram showing a configuration of a conventional rate conversion image encoding device.

FIG. 11 is a flowchart illustrating details of a motion vector / orthogonal transform coefficient calculation unit.

FIG. 12 shows image blocks M, A, A1, A2, A3, A
4, B1, B2, B3, and B4.

FIG. 13 is a diagram illustrating a matrix used in a matrix calculation method for calculating an orthogonal transform coefficient calculated in consideration of a motion vector.

FIG. 14 is a diagram illustrating a matrix used in a matrix calculation method for calculating an orthogonal transform coefficient calculated in consideration of a motion vector.

FIG. 15 is a diagram illustrating movement of an 8 × 8 block when the Sdlm matrix is multiplied from the left.

FIG. 16 is a diagram illustrating movement of an 8 × 8 block when the Sdlm matrix is multiplied from the right.

FIG. 17 is a diagram illustrating movement of an 8 × 8 block when a Surm matrix is multiplied from the left.

FIG. 18 is a diagram illustrating movement of an 8 × 8 block when a Surm matrix is multiplied from the right.

FIG. 19 is a diagram illustrating a method of converting A1 to B1.

FIG. 20 is a diagram showing a configuration of a conventional frame rate conversion method.

[Description of Code] 104 Prediction Error Orthogonal Transform Data Addition Unit 112 Orthogonal Transform Coefficient Subtraction Unit 204 Prediction Error Orthogonal Transform Data Addition Unit 212 Orthogonal Transform Coefficient Subtraction Unit 304 Prediction Error Orthogonal Transform Data Addition Unit 312 Orthogonal Transform Coefficient Subtraction Unit 404 Prediction Error Orthogonal transformation data addition means 412 Orthogonal transformation coefficient subtraction means 504 Prediction error orthogonal transformation data addition means 512 Orthogonal transformation coefficient subtraction means 604 Prediction error orthogonal transformation data addition means 612 Orthogonal transformation coefficient subtraction means 704 Prediction error orthogonal transformation data addition means 705 Orthogonal transformation Coefficient selection means 707 Motion compensation presence / absence (in) determination means 709 Initial image capture selection means 711 Intra-frame / inter-frame (in) determination means 712 Orthogonal transformation coefficient subtraction means 715 Frame rate conversion control means 919 Addition means 921 Pitch means 922 selector 924 adding unit 927 selector 928 subtractor 931 selector 934 adding means 937 Selector

Claims (7)

[Claims] 1. When communicating hybrid-coded image data between terminals, one terminal is a1, the other terminal is a2, the transmission speed of a communication line to which the terminal a1 is connected is b1, and the terminal is a1. The transmission speed of the communication line to which a2 is connected is b2, and the processing capability of terminal a1 is c.
1. Assuming that the processing capacity of the terminal a2 is c2, image data with a high frame rate is input from the terminal a1 side, and the number of frames output to the terminal a2 side is represented by the transmission speed b.
When 1 ≧ b2 and the processing capability of the terminal is c1 = c2, the number of frames that can be transmitted is set to b2. When the transmission speed is b1 ≧ b2 and the processing capability of the terminal is c1> c2, transmission is performed at b2. Set to the smaller of the number of frames that can be processed and the number of frames that can be processed in c2,
The transmission speed is b1 <b2 and the processing capability of the terminal is c1>
In the case of c2, the number of frames that can be transmitted in b1 or the number of frames that can be processed in c2 is set to the smaller number. If the transmission speed is b1> b2 and the processing capability of the terminal is c1 <c2, b2 Is set to the smaller of the number of frames that can be transmitted by c1 and the number of frames that can be processed by c1, and the frame rate of the high frame rate image data input from the terminal a1 is converted to a frame to the terminal a2. A frame rate conversion method characterized by outputting as low-rate image data. 2. Inputting hybrid-coded image data for performing orthogonal transformation on an inter-frame prediction error accompanied by motion compensation with a high frame rate, and performing orthogonal transformation on an inter-frame prediction error accompanied by motion compensation. The frame rate is obtained by combining a plurality of frames of hybrid-coded image data into a single frame of hybrid-coded image data that performs orthogonal transformation on an inter-frame prediction error without motion compensation. A frame rate conversion method for converting and outputting as hybrid-coded image data that performs orthogonal transformation on an inter-frame prediction error with low frame rate and without motion compensation. 3. Hybrid image data for performing orthogonal transformation on an inter-frame prediction error involving motion compensation with a high frame rate, and hybrid performing orthogonal transformation on an inter-frame prediction error without motion compensation. Hybrid encoded image data that inputs image data including both encoded image data and performs orthogonal transformation on inter-frame prediction errors with motion compensation, and inter-frame prediction without motion compensation Hybrid that performs orthogonal transform on error Hybrid that performs orthogonal transform on an inter-frame prediction error that does not involve motion compensation by combining a plurality of frames of image data including both encoded image data The frame rate is converted by synthesizing it with the frame of the encoded image data, and the motion at a low frame rate is performed. A frame rate conversion method for outputting as hybrid-encoded image data that performs orthogonal transformation on an inter-frame prediction error without compensation. 4. An image data that includes both hybrid-coded image data for performing orthogonal transformation on an inter-frame prediction error accompanied by motion compensation with a high frame rate and intra-frame coded image data. Input, a plurality of image data frames including both hybrid-coded image data and intra-frame coded image data that perform orthogonal transformation on inter-frame prediction errors with motion compensation Combining into a frame of image data that includes both hybrid encoded image data and intra-frame encoded image data that perform orthogonal transformation on one inter-frame prediction error without motion compensation And performs orthogonal transformation on inter-frame prediction errors without motion compensation at low frame rates. A frame rate conversion method for outputting as image data including both hybrid-coded image data and intra-frame coded image data. 5. A hybrid coded image data for performing orthogonal transformation on an inter-frame prediction error involving motion compensation having a high frame rate, and a hybrid for performing an orthogonal transformation on an inter-frame prediction error not involving motion compensation. Hybrid coded image data in which image data including three pieces of coded image data and intra-frame coded image data are input and orthogonal transformation is performed on an inter-frame prediction error accompanied by motion compensation. And a plurality of frames of image data including three, that is, hybrid-coded image data that performs orthogonal transformation on an inter-frame prediction error without motion compensation and intra-frame coded image data. Hybrid-encoded image data that performs orthogonal transformation on an inter-frame prediction error without motion compensation, and A hybrid that converts a frame rate by synthesizing it into a frame of image data that includes both intra-frame encoded image data and performs orthogonal transformation on an inter-frame prediction error without motion compensation at a low frame rate. A frame rate conversion method for outputting as image data that includes both encoded image data and intra-frame encoded image data. 6. The frame rate conversion method according to claim 1, wherein when converting the frame rate, the frame rate conversion is performed using the frame rate conversion method according to any one of claims 2 to 5. A frame rate conversion method characterized by performing. 7. A frame rate conversion apparatus for converting a frame rate using the frame rate conversion method according to claim 1. Description: