JPH01282981A - Animation coding system - Google Patents

Abstract

PURPOSE:To connect even coding for which a different inverse conversion computing element is used by executing a refreshing operation for the coding so that a refreshing cycle can be variable adaptively. CONSTITUTION:A picture signal decoded at a decoder at a receiving side is stored through an inverse quantizer and an IDCT circuit into a frame memory 302. Refreshing processing is periodically executed, in which the frame memory 302 is updated in order to suppress a mismatch error. The necessary refreshing cycle is adaptively determined by a parameter to indicate the actual condition of the coding and the cumulative characteristics of the mismatch error for the CODEC of the other party.

Description

DETAILED DESCRIPTION OF THE INVENTION [Object of the Invention] (Field of Industrial Application) The present invention relates to a video encoding method used in a TV conference system, a TV telephone system, and the like.

(Prior Art) In recent years, in this information society, so-called video conferences and video telephones that connect people are increasingly required. To realize such a device, discrete cosine transform (
Interframe predictive coding (DCT) and motion compensated DCT coding are effective methods, and much development is underway from basic research to actual device development.

However, one issue that has become a problem in this active research and development is the incompatibility of codecs due to subtle differences in formats. A particular problem is the disconnection caused by the accumulation of errors that occurs when the inverse transform arithmetic units on the transmitting side and the receiving side are different.

Even if there are slight differences in calculation accuracy or algorithm configuration,
Unless the inverse transformers are exactly the same, there is a possibility that the images reproduced by the inverse transform will be different on the transmitting and receiving sides. This difference between transmissions is called a mismatch error.

In fact, the mismatch errors that occur when processing one frame are very small, so unlike still image encoding, image quality does not deteriorate due to mismatch errors when processing one frame. However, when inter-frame prediction is used as in most video encodings, the mismatch that occurred in the previous frame is added to the mismatch that occurred in the next frame through the prediction process, and the errors accumulate one after another on the receiving side. Resulting in. Therefore, the image quality on the receiving side deteriorates, causing great harm to communication.

Currently, video conferencing and video telephone systems are attracting attention all over the world, and research and development of Kobutsuku, LSI conversion, etc. are being carried out independently in various places, and it is thought that m is about to occur during this period. Therefore, it is considered that unless this problem is solved, it will be extremely difficult for video conferencing and video telephone systems to become widespread.

A solution to this problem currently does not exist and is strongly needed.

(Problem to be solved by the invention) As described above, in conventional interframe prediction and transform coding using motion compensation, it is a serious problem that coding devices having different inverse transform arithmetic units cannot be connected. can be,
They are trying to hinder the spread of video conferencing and video calls. An object of the present invention is to make it possible to connect different encodings to the inverse transform arithmetic unit.

[Structure of the Invention] (Means for Solving the Problems) In order to suppress accumulated mismatch errors, the present invention has the following features:
Refresh processing such as updating the frame memory on the receiving side is performed periodically, and the required refresh period is adapted to the parameters representing the current state of encoding and the cumulative characteristics of mismatch errors for the other party's codec. This is a video encoding method determined by

(Operation) In this way, in the present invention, it is possible to estimate how much mismatch errors have accumulated based on the cumulative characteristics of mismatch errors caused by different conversion arithmetic units and the parameters representing the current state of encoding. Accumulation of mismatches can be suppressed as necessary, and communication with an encoder having a continuous transform arithmetic unit is possible.

(Example) Embodiments of the present invention will be described in detail below with reference to the drawings.

First of all, FIG. 1 shows motion compensation (abbreviation: M
C) It is a block diagram of the transmission side of encoding based on discrete cosine transform (abbreviation: DCT).

A motion vector is determined by a motion vector detector 102 using an input image signal (current frame) and an already encoded and reproduced image signal stored in the frame memory 101 (previous frame). A portion of the frame stored in the memory 101 is extracted as an interframe prediction signal by the variable delay 103 to which the vector is input, and is subtracted from the signal of the current frame by the subtracter 104.

The interframe prediction residual signal that has passed through the subtracter 104 and the raw current frame signal are input to a prediction determiner 105, where one of the signals is determined and input to a DCT circuit 106. The data transformed by the DCT circuit 106 is quantized by a quantizer 107. However, this quantizer is a quantizer whose characteristics can be adaptively changed by buffer control parameters.

The specific mechanism of the quantizer 107 includes various types such as linear quantization with variable step width, nonlinear quantization based on different probability distributions, and nonlinear quantization with variable step width. quantization can be considered. This embodiment will now be described in detail using linear quantization with a variable step width as shown in FIG. 2, taking one of them as an example.

The data passed through the quantizer 107 is encoded by variable length coding 108, multiplexed with prediction determination information and motion vector information by a multiplexer 109, input to a buffer 110, and transmitted to the receiving side.

Also, in the local decoding loop of the transmitter, the quantizer 10
The data from 7 is dequantized by dequantizer 111, and the data from I
The DCT circuit 112 performs inverse transformation.

In the adder 113, a reproduced image is obtained by reaching the signal that has passed through the IDCT circuit 112 and the number (j) specified by the switch, and is stored in the frame memory 1011.

Since the countermeasure against mismatch error in this embodiment uses buffer control information as a parameter representing the current state of encoding, the details of the buffer control method in this embodiment will be described. In order to prevent the buffer 110 from overflowing due to the changing amount of generated information, buffer control information is calculated based on the buffer's remaining capacity and is input to the quantizer 107 and the prediction determiner 105. Although various parameters can be considered as specific buffer control information, the step width S shown in FIG. 2 is used as a buffer control parameter in accordance with the linear quantization having a variable step width as described above.

The details of the mismatch error accumulation countermeasures in this embodiment, that is, the refresh processing and its adaptation will be explained below. As stated above, the mismatch error accumulates on the receiving side due to the use of interframe prediction. A third diagram showing a block diagram of the receiver of this embodiment.
As can be seen from the figure, in the interframe mode selected based on the prediction judgment information (the position directly above the switch 301), the mismatch errors that have been collected up to now are added to the new mismatch errors that have entered. , accumulate.

Updating the contents of frame memory 302 in the present invention changes switch 301 to intraframe mode (left position of switch 301) and updates (refreshes) the contents of frame memory 302.

However, with such measures, there is a possibility that the encoding efficiency will decrease due to performing refresh processing as necessary or not performing refresh processing sufficiently, so optimization of the refresh cycle is required. However, the refresh cycle that provides the best encoding efficiency varies greatly depending on the human input image and the communication bit rate, so if a fixed refresh cycle is adopted, a versatile refresh process cannot be realized.

Therefore, in this embodiment, the refresh period is adaptively changed as follows. The experimental results showed that there is a strong relationship between the error accumulation rate and the conversion characteristic, so the buffer control parameter is used as a parameter that represents the current state of encoding, and the refresh period is determined adaptively. For details of this technique, an example will be given, and this embodiment will be described below using FIG. 4, which is a detailed block diagram of the prediction judger 105.

Using the quantization step width parameter S output from the buffer 110 and the block signal from the clock 402, the average value i of S is calculated by the averaging circuit 401, and the average value i of S is calculated by the averaging circuit 401.
03. Further, the number of cycles of the clock signal generated from the clock 402 is counted by a counter 404, and R
Input to OM403. With these two human signals,
A refresh signal is output from ROM 403. The refresh signal is a zero field, the interframe prediction residual signal is the output of the prediction judger 105, and the average circuit 40 is a non-zero field.
1 and the counter 404 is reset, and the signal of the current frame becomes the output of the prediction judger 105.

An example of the contents of the ROM 403 is shown in FIG. The ROM input signal shown on the horizontal axis is the average value i of the quantization step width,
The vertical axis is the output of counter 404. T is a time indicating one cycle of the clock 402. The data filling this table indicates the value of the refresh signal (1 refresh, 0 no refresh). Therefore, if information about how much refresh is required for a quantization parameter is stored in this ROM, the optimum refresh cycle for that parameter can be realized. For example, in the case of the ROM shown in diagram M5, if the average quantization step width i is 2, the refresh period is 2T.
seconds, and when i is 8, it becomes 5T seconds.

Note that it is also possible to expand or reduce the scale of the ROM 403. The contents can be set based on theoretical analysis or empirical know-how, but in this embodiment, the following setting method is used.

② Random number data generated by a computer and actual moving images are used as training data.

29. For each parameter that represents the current state of encoding. Calculate the temporal cumulative characteristics of mismap errors with fixed parameters. (Express the error using an arbitrary distortion scale.) 3. Set the time until the error accumulates above an arbitrary threshold as the optimal refresh cycle.

The effects of the above embodiment will be explained. Figure 12 shows the characteristics of mismatch error accumulation for two scenes,
This was determined through an experiment. The horizontal axis shows rSNLO3SJ, which indicates how much the SN ratio of the reproduced image deteriorates when there is a mismatch in the inverse DCT compared to when the inverse DCTs of transmission and reception are the same. As can be seen from Figure 12, the cumulative speed of standard image scene A for another TV conference is about 4 times faster than standard image scene B for TV conference, and the cumulative characteristics of mismatch errors vary depending on the input image. Quite different. Therefore, even if it is attempted to perform refresh processing using a fixed period, it is impossible to deal with all scenes due to the reality shown in FIG. 12.

FIG. 13 is an ideal diagram showing the characteristics of mismatch error accumulation for scene A and scene B by applying the above embodiment of the present invention. The dotted line in Figure 13 is the RO above.
This is a threshold value used in the method of setting M contents. As shown in FIG. 12, since the average value i of the quantization step width differs depending on the scene, the optimum refresh period is selected for each scene so that the cumulative error does not exceed the threshold value.

The reason why it is possible to select a refresh period close to the optimum by changing the quantization step width is that the relationship between the quantization step width and the mismatch error accumulation was confirmed through experimental results, as shown in Figure 14 below. . FIG. 14 shows the quantization step width, S.

This shows the mismatch error that accumulates when is fixed, and makes it clear that the accumulation of errors strongly depends on the value of S. This data shows that the present invention is highly effective.

Next, a second embodiment will be described.

FIG. 6 is a block diagram of the transmitting side of the second embodiment of the present invention. This is almost the same sign as in the first embodiment, but since buffer control is performed by a conversion plane filter with an adaptive pass region, when the buffer amount is large, the image quality deterioration due to buffer control is not caused by quantization noise, but by blurring. This is a unique effect that results in a subjective image quality different from that of the encoding in the first embodiment. Also, this example uses a completely different parameter, not the quantization step width.
This shows that the refresh cycle can be determined by using it as a parameter that represents the current state of encoding.

As in the first embodiment, data stored in the frame memory 601 is delayed by a variable delay 603 based on the motion vector determined by the motion vector detector 602, and the current frame signal inputted by the subtractor 604 is , and input to the prediction judger 605. ' The prediction judger 605 outputs the current frame signal and one of the interframe prediction signals from the subtracter 604 as in the first embodiment, transforms it in the DCT circuit 606, and converts it in the transform plane filter 615. It is processed and quantized by quantization rA607.

The data passed through the quantizer 607 is sent to the variable length encoder 60
8, multiplexed with motion vector information, prediction determination information, and buffer control information in a multiplexer 609, input to a buffer 610, and transmitted. Further, the data that has passed through the quantizer 607 is dequantized by the dequantizer 611, and the data is dequantized by the dequantizer 611.
The T circuit 612 performs inverse transformation, the data specified by the switch 614 and the adder 613 reach the data, and the frame is output as a reproduced image.
It is input to memory 601.

Switch 614 is actuated by a signal from prediction determiner 605. That is, when inter-frame prediction is selected by the prediction determiner 605, the signal output from the variable delay 603 is input to the adder 613, and when the signal of the current frame is selected, the signal having a value of zero is input to the adder 613. 613 is performed. Note that, as in the first example, since the input image is divided into blocks, the above processing is performed on a block-by-block basis.

Next, the buffer control operation performed by the conversion plane filter 615 and the refresh method based on the control information will be described in detail. FIG. 7 is a diagram showing region division of a block transformed by DCT. Each region is designated by a number, from the first region containing only DC components to the sixth region containing high frequency coefficients. Therefore, in order to realize adaptive reduced-pass filtering, it is sufficient to input one region number as a parameter to the conversion plane filter 615 and pass all regions up to that region. For example, if you input the number 3 into the conversion surface filter, the number 1, the number 2, and so on. A third area passes through and a fourth area passes through. Fifth. The coefficients in the sixth region are truncated.

Buffer control using the above filter transmits only the region corresponding to the low-frequency transform coefficient when the amount of data in the buffer 610 is large, and when the amount of data in the buffer 610 is small, it is transmitted as shown in FIG. It is like transmitting even areas up to the sixth area.

In this embodiment, the refresh processing method in the above buffer control method is as follows. Since the configuration of the prediction determiner 605 that realizes the refresh processing of this embodiment is the same as the configuration of the prediction determiner 105, only the differences between the prediction determiner 605 and the prediction determiner 105 will be explained with reference to FIG. 4. explain. In this embodiment, the input S to the averaging circuit 401 is not the quantization step width but a parameter specifying the conversion surface area to be transmitted, and the contents of the ROM 403 should be different from those in the first embodiment. ROM40
Settings for the contents of item 3 can be made using the setting method described above. Therefore, if the input of the averaging circuit shown in FIG.
This is the operation performed in step 5.

Next, a third embodiment will be described.

FIG. 8 is a block diagram of a transmitter according to a third embodiment of the present invention. This shows that the present invention is applicable even when a frame dropper is added as a buffer control method to the encoding of the first embodiment and there are a plurality of parameters representing the current state of encoding.

In this encoding, first the frame dropper 8
On the 15th, the piece is dropped. The amount of frames to be dropped is determined by the amount of data in the buffer 810, and that information is input to the frame dropper 815 as a buffer control parameter. Since the signal that has passed through the frame dropper is encoded using an encoding having exactly the same configuration as in FIG. 1, the details thereof will be omitted here.

However, in this embodiment, since there are multiple buffer control parameters (frame dropping parameters and quantization step width parameters), the prediction judger 805 differs from the pre-i11 judger 105 in the first embodiment, and the prediction judgment is as follows. vessel 805
This will be explained in detail using FIG. 9, which is a detailed block diagram of.

Assuming that C is a frame drop parameter and S is a quantization step width, the frame drop quantization parameters are averaged by averaging circuit 907 and averaging circuit 902, respectively, and input to ROM 904, as shown in FIG. In the ROM 904, a refresh signal is read out using these two input signals and a signal from the counter 905. When the refresh signal is zero (not refreshed), the inter-frame prediction residual is output from the prediction judger 805, and when the refresh signal is non-zero (refreshed), the signal of the current frame passes through and the average circuit 901 .902 and counter 905
is reset to calculate the next refresh period.

Regarding the contents of the ROM 904, use the setting method explained above, and use the know-how that the more pieces there are, the slower the accumulation of mis-match errors will be.
It is possible to set the

Next, a fourth embodiment will be described.

FIG. 10 is a block diagram of a preliminary #1 determiner used in the fourth embodiment of the present invention. Figures 1 and 6.

When this is applied to the prediction determiner 105, 605, 805 in FIG. 8, a fourth embodiment is obtained. The cumulative characteristics of mismatch errors vary depending on the inverse conversion accuracy of the opponent's Kobukku. Therefore, refresh processing devised in pairs for one partner's controller is not necessarily optimal for other partner's controllers. This example shows that the present invention can also be applied to multiple bumps.

The operation of FIG. 10 is as follows. First, when communication with the other party begins, certification information of the other party is input to the lock cycle setting device 1001. Therefore, the clock cycle is determined depending on which partner Kobuku is, and the clock 1002
From ° a signal with that period is generated. Then, the parameters 1.2.3 that represent the current state of encoding are the average circuit 100
3°1004.1005 is averaged and input to ROMI 007. A refresh signal is output based on the average value manually entered in the ROM 1007 and the output of the counter 1006. The function of this refresh signal is the same as in the above embodiments, and its value determines whether to pass the interframe prediction residual or the current frame, and reset the averaging circuits 1003 to 1005 and the counter 1004.

Next, the unique effects of this embodiment will be explained.

Since the cumulative characteristics of mis-match errors differ depending on the opponent's Kobuk, a measure against mis-match errors designed for one opponent's Kobuk is not necessarily optimal. This will be explained using FIG. 11. FIG. 11 is a diagram showing the cumulative characteristics of mismatch errors, as used in the above refresh cycle setting method, and is hypothetical, but it is fixed at the values of parameters 1.2.3 shown in FIG. 10. In the case of two different opponents, it shows the characteristics of Kobutsuku.

As you can see from the image, if the refresh period for the parameter value is set to the point where the error exceeds the threshold as in the above setting method, the optimal period for Kobuku A is 4T.
He becomes a human and Kobutsuku B's becomes 2TA. Therefore, if the period of the clock 1002 is TA, and the optimized ROM 1007 for Kobukku A is used to communicate with Kobukku B, refresh will not be performed until the output of the counter 1006 reaches 4TA, so that the distortion that can be tolerated by the miss φ match error will occur. It has accumulated about twice as much. There is a high possibility that image quality will be significantly affected.

In order to avoid the above problems, it is conceivable to create an optimal ROM for each partner controller and use the appropriate ROM depending on the connected partner controller, but in this embodiment, as an example, The above problem is solved by introducing the clock period setter 1001. In other words, in the example shown in FIG. 11, when communicating with the other party B, the period of the clock 1002 is T (TT), B
BI A If a ROM created for Kobukku A is used, an optimal countermeasure for Kobukku B can be realized. In this way, you can respond to any opponent.

In the above embodiments, the buffer control parameter was used as a parameter representing the current state of encoding, but not only the buffer but also parameters calculated elsewhere, such as the power of the AC component of the prediction residual signal, the transmitted non-zero Parameters such as the number of coefficients, the signal-to-noise ratio of the reconstructed image relative to the current signal, etc. are also applicable. Also, instead of interframe prediction using motion compensation, interframe difference (corresponding to a motion vector of zero), predictive DCT circuit 106.6
06,806 instead of other transformations (e.g. sine transformation, K
L transformation, etc.) can be adopted.

[Effects of the Invention] As explained above, according to the invention, when performing refresh processing necessary to suppress mismatch errors, the refresh period is adaptively adjusted using parameters representing the current state of encoding. By changing the algorithm, optimal processing can be performed on any human-generated image, and by using the cumulative characteristics of mis-match errors in opponent encoding and devising refresh processing, optimal processing can be performed on any opponent's image. To realize video encoding in which optimal processing can be applied to any partner, thereby avoiding wasteful information generation due to excessive refresh processing and image quality deterioration due to insufficient refresh processing. I can do it.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing the transmitting side of the first embodiment of the present invention, FIG. 2 is a diagram showing the characteristics of quantization used in the first embodiment, and FIG. 3 is a block diagram showing the transmitting side of the first embodiment of the present invention. A block diagram showing the receiving side of the example, FIG. 4 is a detailed block diagram of the prediction judger, and FIG.
The figure shows an example of the contents of the ROM 403 in Fig. 4, Fig. 6 is a block diagram showing the transmitting side of the second embodiment of the present invention, and Fig. 7 is an example of area division of converted Brata. FIG. 8 is a block diagram showing the transmitting side of the third embodiment of the present invention, FIG. 9 is a block diagram showing the prediction judger of the third embodiment, and FIG. FIG. 11 is a block diagram showing a prediction determiner according to a fourth embodiment of the present invention, and is a diagram showing a hypothetical example of cumulative characteristics of mismatch errors. 12th
The figure is a diagram of experimental results showing the accumulation of mismatch errors due to differences in scenes. FIG. 13 is a diagram showing the effects of the first embodiment. FIG. 14 is a diagram of experimental results showing the relationship between mismatch error and quantization step width. 101.301,601,801・・・・・・・・・
...Frame memory 102.602.802.
・・・・・・・・・・・・・・・・・・Motion vector detector 103.603,803・・・・・・・・・・
・・・・・・・・・・・・Variable delay 104.604.11
04・・・・・・・・・・・・・・・・・・ Subtractor 105.1305.805・・・・・・・・・・・・
...... Pre-11 judge 106.806,80
6・・・・・・・・・・・・・・・・・・DCT
Circuit 107.1307.807・・・・・・・・・・・・
・・・・・・・・・・・・・・・・・・Quantizer 10
B, BO8,808・・・・・・・・・・・・・・・
・・・・・・・・・・・・・・・Variable length coding 109.8
09.809・・・・・・・・・・・・・・・・・・
......Multiplexer 110.810.81
0・・・・・・・・・・・・・・・・・・・・・
...Buffer ttt, eti, gtt...
・・・・・・・・・・・・・・・・・・・・・・・・
...Inverse quantizer 112.812,812...
・・・・・・・・・・・・・・・・・・・・・・・・ ID
CT circuit 113.613.813・・・・・・・・・
・・・・・・・・・・・・・・・・・・ Adder 11
4.301,405,814,814,908.100
8...Switch 401.901, 902.10
03.1004.1O05・・・・・・・・・Average circuit 402.903.1002・・・・・・・・・・・・・・・
・・・・・・・・・・・・・・・・・・Clock 403.
904.1007・・・・・・・・・・・・・・・
・・・・・・・・・・・・ROM404.905.1
006・・・・・・・・・・・・・・・・・・・・・
・・・・・・Counter 615・・・・・・・・・
・・・・・・・・・・・・・・・・・・・・・・・・
・Conversion surface filter 815・・・・・・・・・・・・
・・・・・・・・・・・・・・・・・・・・・・・・
...Komaochi device 1001...
・・・・・・・・・・・・・・・・・・Clock cycle setting device. Work v1 Ii Masanori Kensuke Chika Same as Mitsuyuki Matsuyama ♀ 2nd figure ♀ N Shake 07-'I-6n cross S out nme Riff L7 body I [(Me - I month 5th figure Mr. I construction collection No. 7 Figure Z JO (I) Sutsutsu? JO Riri product 0 beat R 5--zu 0 V1z 20 Riri

Claims (2)

[Claims] (1) A hybrid video encoding device in which inter-frame prediction or motion-compensated inter-frame prediction is combined with transform encoding, characterized in that a refresh operation for encoding is adaptively performed with a variable period. Video encoding method. (2) A video encoding method characterized in that when adaptively determining the period of the refresh operation described above, the period is determined using parameters representing the current state of encoding and error accumulation characteristics of the connected encoding.