JPH0449788A - Inter-frame coding device - Google Patents

Abstract

PURPOSE:To improve the working efficiency of a processor by judging the margin degree of the processing of the respective processors and increasing/ decreasing the number of searching points to execute the searching of a shift vector by movement compensation while adjusting to the marging degree. CONSTITUTION:A transmission memory does not become vacant by the arithmetic neck of a processor part by taking the number of searching points of the first stage smaller than the number of times of average operation per block m0. Besides, spare time does not exists in the processing of the processor part by taking the number of searching points to the (n)th stage sufficiently larger than the m0. Transmission data are generated newly and data are transmitted to a transmission line every time when the above-mentioned movement compensation on the one block and the encoding of other vector quantization and so on are executed, and since a transmission memory residual quantity R is always changed, movement compensation predictive encoding is executed while taking the transmission memory residual quantity R into consideration by the next block.

Description

[Detailed Description of the Invention] [Field of Industrial Application] The present invention relates to an interframe encoding device that performs motion compensation for moving images and compression encoding of image bands using one or more processors. .

[Conventional technology]

FIG. 3 is a block diagram showing the features of a conventional interframe coding device disclosed in, for example, Japanese Patent Application Laid-open No. 55-158784. In the figure, 1 has one or more processors; A processor unit performs compression encoding of the image band in addition to motion compensation of the video signal, and 2 is an input memo for temporarily storing the input digital image signal a to be encoded! Js
3 #'i, A s B 1 used for encoding
4 is a local memory for temporarily storing data generated during the encoding calculation process; 5 is a frame memory for storing the image signal of the previous frame; A transmission memory 6 converts the vector and encoded data into transmission data for the transmission path T and temporarily stores the data, and 6 sequentially transmits the data stored in the transmission memory 5 to the transmission path 7 in synchronization with the transmission speed. ,
Transmission control that monitors the remaining capacity of the transmission memory 5, constantly transmits control data b to the processor section 1, and controls the amount of transmission data per frame to be nearly constant by increasing or decreasing the amount of transmission data. Part 1 is a transmission line.

Next, the operation will be explained.

When transmitting a television signal (hereinafter referred to as a TV signal), interframe prediction is used in which the difference signal between adjacent frames is encoded and transmitted. However, in simple interframe prediction, The statistical properties of the prediction error signal vary accordingly, and for images with large motion, the compression ratio decreases because the difference signal between adjacent frames increases. Motion-compensated predictive coding is used as a countermeasure for this, but here, this motion-compensated predictive coding will be explained first. If the subject is moving, a pixel that gives the best prediction is searched for from the previous frame, and a change in its position is determined to be movement.

For example, divide a TV screen into small blocks as shown in Figure 4, shift each block based on the same position in the previous frame (this shift amount is called a shift vector), and calculate the difference from this difference signal. The value of the evaluation function is determined, and the shift vector that minimizes this value is set as the motion vector for that block (as a known prior art, for example,
Publication No.

JP-A-55-15B784, etc.).

In this way, the motion vector V of the image is detected, the previous frame signal is shifted by this motion vector, the difference from the current frame signal is determined, and this difference signal and motion vector are encoded and transmitted. If the movement is ideal and the motion vector can be detected accurately, the inter-frame difference from the previous frame moved by this motion vector will be zero. That is, ideally,
In this case, it is not necessary to send a prediction error signal only by sending motion vector information.

Additionally, a method is being considered to further improve coding efficiency by predicting the background that appears after a moving object separately from the moving object.

Although the motion vector is detected in this way, - general TV
A fairly large movement occurs in the liili image, and it is necessary to calculate the evaluation function for many shift vectors (for example, when detecting the movement of 8 samples on the left and right and 8 lines on the top and bottom, it is (2X8+1) X
(2X8+1)=289 shift vectors will be evaluated for the above-mentioned evaluation function values).

Since this may cause problems in terms of circuitry or time, a method has been considered to reduce the amount of calculations required by dividing the motion vector detection into multiple stages.

First, in the first stage (Fig. 5(a)), A, B.

The above-mentioned evaluation function values are determined for the first shift vector group indicated by C, D, and E, and among them, the shift vector with the minimum evaluation function value is determined. For example, search points A-E
If D is the shift vector that minimizes the evaluation function value, then in the second stage (Figure 5(b)) D
A second shift vector group placed in the vicinity of , ~D,
Similarly, the evaluation function value is determined for D, ~D, and the shift vector with which the evaluation function value becomes the minimum is determined. If the evaluation function of ri is the minimum, then the third stage (Fig. 5 (C)
), a third shift vector group placed near D, , ~D,1. For example, find the evaluation function value for D, 1.
If is the minimum, then 1. Let be the motion vector for that block.

At this time, the signal of the previous frame is shifted to the vector D11.
Using the shifted signal as a predicted signal, a difference signal between this predicted signal and the current frame signal and a shift vector D91.
is encoded as a motion vector and transmitted. The above is the first
This is an explanation of the encoding when D becomes the minimum evaluation function in the step, but motion estimation is performed in the same way even when other shift vectors become the minimum evaluation function. Further, here, a case of three-stage detection has been described, but even if there are more stages, all detections are the same. Fourth
As shown in the figure, even if it is necessary to obtain evaluation functions for 200 or more shift vectors in the conventional method, according to this conventional example, if the process is performed as shown in FIG. pieces, 2nd stage 9 pieces, 3rd stage 9 pieces, total 2
It is only necessary to obtain evaluation functions for three shift vectors, and the amount of required calculations can be reduced.

Next, the operation of the interframe encoding device will be explained. When the DigiMole image 9JJ signal a to be encoded is taken into the input memory 2, the processor section 1 performs motion compensation and vector quantization (electronics J
, June 1989-pp.

77-80 and July issue = PP, described in 45-47), etc. The previous frame image used in this encoding (
The image signal) is stored in either A or B of the frame memory 3 and can be accessed at any time. For example, if the image from one frame before is stored on the A side of the Everframe memory 3, the image from 7 frames before is inserted from this A side.
Perform encoding. In addition, a new image decoded and reproduced in this process is written to the B side of the frame memory 3, and the operations of A and B of the frame memory 3 are switched when all one frame has been processed. To encode the next frame, the previous frame image is stored on the B side of the frame memory 3, and this is used for encoding, and the decoded and reproduced image is stored in the frame memory 3. Write on the A side. By repeating this, interframe encoding is performed continuously.

The local memory 4 temporarily stores data generated during the encoding process. For example, data such as which block is being encoded at a certain point in encoding, or which shift vector is being calculated (determining the evaluation function value) during motion compensation, or the evaluation function value, etc. keep it.

Then, the motion vector obtained by motion compensation and the encoded data obtained by the encoding such as vector quantization are converted into transmission data for the transmission path T and written to the transmission memory 5, and then transmitted. The control unit 6 sequentially transmits the data stored in the transmission memory 5 in synchronization with the transmission speed of the transmission line 1. Furthermore, since the amount of transmission data such as motion vectors and other vector quantization generated by the processor unit 1 varies from block to block to be encoded, the transmission memory 5 has a certain capacity and is capable of absorbing these variations. Must. For this purpose, the transmission control unit 6 constantly monitors control data b such as the remaining amount of transmission memory and sends it to the processor unit 1, thereby controlling the encoding parameters such as vector quantization and controlling the amount of data to be transmitted. is controlled so that the amount of data transmitted per frame is approximately constant. but,
If the control is not complete and there is no more data to send, the transmission line 7
In addition, when the transmitted data becomes large, frames of the image may be dropped.

The above is a case where the processor section 1 is sufficiently fast, and the control of transmission data can be improved by software. The problem here is the computing power of the processor unit 1, but if the encoding is slow and the data to be transmitted is lost and idle occurs, the transmission efficiency will drop and the image quality will deteriorate.

Therefore, if the minimum required computing power is e, then 'I' FrnaxxNxEii (times/second)Fma
x: Maximum number of transmission frames (frame 7 seconds) N:
Number of blocks in one frame EB: Average amount of calculations (times) for one block. As mentioned above, coding such as vector quantization may be omitted for blocks with small interframe difference values, and since the amount of calculation for each block varies, the capacity of the transmission memory 5 may be increased. Then, the calculations are sufficiently averaged. However, since the average amount of computation for one block changes depending on the type of image, the actual computing power will be somewhat larger than e, so if the computing power cannot be increased to 11 due to the hardware configuration, etc., the number of searches for shift vectors by motion compensation is reduced and the average calculation of one block is
Sometimes x is made smaller.

[Problem to be solved by the invention]

Since the conventional interframe encoding device is configured as described above, it only performs motion compensation for a small number of search points with enough margin to prevent idleness on the transmission path, depending on the processing capacity of the processor. There was a problem in that the effect of motion compensation was halved and image quality deteriorated. Another problem is that even if a shift vector with a smaller (similar) evaluation function value is found at the initial N] stage of the search, the process is not terminated and the search must continue in vain.

This invention was made to solve the above-mentioned problems, and it takes full advantage of the efficiency of the processor and produces high-quality images.
iii. It is an object of the present invention to obtain an interframe coding device that enables self-output.

[Means to solve the problem]

The interframe encoding device according to the first claim determines the processing margin of each processor from the remaining amount of transmission data stored in the transmission memory, and searches for a shift vector by motion compensation when there is margin. The number of search points is increased (the number of searches is increased).

The interframe encoding device according to the second claim prioritizes shift vector search processing for motion compensation for a predetermined number of blocks subsequent to the block being encoded, performs the search according to the core priority order, and performs encoding. The evaluation function value of the shift vector of the search point is averaged for each block.

[Effect]

The interframe encoding device according to the first claim is configured to determine the processing margin of each processor and increase or decrease the number of search points for searching for a shift vector by motion compensation in accordance with the core margin. Improve processor efficiency.

The interframe encoding device according to the second claim prioritizes each motion compensation search process for a predetermined number of blocks prior to encoding, and searches for motion compensation according to the priority order. , the evaluation function values of the shift vectors at the search points for each encoded block are averaged.

[Embodiments of the invention]

An embodiment of the present invention will be described below with reference to the drawings.

Note that the configuration of the interframe encoding device according to the first and second claims is the same as that of the conventional interframe encoding device (FIG. 3), so a description thereof will be omitted.

Next, the motion compensation predictive coding operation of the 7-frame relational coding apparatus according to the first aspect of the present invention will be described with reference to FIG.

If the average amount of calculations other than motion compensation for one block is EBO, the required calculation capacity of the processor unit 1 without motion compensation is 6.
Q is 'o= Frnax ' N X EBO (times/second) F
rnax: 'lk Number of large transmission frames (frame 7 seconds) N: Number of blocks in one frame, and the calculation capacity e (e>
eo) to e. The remaining capacity after subtracting is used for motion compensation. The average amount of computation EBM for motion compensation per block is EBM, and the shift vector search can be performed an equivalent number of times (hereinafter referred to as mo) on average. However, depending on the image quality, the control of the transmitted data may be uneven, and frames may be dropped.
The number of transmission frames becomes less than F'max and 60 also changes. Therefore, depending on the transmission memory remaining amount R, the search is controlled to be performed in the next step up to the first stage in FIG.

R<R,... Search R up to the first stage, <R<R,... Search Rn up to the second stage, -+<R
...Search up to the nth stage Here, the search points of the first stage (
In the first part, 5A) is m above. By making it smaller than , the transmission memory 5 will not become empty due to calculation bottlenecks in the processor section. Also, the number of search points up to the %nth stage is m. By making the processor section 1 sufficiently larger,
There will be no free time for processing.

Every time the above-mentioned motion compensation and other encoding such as vector quantization are performed for one block, there is new transmission data and data sent to the transmission path T, and the remaining transmission memory i
Since lR constantly changes, the next block is subjected to motion compensated predictive coding in consideration of its remaining transmission memory capacity R.

Regarding the selection of search points from the first stage to the nth stage, fixed points are selected from points with high frequency of occurrence in Figure 1, but this is different from the multi-stage variable points shown in the conventional example. Alternatively, the already determined motion vectors of surrounding blocks or points near them may be used.

Next, the motion compensation predictive coding operation of the interframe coding apparatus according to the second claim will be explained using FIG. 2.

If only the interframe encoding device according to the first claim is used, the number of search points will vary meaninglessly from block to block, and the search process will continue even if a motion vector with a small evaluation function value is found in the initial stage of the search. Since there may be some waste, the system is designed to improve this.

Coding processing such as motion compensation vector quantization does not need to be performed for each block at once; it is sufficient that motion compensation precedes it. Therefore, a shift vector is searched by giving priority to motion compensation up to t blocks (t: fixed) counting from the block being encoded. As described above, the total number of search stages is determined by the remaining transmission memory amount R, and a total of i stages of search are performed (R, <R<Ri).

If the search for shift vectors performed in encoding up to the previous block is indicated by Block 2) is the second stage lv
Well,...(t-1) block has completed up to the second stage, and the evaluation function values up to each stage are shown numerically within the frame. In Figure 2, the example is when 1 = 11 steps, but the first priority is to obtain the evaluation function value for the first time! , the block searches its first stage. The evaluation function value at that time is 210. Next, in the t block,
Let it be the block with the maximum evaluation function value up to each stage.

Finally, in FIG. 2, the seventh block, which has the evaluation function 290, performs a search for a total of 11 stages according to the sequential numbers. When everything is finished, the block to be encoded this time (
Encoding such as vector quantization of block 1) is performed, and this information (for motion compensation, measurement function value f: motion vector data to be minimized) is sent to the transmission memory 5. Similar processing continues for the next block and beyond.

As for how to prioritize, it may be as simple as the one described above, but in reality, better effects can be obtained by classifying and controlling various cases.

〔Effect of the invention〕

As described above, according to the invention according to the first claim, since the processing margin of each processor is determined and the number of search points for searching for a shift vector by motion compensation is controlled, the processor can be It has an effect that can be used to a limited extent.

Further, according to the invention according to the second claim, the motion compensation search process for each predetermined number of blocks before encoding is performed in accordance with the priority order, so that the motion compensation in each block is Since the evaluation function value of the vector is averaged and becomes a small value, it has the effect of making it possible to output an image with a higher image quality.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram illustrating a motion compensation search operation of an interframe encoding device according to an embodiment of the invention according to claim 1;
FIG. 2 is a diagram illustrating a motion compensation search operation of an interframe encoding device according to an embodiment of the invention according to the second claim;
The third section is a block diagram showing the configuration of a conventional interframe encoding device, and FIGS. 4 and 5 are diagrams explaining the conventional motion compensation search operation. In the figure, 1 is a processor section, 2 is an input memory, 3 is a frame memo IJ, 4 is a local memory, 5 is a transmission memory, 6 is a transmission control section, and T is a transmission path. In addition, the same symbols in the figures indicate the same or equivalent parts. Patent Applicant: Mitsubishi Electric Corporation Figure x 9 Previous page I] -/R's sign 1: Line 1 υ converted t? Drinking ○: Red-colored mouth/tsuku, 51 nioteu a! Sword (Ji 1119 A helmet)
Pleasure diagram

Claims (2)

[Claims] (1) In an interframe coding device that uses one or more processors and performs motion compensated predictive coding, the margin of coding processing of each processor is determined, and motion is performed in the free time of the coding processing. An interframe encoding device characterized by controlling the number of search points in a search for a shift vector by compensation. (2) In an interframe encoding device that uses one or more processors and performs motion compensated predictive encoding, each motion of a predetermined number of blocks is An interframe encoding device characterized in that a shift vector search process using compensation is prioritized, and a shift vector search using motion compensation is performed in accordance with the priority order.