JPH08322019A - Method and apparatus for up-conversion with motion compensated - Google Patents

Abstract

PROBLEM TO BE SOLVED: To achieve a method and a device for motion compensated upconversion in which only a slight line storage capacity is required. SOLUTION: Spatial line memory structure is used for motion evaluation device ME, by considering a line memory in a noise-filtering means SF and a main block motion vector and a corrected sub-block motion vector which is ultimately adapted to a motion compensated interpolation device MC1 are used. The main block motion vector is calculated during a field section of one output field, and the sub-block motion vector is calculated when it is suitable to use the vector in the next output field during the repetition of a first output field.

Description

Detailed Description of the Invention

[0001]

FIELD OF THE INVENTION The present invention relates to a method and apparatus for motion compensated upconversion.

[0002]

2. Description of the Related Art The conventional TV standard is 50, 59.9.
It has a field frequency of 4 or 60 Hz. When such an image is displayed, line-shaped flicker occurs in the case of a specific image content, or large area-shaped flicker occurs in the case of a bright image, especially on a large screen. Such artifacts occur when the display upconversion is performed at the receiver and the field frequency is 2
When doubled, it is reduced. However, simple up-conversion algorithms like AABB induce new motion artifacts. In order to prevent the above artifacts, motion compensation, which requires a large amount of memory, is performed in the receiver.

[0003]

SUMMARY OF THE INVENTION One object of the invention is to disclose a method of motion-compensated upconversion which requires low storage capacity, in particular low line storage capacity. The object is achieved by the method according to claim 1. Another object of the invention is to disclose an apparatus utilizing the method of the invention described above.
The object is achieved by a device according to any one of claims 24 to 27.

[0004]

Motion-compensated up-conversion requires motion estimation and motion-compensated interpolation. During motion vector processing, multiple line memories are required for the motion estimator. Since such a line memory requires a large amount of chip area, two integrated circuits for motion-compensated up-conversion are required due to the direct circuit configuration. According to the invention, the storage capacity, in particular the number of line delays, is reduced, which makes it possible to build a motion-compensated up-converter with only one integrated circuit (excluding the field memory).
This uses a spatial line memory structure for the motion estimator, taking into account the line memory in the noise filter means, and the modifications applied to the main block motion vector and finally the motion compensated interpolator. It is realized by using

One field memory is used to double the field frequency. This may be done either before or after the motion evaluation. Since the available field processing time is doubled before doubling the field frequency, it is usually attempted to perform motion estimation in the time domain in order to obtain lower cost, slower circuits. The second field memory is used in connection with the motion estimator. The third field memory performs motion-compensated interpolation with noise compression on high field frequency signals, and thus may be motion-compensated with predictors, iterative filter means, and appropriate processing time delays. Required for interpolation. The present invention is based on, for example, the interlace system of 50 / 59.94 / 60/50 Hz to 100 / 119.88 / 120.
/ 75Hz interlace up-conversion, 50 / 59.94 / 60 / 50Hz interlace to 100 / 119.88 / 120 / 75Hz
100 / 119.88 / 120 / 75H from 50 / 59.94 / 60 / 50Hz proscan system up-conversion to proscan system
It can be used for up-conversion of z to the pro-scan method.

One advantage of the present invention is that even when spatial and iterative temporal noise compression is added to motion compensated upconversion (MCU), in a further developed embodiment only two field memories are used. That is, all of the above functions can be fully performed. This is especially true under the condition that the motion compensated interpolation receives a video input signal that is partially identical to the motion estimator and after the erroneous motion vector is calculated, the temporal filter does not propagate the error. This is achieved by using a simple circuit.

Tests have shown that the absence of error propagation in the temporal filter is due to the type of control used for temporal noise compression, and that the error is field to field attenuated. . Another advantage of the present invention is that it only needs to store motion vectors of large main blocks, so that the capacity of the vector memory can be small. This is accomplished by calculating a modified sub-block motion vector from the main-block motion vector just before it is needed in the process so that the sub-block motion vector need not be stored. It

The motion vector of the main block is calculated during one output field interval, and the sub-block motion vector is calculated when it is just right for use in the next output field during the repetition of the first output field. It is possible. The calculation of motion vectors and / or temporal filters has the advantage that it can be controlled by the calculated noise level.

In principle, the method of the invention is such that a field of source field frequency is transformed to form a field of double field frequency and motion estimation is performed,
The motion-compensated interpolation on the double field frequency field is suitable for motion-compensated up-conversion of an interlaced video signal performed using motion information from motion estimation, and according to the method of the present invention. During motion estimation, the motion vector of the main block and the motion vector of the sub-block smaller than the main block are calculated for one input field, and the pixels required for this calculation are Means, while motion compensated interpolation is performed on the motion vector of the sub-block during the next output field.

Further advantageous embodiments of the method according to the invention are described in the respective subclaims. In principle, the device for motion-compensated up-conversion of an interlaced video signal according to the invention: calculates the motion vector of the main block and the motion vector of a sub-block smaller than the main block for one input. Second storage means having line storage means for storing the pixels used to store the input and output connected to a motion estimator for field-to-field motion estimation; A subsequent first storage means for converting the output field of the second storage means into a field of double field frequency; and-receiving a motion vector of the sub-block from the motion estimator, into a next output field. Motion compensation is performed from field to field on the output signal obtained from the first storage means, and the motion compensated upcon Or a third storage means having its inputs and outputs connected to a motion-compensated interpolator for generating a version output signal, or the device: -converts a source field into a field of double field frequency. A first storage means used for calculating the motion vector of a main block and a motion vector of a sub-block smaller than the main block for one input field; A second estimator which has means and a field-to-field motion estimator, whose inputs and outputs are connected to the motion estimator; and whose inputs and outputs are from the motion estimator to the sub-blocks. Receives a motion vector and moves from field to field on the output signal obtained from the output of the second storage means during the next output field. Compensating and comprising a third storage means connected to a motion compensated interpolator for producing a motion compensated up-conversion output signal, or alternatively the device comprises: a source field of double field frequency A first storage means used for converting to a field; a motion vector of the main block for one output field is calculated and stored in the storage means and then the main block motion for the next output field A subsequent second storage means, the input and output of which are connected to a motion estimator that calculates a motion vector of a sub-block smaller than the main block from the vector and performs motion evaluation from field to field; On the output signal obtained from the output of the second storage means during the next output field, receiving the motion vector of the sub-block from the evaluator Motion-compensated interpolator for generating motion-compensated up-conversion output signals from the field to the motion-compensated interpolator, and a third storage means having its input and output connected, or The device comprises: -a first storage means used for converting the source field into a field of double field frequency; -the input signal of the second storage means is obtained from the output signal of the first storage means. And motion compensation from the motion evaluator to perform motion compensation from field to field on the output signal obtained from the output of the second storage means to generate a motion compensated upconversion output signal. A second input of the motion compensated interpolator, the signal being supplied to the first input of the interpolated interpolator A second storage means for receiving the output signal of the second storage means at a second input,
The signal at the first input is obtained from the output signal of the first storage means, calculates the motion vector of the main block for one output field, stores it in the storage means and then for the next output field. , A motion estimator that performs motion compensation from field to field by calculating a motion vector of a sub-block smaller than the main block from the motion vector of the main block.

Further advantageous embodiments of the device according to the invention are described in the respective subclaims.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT A preferred embodiment of the present invention will be described below with reference to the accompanying drawings. The motion-compensated up-conversion process has the following major steps: -motion-compensating horizontally and vertically only luminance; -selective spatially and temporally for luminance and temporally only for chrominance. Noise-compression in the following steps; and luminance- and chrominance motion-compensated interpolation steps.

The motion compensation evaluator calculates a motion vector represented by MV, and the MV is accompanied by a reliability CL and a noise level N per field. This result is used to generate motion compensated output pixels. In the following, in the 50/60 Hz range: -even source field = source field
1, 5 ,. ． ． , -Odd source field = source field
3,7 ,. ． ． , In the 100/120 Hz range: -output field 1 ← from source field 1-output field 2 ← combination of source field 1 and source field 3-output field 3 ← combination of source field 3 and source field 5 -Output field 4 ← Assume a combination of source field 3 and source field 5.

Input and output samples are represented in 8-bit words from 0 = LSB (least significant bit) to 7 = MSB (most significant bit). Video input format (4: 1:
1) is: -Y = 16 (black). ． ． 240 (white), blank value =
16-U, V = -112. ． ． 0. ． ． Assume that +112 and blank value = 0 (two's complement).

The video output format (before D / A conversion) is: Y = 0 (extreme black). ． ． 255 (extreme white), blank value = 16-U, V = 0. ． ． 128. ． ． Assume that 255 and blank value = 128. In FIG. 1, a field memory or FIFO (first in first out) FM1 is a motion evaluator ME.
Of the interlaced source signal I. The motion estimator ME operates in the 50 Hz range and comprises two delay arrays LD11 and LD12 each having a delay of 12 lines. The motion estimator ME uses 16 pixels * in the proscan pixel grid, which are horizontally subsampled as described below.
Calculate the 8-line motion vector MV. Motion evaluator M
E is a sub-block motion vector S for the sub-block SB of 2 pixels * 2 lines from the main block vector MBMV.
Calculate BMV further. The number of line delays required depends on the number of lines in the main block, the number of lines in the sub-block, and the feasible range of both vertical motion vector components. In this example, the number of lines in the main block MB is 8, the number of lines in the sub-block is 2, each vertical component of both motion vectors is 1, and a total delay of 12 lines is reached. The line delay has a length of 704/2 = 352 pixels because it stores only horizontally sub-sampled lines.

The main block motion vector MBMV and the sub block motion vector SBSV are calculated in the 50 Hz region, but the motion compensated interpolator M in the 100 Hz region.
Since it is used in CI, it has to be stored in the vector memory RAM in the motion estimator ME. The source frame has, for example, 704 active pixels and 576 active lines, and since the original main block has a size of 32 * 16, two source fields have (7
04/32 = 22) * (576/16 = 36) = 792
Main block motion vector MBMV and 792 * (1
6/2 = 8) * (8/2 = 4) = 25344 sub-block motion vectors SBSV. As described below,
The main block motion vector MBMV and the sub block motion vector SBSV can be encoded by 6 bits each. For each sub-block motion vector SBSV,
The reliability value CL is encoded by, for example, 8 bits.
Therefore, the RAM is 792 * 6 + 25344 * (6+
8) = 359568 bits or 44946 bytes need to be stored. This corresponds to a total line delay memory capacity of 44946 / 704≈64.

The delay arrays LD11 and LD12 are 12
Corresponding to the full line delay of. Thus, the motion estimator ME requires a complete line delay of at least 64 + 12 = 76. In a chip technology of 0.5 μm, one complete line delay requires a chip area of approximately 2 mm ² , and in the above example, the total area of memory chips in the motion evaluator ME is at least 76 * 2 = 152
reach mm ² . However, the maximum chip area should not exceed 85 to 90 mm ² in order to obtain good yield (and thus cheap chips) during chip manufacture. Therefore, at least two integrated circuits (excluding field memory) are required to implement the circuit of FIG.

The output signal of the field memory FM1 is 5
Passing through the 0 Hz to 100 Hz speed-up field memory SPUFM, said speed-up field memory SPUFM is required for the first input of the motion-compensated interpolator MCI and for the operation of the motion-compensated interpolator MCI, and The output performs AABB upconversion directly to the input of another field memory FM2 which is fed to the second input of the motion compensation interpolator MCI. The output signal of the speed-up field memory SPUFM first passes through the spatial noise compression filter means SF and / or the temporal noise compression filter means TF before entering the motion compensated interpolator MCI and the field memory FM2. The output of the motion estimator ME is the motion vector MV and the reliability value C
L and / or noise measure N for motion compensated interpolator M
Send to CI.

Advantageously, the temporal noise compression filter means may be an iterative filter means which combines the spatially noise filtered signal with the motion compensated or output signal of the previous field FM2. Such motion compensation is performed by the predictor PRED. The temporal noise compression filter TF and the motion compensated interpolator MCI are
It can be controlled by the noise level N and the reliability CL as described below.

The output O of the motion compensated interpolator MCI is
The signal is finally motion compensated, up-converted and consequently noise-compressed to be stored or displayed on the screen. The processing time in the motion compensated upconversion MCU circuit can be easily adapted using the appropriate delays shown in the other figures, eg PTD1 and PTD2. The functions of the various blocks are described below with reference to the other figures.

In FIGS. 1 to 5 and 7 (A), parallel bold dashes represent the line delay of the entire length (eg, 704 pixels), and the other dashes store the sub-sampled image data. It shows half the line delay. The heavy dashed line dash is the common line delay used in both the motion compensated interpolator MCI and the predictor PRED, or the motion estimator ME and the spatial noise compression filter SF.

The line delay is used to make available the previously calculated results needed in the blocks of the figure above when the pixels of the current line are output. In FIG. 1, the spatial noise compression filter SF and the motion compensated interpolator MCI each have two full line delays LD1.
3 and LD15 are required. Motion compensated interpolator M
The CI and predictor PRED each require three full line delays LD16 and LD14, which may be common line delays. Thus, the motion-compensated upconversion MCU circuit of FIG.
It requires a memory capacity of at least 76 + 2 + 2 + 3 = 83 full line delays.

Basically, in one embodiment of FIG. 2, the speed-up field memory SPUFM from AABB up-conversion from 50 Hz to 100 Hz is arranged in front of the field memory FM1 and the motion evaluator ME, so The evaluator ME uses the main block motion vector M
This is different from the embodiment of FIG. 1 in that the BMV, the sub block motion vector SBMV, and the reliability CL are not stored. The main block motion vector MBMV and the sub block motion vector SBMV are calculated in the same field period, and thus used in the motion compensated interpolator MCI.
Since it takes some time until the sub-block motion vector SBMV becomes available, the delay LD 28 of 10 lines is used.
Are provided between the output of the speed-up field memory SPUFM and the filter means SF or the field memory FM2. The number of line delays in the line delay LD 28 depends on the block size and the size of the vertical component of the maximum sub-block motion vector SBMV.

A further one-line delay LD 27 is provided for the motion vector MV / reliability CL between the motion evaluator ME and the motion compensated interpolator MCI. In the case of the even output line EOL, the motion vector MV information of the input signal of the line delay LD 27 is transmitted to the predictor PR via the switch SW1.
Sent to ED. The line delay LD 27 is written for one line and read for two consecutive lines.
The reason is that the same sub-block motion vector SBMV vector is applied to two lines.

The number of line delays LD21 to LD26 is
Corresponds to the number of line delays in FIG. This is about 12 + 1
A complete line delay of 0 + 2 + 2 + 3 + 1 = 30 is approximately 6
It is meant to be required for the motion compensated upconversion MCU circuit of FIG. 2 corresponding to a chip area of 0 mm ² . Basically, the embodiment of FIG.
It differs from the embodiment of FIG. 2 in that the first ten line delays of 31 additionally perform the function of the LD 28. This means that the above 10 line delays should be full line delays. Line delay LD32 to LD37
2 corresponds to the number of line delays in FIG. However, the total line delay capacity is reduced by five complete line delays.

Basically, the embodiment of FIG.
3 in that two additional line delays of 1 are commonly used for the line delay LD 43 of the spatial filter SF.
The embodiment is different from the above embodiment. This means that the two additional line delays of array LD41 should be full line delays. Line delays LD42 and LD44 to L
The number of D47 corresponds to the number of line delays in FIG. However, the total line delay capacity is reduced by one complete line delay.

1 to 4, all main block motion vectors MBMV and sub block motion vectors SBMV
Is calculated for each source field.
Basically, in the embodiment of FIG. 5, the vector of the main block motion vector MBMV is calculated in the first field, while the vector of the sub block motion vector SBSV is calculated just before it is used in the next field. It need not be stored. Thereby, on the one hand, the vectors of the 792 main block motion vectors MBMV have to be stored in the memory RAM of the motion evaluator ME in 6-bit units, while on the other hand the motion evaluator ME Used for the line delay LD53 of the static filter SF)
Only two full line delays LD51 and two half line delays LD52 are needed. Line delay LD4
The number of 4 to LD 47 corresponds to the number of line delays in FIG. However, the total line delay capacity is 10 * 704 + 10 * 704 / 2−, which corresponds to approximately 14 complete line delays.
792 * 6/8 = 9966 bytes are reduced.

FIG. 5 shows a field memory or FIFO.
A speed-up field memory SPUFM is used to double the field frequency of the input signal I. The output signal is supplied to a motion estimator ME including a low capacity vector memory, a second field memory or FIOF field memory FM1 and a spatial noise compression filter SF. The output of the field memory FM1 is supplied to the second input of the motion evaluator ME. The output of the motion estimator ME is the motion vector MV of the subblock, the interpolator MCI motion-compensated with other information, and the predictor PRE.
To the first input of D. The output of the spatially noise filtered spatial noise compression filter SF is
It is fed via a temporal noise filter TF to a first input of the motion compensated interpolator MCI and to a third field memory or a FIFO field memory FM2. The output of the field memory FM2 is supplied to the second input of the predictor PRED. The output of the predictor PRED is fed to the second input of the temporal noise filter TF. The output signal O of the motion-compensated interpolator MCI is the final motion-compensated, up-converted and noise-filtered signal to be displayed on the screen.

In FIGS. 1-5 and 7 (A), parallel bold dashes represent full-length (eg, 704 pixels) line delays, and other dashes store sub-sampled image data. It shows half the line delay. The divided bold dash is the motion evaluator M
E, the common line delay used by the spatial noise compression filter SF, the motion compensated interpolator MCI and the predictor PRED, respectively.

The line delay is used to make available the previously calculated results needed in the blocks of the figure above when the pixels of the current line are output. SGS-Motion evaluation chip such as STi3220 from Thomson, S
A two-dimensional motion-compensated interpolation filter such as GS-Thomson IMSA 110, and SGS-Thomson S
An MPEG2 system decoder including a two-dimensional motion compensated interpolator such as Ti3500 is already on the market.

The separate motion-compensated up-conversion MCU arrangement shown in the basic block diagram of FIG.
Only two field memories are needed, the first one in the figure
Field memory or FIFO speedup field memory SPUFM, second field memory or FIFO field memory FM, motion evaluator ME,
A motion compensated interpolator MCI is shown. The processing time required by the motion evaluator ME is indicated by PTD1.
The output of the field memory FM is the second of the motion evaluator ME.
, And a second input of the motion compensated interpolator MCI via a second processing time delay PTD2.

The delay times PTD1 and PTD2 are equal. The delay time of the field memory FM is the delay time PTD1 minus one field. This is between the first and second inputs of the motion estimator and between the first and second inputs of the motion compensated interpolator MCI, respectively, for a total delay time of one field (eg 10 ms). Is obtained.

The block indicated by PTD1 is PTD2.
It further includes a noise compression circuit which receives the input signal from the output of the s and the motion information from the motion estimator. The basic circuit of FIG. 6 is shown in more detail in FIG. In FIG. 6A, the block PTD1 + NR in FIG.
A first line delay LD1 and a processing time delay PTD1
Spatial noise filter SF and temporal noise filter T
F is included in series. The output of the field memory FM is the second line delay LD2 and the second switch SW2.
Of the motion compensated interpolator MCI and the second input of the predictor PRED via PTD2. The second input of the switch SW2 is connected to the output of the field memory FM.

The motion vector MV and, for example, the information regarding the noise level N and the information regarding the reliability CL of the vector are transmitted via the third line delay LD 77 to the motion evaluator M.
From E to the motion compensated interpolator MCI. The output of the line delay LD77 is connected to the first input of the first switch SW1, and the output of the line delay LD77 is connected to the second input of the first switch SW1. Switch S
The output of W1 supplies the motion vector MV to the first input of the predictor PRED. For even output lines EOL, the first input of switch SW1 is operational, while for odd output lines OOL, the second input of switch SW1 is operational. For fields 1 and 2, the first input of switch SW2 is active, while for fields 3 and 4 the second input of switch SW2 is active, ie connected to the output of switch SW2. Has been done.

Similar to FIG. 5, the motion evaluator ME stores a main block motion vector and, in addition, as shown in FIG. 5, a vector memory RAM capable of storing a zero vector of the main block whose function is described below. including. Line delay LD71
To LD77 have the same number and functions as in the case shown in FIG. In FIGS. 6 and 7A, the motion estimator ME receives a non-noise-compressed input signal and a noise-compressed input signal to which a previously calculated motion vector has already been applied, while motion-compensated interpolation The device MCI receives two noise-compressed input signals. Field memory FM
Is the delay of the temporal filter from one field, eg
It has a delay of subtracting two lines.

In FIG. 7B, the output field of the signal O that appears in the case of the input signal I is shown (along with the line number). However, fields F1 and F5
Are both the first fields of the O output frame. The luminance-processed source signal I, 50/60 Hz / 2: 1/625, is 100/120 Hz / 2: 1 by the field memory.
Speed up to / 625 AABB field repeat format. In the timing chart of FIG. 8, the odd (O) / even (E) field F of signal I
1, F3, F5 ,. ． ． And speed up (SPU)
Fields F1, F1, F3, F3, F5 ,. ． ．
And the fields F1, F2, F3, F of the output signal O
4, F5 ,. ． ． It is shown.

The intermediate stage of the above process is proscan conversion, the grid of which is shown in FIG. In the case of the motion estimator ME (FIG. 9A), only the odd source fields of luminance are vertically interpolated in order to place the lines of the odd and even fields on a common vertical grid. In the case of motion compensated interpolators, the source field and both luminance and chrominance proscan grids (FIG. 9B) are generated by a vertical filter.

In FIG. 9, the following signs are used: source grid; lines generated by vertical proscan interpolation; spatial position of output line. F1 is an odd (O) source field, while F3 is an even (E) source field.

FIG. 10A for the odd field F1 and FIG. 10B for the even field F3.
In the motion estimator, both fields are horizontally pre-filtered and sub-sampled as shown in. The grid generated as described above corresponds to the proscan grid sub-sampled in the vertical direction V and the horizontal direction H, as shown in FIG. The horizontal prefilter has a coefficient (1/1
6, 2/16, 3/16, 4/16, 3/16, 2/1
6, 1/16) and is done only on luminance.
Filtering is done by first adding and then truncating.

In FIG. 10, the following designations are used: x source grid; x or o Proscan grid. The value of ◯ is calculated by averaging the value of × vertically; □ Vertically and horizontally sub-sampled proscan grid (= first, third, fifth ,. Target pixel). This is a grid used for motion estimation, confidence and noise level calculations; o Subsampling. The center pixel, indicated by the large circle, is the location of the output pixel.

FIG. 11 shows the main block MB and the sub block SB in the grid of the proscan, which are subsampled in the vertical and horizontal directions. Sub block SB
Has a dimension of 2 * 2 pixels. The main block which is not sub-sampled has a size of 32 pixels * 16 lines in one frame and a size of 32 pixels * 8 lines in one field. The main block has a size of 16 pixels * 8 lines in a proscan grid subsampled in the horizontal and vertical directions.

The motion estimator has the advantage of being able to do double-sided block matching on a large luminance main block indicated by MB. The main block has a size of 16 pixels * 8 lines. Block matching on both sides is described in more detail in the applicant's international patent application PCT / EP94 / 02870. In the motion estimator, each feasible motion vector MV is applied to each main block MB. For each motion vector, the main block of the previous field,
The error E is calculated by accumulating the difference (two's complement) in the absolute value of the corresponding pixel in the current field along the direction of motion. The vector that gives the best match, ie the smallest error, is then selected as the main block motion vector indicated by MBMV.

If there are several equal minimum errors, ie some equally good motion vectors, the motion vectors are chosen in the following order: First motion vector with horizontal component closest to zero. MV 2nd motion vector MV with positive horizontal component MV 3rd motion vector MV with vertical component closest to zero 4th motion vector MV with positive vertical component Motion vector MV is horizontal component MVH and vertical component MVV
Composed of and. The horizontal component is positive when moving across the screen from left to right. The vertical component is positive when moving from top to bottom on the screen. In FIG. 12, the motion vector M is calculated for each next main block MB in the field to be interpolated.
The previous field PF (z-1) for V to generate,
A specific search window S of the current field CF (z)
A two-sided block butt in WW is shown.

As shown in FIG. 13B, due to the subgrid used, some motion vector (-1
4, -10, -6, -2, + 2, + 6, + 10, + 1
In contrast to 4), a small positional error of half the distance of the sub-grid pixel may occur. This small error is acceptable. In FIG. 13A, vertical motion vector components MVV: +2, 0, -2 are shown. In principle, a larger number of values for the vertical motion vector MVV can be realized when a longer line delay is spent. Line delays are expensive (requires a large chip area) and are tested to show that the above three values are sufficient for upconversion purposes.

FIG. 13B shows a horizontal motion vector component MVH that can be realized. The codes used have the following meanings: □ Vertical and horizontal sub-sampled proscan grids; ○ Exact position for calculating motion vectors; ● Slight error in calculating motion vectors Position.

One feature of the present invention is the use of noise measurements to control motion compensated upconversion. An example of a noise measurement that can be used in the present invention is described in the applicant's European patent application 0562407. A further noise measuring method will be described in combination with FIG. For TV standards B and G,
The operational image ACP has a height of 36 main blocks,
For the television standard M, it has a height of 32 main blocks. One central zone PNL0, or, for example,
(Depending on the features of the realizable picture-in-picture)
The three zones PNL1, PNL2 and PNL3 are used to measure noise. First, the minimum main block error for each zone is calculated, then the three minimums are median filtered to obtain the preliminary noise level PNL. In the current source field z: PNL = Median [PNL1, PNL2, PNL3], where PNL is divided by 16: PN _(z) = PNL / 16. The final noise level N _(z) is formed by further median filtering: N _(z) = median [N _(z-1) +2, [PN _(z) + PN
_(z-1) ] / 2, N _(z-1) -2] {truncated}.

Therefore, the noise estimate is updated once per input field, with ±± between fields.
It can be changed with a maximum value of 2. Since N _(z) is not known until the end of the current operational field, it is possible to update N _(z) during the blanking period following the main block motion vector MBMV. N _(z) is used by adaptive temporal noise compression. N for 11-bit word length
_(z-1) is limited to between 2 and 2045 to prevent underflow and overflow when calculating the median value. N _(z) or some of its bits can be read via the I2C bus.

Initialization at power-on is: PNL, P
N _(z) , PN _(z-1) = 0 and N _(z-1) = 2. A further feature of the invention is that it controls motion compensated up-conversion, i.e. detects and takes into account periodic structures that correct the motion vector found by block matching on both sides. An example of the detection of periodic structures that can be used in the present invention is found in the European patent application No.
5732 specification.

Motion estimation is very important in periodic structures. There are some vectors that show very good matches, i.e. very small errors. Therefore, correction of the periodic structure in the horizontal direction is preferably performed.
From the block matching process, the realizable corresponding error of each main block motion vector is known. In theory,
As can be seen from FIGS. 15A to 15D, the horizontal periodic structure gives some error minima within the range of rows but some within the same row. Maximum error occurs. In FIG. 15, the upper row of pixels shows the line n of the source field (z), and the lower row shows the line n of the source field (z−1). (A), (B), (C), and (D) of FIG. 15 are the first of the periodic image luminance signals (for example, fences) shown on the lower side of (A) of FIG. The second, third and fourth feasible motion vectors are represented. Therefore, the current main block is considered to have a periodic structure if there is a large difference between the maximum and minimum error, and if there is a true second minimum. If it is confirmed that the block has a periodic structure, the motion vector belonging to the main block that gives the smallest adjacent error on either the upper or left side is the motion vector found for the current main block. Replace. If they are the same, the vector of the left main block has priority.

For the first, ie upper left main block in the field, the correction of the periodic structure is switched off. For the rest of the rows of the first main block, only the left-hand candidates are available. For the rest of the columns of the first main block, only the upper candidate is obtained. This yields the following algorithm: [(true second smallest row error exists) and (second smallest row error-minimum row error <maximum block error / 2. )] If (if the current block is not the leftmost then main block motion vector MBMV = main block motion vector MBMV _{left then} if the current block is not the top and MBMV _up gives the best match) Main block motion vector MBMV = main block motion vector MBMV _up ) otherwise main block motion vector MBMV = (corresponding to minimum block error) main block motion vector MBMV The second smallest row error is the horizontal direction. Must be a true minimum with greater error on both sides of. Therefore, the row boundaries can be considered to represent a very small error, ie error = zero.

The second smallest row error is used only to determine if the current block is periodic, so if there are several identical second smallest row errors. It doesn't matter which error is finally used. FIG. 16 shows the block C of the current vector range.
An example of a UB, a corresponding left block LEB and an upper block UPB is shown. MVH is a horizontal motion vector component, MVV is a horizontal motion vector component, MVUP
Is the location of the motion vector selected from the upper block, M
VLE represents the location of the motion vector selected from the left block.

MBE locates the error location of the largest block,
Where MRE is the location of the largest row error, 2MRE is the second smallest row error, MINRE is the smallest row error that matches the smallest block error MINBE in the above example, and CHR is the smallest. Represents the selected row corresponding to the erroneous row of the block. Location MVLE and MVU
The arrows starting with P respectively represent the movement vector selected according to the corresponding minimum error CERR from the adjacent blocks.

In the even output field, the horizontal motion vector component MVH has a slight bias towards zero to avoid ambiguity due to noise. The vertical motion vector component MVV is more strongly biased towards zero by the vertical interpolation of one of the two fields used to calculate the motion vector. It is important to avoid the problem of vertical movement, as shown in FIG. 17 with the vertical main block boundary VMBB. An odd interpolated field OIF is formed between the odd source field OSF and the even source field ESF.

For example, B is a black pixel with a value of 16, G is a gray pixel with a value of 128, and W is a white pixel with a value of 240. The main block motion vector MBMV (0,0) is
An error of 16 pixels * value 128 = 2048 from the bottom row of the main block above the vertical main block boundary VMBB. Main block motion vector MBMV (0, +
2) produces zero error.

Vertical zero forcing: if Error (h, 0) ≤ [Error (h, v) +2048] main block motion vector MBMV = h, 0 otherwise main block motion vector MBMV = h, v Horizontal zero coercion (using the above result): if Error (h, 0) ≤ [Error (h, v) + 64 + ZFE + N _(z) / 4] main block motion vector MBMV = 0, v otherwise Main block motion vector MBMV = h, v ZFE = Zero forced even field, ZFE =
0, 1, 2 or 4 are programmable via the I2C-bus. The default is: ZFE = 1. N
_(z) is the above noise level.

Zero forcing of output field 3: Field OF3 is very marginal and causes line flicker if no real zero motion is detected. However, the main block zero vector MBZV is obtained directly from the main block motion vector MBMV because the horizontal and vertical zero forcings for calculating the main block motion vector MBMV are separated.

If (main block motion vector MBMV = 0), main block zero vector MBZV = 0 otherwise main block zero vector MBZV = 1.
Generate motion vectors. Horizontal motion vector component M
There are 16 possible values for VH, which can be encoded with 4 bits. Vertical motion vector component MVV
Has three possible values and can be encoded with 2 bits. A fully motion picture (see Figure 14) yields 792 main blocks per current picture (z) 3.
It has 6 * 22 main blocks, each main block having a word length of 6 bits and a vector memory RAM of the motion evaluator ME.
Stored in. The vector memory stores the value of the main block zero vector MBZV for the calculation of the output field OF3.

The next step is, for example, for smaller areas known as sub-blocks having dimensions of 2 * 2 pixels on the sub-sampled proscan grid or 4 * 4 pixels on the proscan grid. Using the above information to calculate the unique motion vector. An example of sub-block motion vector calculation is described in more detail in European Patent Application No. 94113494. The advantage derived from it is that it is calculated just before the sub-block motion vector is needed, so it does not have to be stored and a small amount of RAM is available in the motion estimator. The main block vector is calculated in the first field and the sub block motion vector is calculated in the next field.

By applying the three spatially closest adjacent main block motion vectors to the particular sub-block and calculating the three corresponding sub-block errors SE, three motion vectors for a particular sub-block are obtained. One of the two spatially closest adjacent main block motion vectors is selected. In FIG. 18, the main block C
Is divided into four areas. The candidate motion vector of the sub-block in the upper left area is selected from the motion vectors of the closest main blocks A, B and C. The candidate motion vector of the sub-block in the upper left area is selected from the motion vectors of the closest main blocks A, C and D. The candidate motion vector of the sub block in the lower left area is
Selected from the motion vectors of the closest main blocks B, C and E. The candidate motion vector of the sub-block in the lower right area is selected from the motion vectors of the nearest main blocks C, D and E.

The error of the sub-block is calculated by accumulating the difference d _i of four pixels in the area of the sub-block SB between the field (z) and the field (z−1) as shown in FIG. SE (x, y) = | d ₁ | + | d ₂ | + | d ₃ | + | d ₄ |, where x and y are one of the three main block motion vectors. Represents an ingredient.

The main block motion vector MBMV (x, y) that yields the smallest error SE (x, y) has been preselected for the current block. Then, a 3-tap horizontal median filtering operation is performed on each component SBMVH and SBMVV of the sub-block motion vector to eliminate the erroneous vector. The obtained motion vector is used to calculate the pixels of the even output field belonging to the subblock. x and y are
These are the positions of the sub-blocks in the horizontal and vertical directions, respectively.

It is an advantage that the calculated sub-block error SE directly represents the reliability of the motion vector. When the sub-block error SE has a small numerical value, the reliability CL is large, and when the sub-block error SE has a large numerical value, the reliability CL is small. Applicant's European patent application No. 94115733 describes the measurement of the reliability in more detail.

The reliability CL can be calculated by the motion estimator ME according to the following formula: CL "= median [SE _(x-1) / 2, SE _(x) / 2, SE _{(x + 1) )} / 2] CL = CL ″ / CLdiv {truncated} The reliability CL is limited to, for example, 8 bits. CLd
iv is, for example, a preselected multiple 1, 2, 4, 8 programmable by the I2C bus. The default value is: CLdiv = 2.

A horizontal median filter is applied to ensure that if the confidences are good on both sides of the block boundary, they remain good across the boundaries, while the unfiltered confidence is not good. . Due to the soft switches used in the chrominance processing, it is possible to calculate the chroma confidence CC in the motion estimator ME: C
C = CL / 2.

Further, the reliability CL and the temporal noise compression on / off signal TNRS irrespective of the noise level N are
Generated by the motion estimator ME as follows: if (CL * CLdiv <2 * N) temporal noise compression on / off signal TNRS = 1 otherwise temporal noise compression on / off signal TNRS = A further feature of the invention is noise compression. For noise compression, there are two stages of noise compression, that is, spatial noise compression SNR.
And there is a temporal noise compression TNR. Spatial noise compression is preferably performed only on luminance, but it is also applicable to chrominance, while temporal noise compression is preferably performed on both luminance and chrominance. FIG. 20 shows a common principle for noise compression in the embodiment according to FIGS. 1 to 7A. No additional processing time or line delay is shown. The speed-up input signal ISPU is first processed by the spatial noise compression SNR means. However, the order of the spatial noise compression SNR and the temporal noise compression TNR may be reversed.

The spatial noise compression SNR is the noise level N
When N is low, eg N ≦ 4, it is bypassed to avoid possible resolution degradation.
The N is obtained from the motion evaluator ME. Spatial noise compression is enabled / disabled via the I2C bus. The default is operational. The spatial noise compression filter is constituted by a horizontal and / or vertical and / or diagonal low pass filter, or
It may be constituted by a dimensional or directional median filter. For directional filtering, the filter direction is
It is a function of the correlation of the central pixel with the corresponding neighboring pixel. The filter operates, for example, in a window of 3 * 3 pixels and reduces impulse noise but preserves textures, details and edges. The filtering result corresponding to the direction of least correlation is selected as the final output pixel value.

Referring to FIG. 20, the output of the spatial noise compression passes through the temporal noise filter TNR, and the output of the temporal noise filter TNR is input to the motion-compensated interpolator MCI. Basically, a temporal filter combines data from various fields, including the current field. Advantageously, the temporal filter TF (luminance and chrominance) may be an iterative filter with non-linear coefficients. The predictor PRED, the interpolator MCI and the field delay FM are connected behind the temporal filter TF. The temporal filter TF is controlled by the temporal noise filter TNR and the noise level N. The predictor PRED is a field memory FM.
, And the sub-block motion vector SBMV from the motion estimator ME. For luminance, the spatially and temporally noise-compressed source field (z-
The predicted sub-block from 1) is considered in the temporal filter TF together with the sub-block of the spatially noise-compressed source field of the field (z) resulting from the spatial noise compression. The output signal of the temporal noise compression TNR consists of spatially and temporally noise-compressed sub-blocks of the source field (z).

If an erroneous motion vector is calculated due to noise, the motion estimator ME uses a motion-compensated signal constructed with the previously calculated motion vector, and thus iterative temporally. The filter propagates the error with respect to the partially erroneous video information at the erroneous spatial location. Therefore, the calculation of the motion vector which continues in time may be wrong. Due to the coefficients applied to the predicted video information in temporal noise compression, the effects of such disturbances are attenuated from field to field. Careful testing has shown that in practice such errors do not propagate and therefore the cheaper motion compensated upconversion MCU circuit of the present invention can be used.

The coefficients allow at most only about 3/4 of the predicted video signal to be added to the video signal of the current field. For example, in the case of a noisy source signal, if the confidence CL of the motion vector is lower, this part is reduced, so that the possible error propagation is further reduced. The motion-compensated interpolator MCI outputs the signal SBM from the motion estimator ME.
MB, V, N, CL, CC and MBZV
The main block zero motion vector from the last motion estimate of the source field (z-1).

Temporal noise compression on / off signal TNRS
The luminance processing of is shown below: If (TNRSL operational [via I2C-bus]) and (TNRS = 1) temporal noise compression luma is switched on otherwise blur due to evaluation failure. Temporal noise compression luma is bypassed to avoid blurring Default value = TNRSL operational If (TNRSC forced state [via I2C-bus]) or (TNRS = 1) temporal noise compression chroma Is switched on otherwise temporal noise compression chroma is bypassed Default value = TNRSC forced state The predictor PRED uses the sub-block motion vector SBMV to project the previous field in the motion direction, and Lost of non-interlaced images due to interlacing Interpolation, for example vertical averaging, is performed to generate the pixels.

The temporal luminance filter consists of the spatially noise-compressed current field (z) and the spatially and temporally noise-compressed last source field (z-1) output from the predictor. Receive. A combination of both,
Preferably, the non-linear combination is a motion compensated interpolator MC.
Results in I temporally noise-compressed luminance input fields.

The temporal chrominance filter receives the current field (Z) and the temporally noise-compressed last source field (z-1) output from the predictor. Both combinations, preferably non-linear combinations, result in a temporally noise-compressed chrominance input field due to the motion compensated interpolator MCI. The motion compensated interpolator MCI produces four output fields for every two input fields. Odd first output field O
F1 is the noise-compressed source field F1 as shown in FIG. F1 'is the first field from the next input source frame.

21 to 26, x represents an input pixel, □ represents an output pixel, CH represents a chroma source pixel, CI represents a horizontally interpolated chroma pixel, and CO represents a chroma output pixel. It represents. FIG. 23
(B) to (B) of 26, due to the block matching on both sides, the chroma pixel CI is required to have its position not within the multiple of 4 grid used for the chrominance predictor PRED. Be careful of the points.

The processing of the third output field OF3 is very special as shown in FIG. The noise-compressed source field F3 grid does not match the output field OF3 grid due to interlacing and up-conversion. Therefore, the output field OF3
Main block zero vector MBZ for constructing the pixel values of
Depending on the value of V, two distinct filter types, eg vertical filters with coefficients (1/2, 1/2),
One of the vertical-temporal median filters is applied as follows: if main block zero vector MBZV (z-1) = 0 vertical-temporal median filter else vertical filter Expressed in dashes The bounded area indicates the pixels associated with the vertical-temporal median filter. Region 0 shows the pixels associated with the vertical filter.

For each pixel in the even output fields OF2 and OF4, a motion-compensated pixel and a fallback (f
all back) Pixels are calculated. The final luminance output is a combination of the value of the motion-compensated pixel by the soft switch shown in FIG. 27 and the value of the fallback pixel, depending on the reliability CL. Advantageously, different processing modes can be used for the calculation of the even output fields OF2 and OF4, and the modes are made selectable via the I2C bus.

Mode 1 (default): A vertical temporal median filter is used as described with respect to FIG. The pixels of the output field OF2 are constructed by taking the vertical temporal median (applied to the direction of motion) of the noise-compressed fields F1 and F3. In other words, the output pixel has a horizontal vector component M as shown in FIG.
It is the median value of the pixels f, g, and h as shown in FIG. In (B) of the figure, the output line n (OLn) of the field F2 is (horizontal direction) between the lines n and n + 2 (Lnn + 2) of the field F3 and the line n-1 (Ln-1) of the field F1. -In the time range).

The pixels of the output field OF4 are constructed by taking the vertical temporal median (applied to the direction of motion) of the noise-compressed fields F3 and F5. In other words, the output pixel has the median value of the pixels i, j, and k as shown in (A) of FIG. 24 for the vertical vector component MVV and (B) of the figure for the horizontal vector component MVH. is there. In (B) of the figure, the output line n (OLn) of the field F4 is (in the horizontal direction-temporal range) between the line n (Ln) of the field F5 and the line n (Ln) of the field F3. It is shown.

The fallback pixels of the output fields OF2 and OF4 are calculated in the same way as the motion-compensated pixels above, but zero motion, ie the sub-block motion vector SBMV = (0,0), is assumed. Fallback pixels are described in European Patent Application No. 941156 by the applicant.
No. 34 can be calculated. Mode 2: A linear averaging filter is used as described with respect to FIG. The output field OF2 is a linear average of the noise-compressed fields F1 and F3. In other words, the output pixel is the vertical vector component M
FIG. 25A shows a horizontal vector component MVH for VV.
On the other hand, as shown in FIG.
Moreover, it is 1/2 of the pixel gh. In FIG. 6B, the output line n (OLn) of the field OF2 is (horizontal direction-time) between the line n + 1 (Ln + 1) of the field F3 and the line n-1 (Ln-1) of the field F1. (To the extent possible).

The output field OF4 is a linear average of the noise-compressed fields F3 and F5. In other words, the output pixel is the vertical vector component MVV.
On the other hand, in FIG. 26A, the horizontal vector component MVH is 1/2 of the pixel i and 1/2 of the pixel jk as shown in FIG. In (B) of the same figure, the output line n (OLn) of the field OF4 is (in the horizontal direction-temporal range) between the line n (Ln) of the field F5 and the line n (Ln) of the field F3. It is shown.

Similar to mode 1, in mode 2 the fallback pixels of the output fields OF2 and OF4 are calculated in the same way as the motion compensated pixels above, but with zero motion, ie the sub-block motion vector SBMV =
(0,0) is assumed. The circuit of FIG. 27 is provided at the output of the motion compensated interpolator. The pixel OOFP in the odd output field is supplied to the first input of the switch SWF. The motion compensated pixel MCP is a soft switch S
It is supplied to the first input of SW. The fallback pixel is supplied to the second input of the soft switch SSW and the confidence value CL is supplied to the third input of the soft switch SSW. The output signal of the soft switch SSW is (0 ...
To the second input of the switch SWF via limiting means LIM3 (limited to 255). For odd output fields, the first input of the switch SWF is the control signal OF.
Is used to connect to the output OUP. For even output fields, the second input of the switch SWF is connected to the output OUP.

The soft switch SSW can be constructed in the following way: The signal from the first input is subtracted from the signal at the second input in the subtraction means SUB.
The output is supplied to the first input of the minimum value calculation means MIN. The reliability signal CL from the third input is supplied to the second input of the minimum value calculation means MIN and is supplied to the two's complement means 2CO.
Is supplied to the second input of the maximum value calculating means MAX. The first input of the maximum value calculation means MAX receives the output signal of the minimum value calculation means MIN. Maximum value calculation means MAX
In the coupling means ADD is added to the input signal of the first input of the soft switch SSW and the result is supplied to the output of the soft switch SSW.

Chrominance Processing Basically, the chrominance (U and V) processing is performed according to the luminance processing. Unless otherwise noted, the same block diagram is used for chrominance processing. Preferably, there is at least one of the following differences: 1) The horizontal component MVH of the motion vector is rounded to the nearest multiple of 4 before being applied to chrominance to match the sampling grid of chrominance. The table below shows that the components of the motion vector obtained from the motion estimator ME are luma and chrominance, so the predictor PRED
And the scheme applied to the motion compensated interpolator MCI:

[0083]

[Table 1]

2) To allow point 1), an additional horizontal linear averaging filter is required in the motion compensated interpolator MCI. Therefore, a 4: 2: 2 format is generated from a 4: 1: 1 format before the motion vector is applied to the chrominance pixels. 3) No spatial noise compression SF is performed.

4) Interpolation of the even output fields is performed as in mode 2 for luminance (linear averaging, no programmability). 5) Chrominance confidence CC (used only for chroma of softswitch SSW) is half of luminance confidence CL: CC = CL / 2 6) with linear averaging (1/2, 1/2) 4: 2: 2
→ Convert 4: 4: 4.

FIG. 28 shows the input and output of the speed-up field memory SPUFM and the field memory F.
The overall timing charts for the M inputs and outputs and the output signals of the motion estimator ME and the motion compensated interpolator MCI are shown. F1NR ,. ． ． ． ,
F5NR represents a noise-compressed field.

A special calculation is performed on F3NR ".
As described above, when the value of the main block zero motion vector MBZV is 0, the field F that has not been noise-compressed yet.
A vertical temporal median filter with access to 5 is applied. Since the sub block motion vector SBMV and the reliability CL are unknown, the main block zero vector MBZV (z-1) is applied. This causes a small temporal error. However, tests have shown that this error is negligible.

The circuit shown in the present invention can be integrated on one or a plurality of chips and can be controlled by the I2C bus. Some of the circuitry may be replaced by appropriate software and processor means.
The invention can be used in standard definition of 4: 3 or 16: 9 or in high definition devices such as televisions, satellite receivers, video cassette recorders and video disc players. is there.

In the case of a digital video signal, motion information may be added. In such cases, the present invention may utilize pre-existing motion information, thereby omitting the processing steps associated with the generation of motion information, or adapting according to pre-existing motion information.

[Brief description of drawings]

FIG. 1 is a block diagram of a first embodiment of a motion compensated upconverter.

FIG. 2 is a block diagram of a second embodiment of a motion compensated upconverter.

FIG. 3 is a block diagram of a third embodiment of a motion compensated upconverter.

FIG. 4 is a block diagram of a fourth embodiment of a motion compensated upconverter.

FIG. 5 is a block diagram of a fifth embodiment of a motion compensated upconverter.

FIG. 6 is a basic block diagram of a motion-compensated up-converter with two field memories.

FIG. 7 is a basic block diagram of a motion-compensated upconverter with two field memories and noise compression.

FIG. 8 is a timing chart of main steps of up-conversion performed in FIGS. 2 to 7.

FIG. 9 is an explanatory diagram of a proscan grid generated by a motion-compensated interpolator and an evaluator.

FIG. 10 is an explanatory diagram of a horizontal prefilter and sub-sampling.

FIG. 11 is an explanatory diagram of main blocks and sub blocks.

FIG. 12 is an explanatory view of the principle of the both-side block butting.

FIG. 13 is an illustration of small positional error for some motion vectors obtained from subgrids used in double-sided block matching.

FIG. 14 is an explanatory diagram of a noise measurement zone.

FIG. 15 is an explanatory diagram of vectors that can be realized in a periodic structure.

FIG. 16 is a detailed view of a current vector block and adjacent vector blocks.

FIG. 17 is an explanatory diagram of a vertical motion vector component and vertical movement.

FIG. 18 is an illustration of a sub-block region and associated main block in which motion vectors are used to generate sub-block motion vectors.

FIG. 19 is a diagram showing a three-dimensional grid together with an error of a sub block.

FIG. 20 is a simplified block diagram of a noise compressor.

FIG. 21 is an illustration of a first output field of a motion compensated interpolator.

FIG. 22 is an explanatory diagram of a third output field of the motion-compensated interpolator.

FIG. 23 is an illustration of a second output field of the motion compensated interpolator during the first mode.

FIG. 24 is an illustration of a fourth output field of the motion compensated interpolator during the first mode.

FIG. 25 is an illustration of a second output field of a motion compensated interpolator during a second mode.

FIG. 26 is an illustration of a fourth output field of the motion compensated interpolator during the second mode.

FIG. 27 is an explanatory diagram of soft switches and even / odd selection.

FIG. 28 is an overall timing chart.

[Explanation of symbols]

2CO 2 Complement Means 2MRE Second Minimum Row Error ACP Motion Image ADD Combiner CC Chrominance Confidence CERR Minimum Error CH Chroma Output Pixels CHR Selected Row Corresponding to Errored Row of Minimum Block CI Interpolated Chroma Pixel CL Luminance Confidence CO Chroma Output Pixel CUB Current Block CUF Current Field d ₁ , d ₂ , d ₃ , d ₄ Filtering Direction EOL Even Output Line ESF Even Source Field FBP Fallback Pixel F1, F3 Noise Compressed source field FM, FM1, FM2 field memory I input ISPU speedup input signal LD11 ,. ． ． , LD16, LD21 ,. ． ． , LD
28, LD31 ,. ． ． , LD37, LD4
1 ,. ． ． , LD47, LD51 ,. ． ． , LD57
Line delay LEB Left block LIM3 Limiting means MAX Maximum value calculating means MB Main block MBE Maximum block error location MBMV Main block motion vector MCI Motion compensated interpolator MCP Motion compensated pixel ME Motion estimator MIN Minimum value Calculation means MINBE Minimum block error MINRE Minimum row error MRE Maximum row error location MV Motion vector MVH Horizontal motion vector MVLE Left block motion vector location MVUP Upper block motion vector location MVV Vertical motion vector N Noise level NR Noise compression O, OUP Output OF Control signal OF1, OF2, OF3, OF4 Output field OIF Odd interpolated field OLn Output line OOFP Odd output field pixel OOL Number Output Line OSF Odd Source Field PF Previous Field PNL0, PNL1, PNL2, PNL3 Center Zone PRED Predictor PTD1, PTD2 Delay RAM Vector Memory SB Subblock SBMV Subblock Motion Vector SF Spatial Noise Compression Filter SE Subblock Error SNR Spatial noise compression SPUFM Speed-up field memory SSW Soft switch SUB Subtractor SW1, SW2, SWF switch SWW Specific search window TF Temporal filter TNR Temporal noise compression TNRS Temporal noise compression ON / OFF signal UPB Upper block VMBB Vertical Orientation main block boundary

Claims (27)

[Claims] 1. An interlace in which a field of source field frequency is transformed to form a field of double field frequency, motion estimation is performed, and motion compensated interpolation is performed on the field of double field frequency. A method of motion-compensated up-conversion of a video signal of the method, wherein in the motion estimation, a motion vector of a main block and a motion vector of a sub-block smaller than the main block are
The pixels calculated for one input field and the pixels required for this calculation are stored in the line storage means of one input field, while the motion-compensated interpolation is performed in the sub-blocks of the subsequent output fields. A method characterized by being performed on a motion vector. 2. The motion vector of the main block is calculated and stored in the storage means of one output field and the motion vector of the sub-block is then stored for the subsequent output field of the stored main block. The method of claim 1, wherein the method is calculated from motion vectors. 3. Spatial noise compression prior to said motion-compensated interpolation, in particular using at least some of said line storage means together with said spatial noise compression, at a field of said double field frequency. Method according to claim 1 or 2, characterized in that it is carried out above. 4. Method according to claim 1, characterized in that temporal noise compression, in particular iterative temporal noise compression, is included in the motion-compensated interpolation. 5. To perform the motion-compensated interpolation, the motion vectors of the sub-blocks are, for each output sub-block vector, the three main sub-blocks closest to the current sub-block including the current main block. Method according to any one of claims 1 to 4, characterized in that it is calculated from the motion vectors of the main block by selecting a vector from the motion vectors of the main block in the block. 6. The spatial noise compression is performed only on luminance, and the temporal noise compression is luminance and / or
Alternatively, the method is performed on chrominance. 7. The method according to claim 1, wherein the motion-compensated interpolation is performed on luminance and chrominance. 8. Four output fields are generated during said motion compensated interpolation of each pair of source fields: a first output field is obtained from the first source field in the current pair. The second output field is the first in the current pair
And a second source field combination; the third and fourth output fields are a combination of the second source field in the current pair and the first source field in the next pair. 8. A method according to any one of claims 1 to 7, characterized in that it is present. 9. Method according to claim 1, characterized in that during the motion estimation, a Proscan transform is used for every other source field. 10. The method of claim 9, wherein during the motion estimation, the proscan transform is based on a grid of horizontally subsampled pixels. 11. Method according to claim 1, characterized in that a block matching on both sides is performed during the motion evaluation. 12. Method according to claim 1, characterized in that during the motion estimation a detection of the periodic structure used for correcting the motion vector is carried out. 13. The method of claim 1, wherein during the motion estimation one or both components of the motion vector are forced closer to zero depending on the calculated noise level.
13. The method according to any one of 1 to 12. 14. Median filtering in the horizontal or vertical direction is performed on one or both components of the motion vector of the sub-block.
The method according to any one of 3 above. 15. The method according to claim 5, wherein the reliability of the motion vector of the sub-block is calculated from the error of the sub-block, and the chrominance reliability of the chrominance depending on the reliability can be further calculated. Method described in section. 16. The method of claim 15, wherein the spatial and / or temporal noise compression is controlled by the confidence and chrominance confidence, respectively. 17. Method according to claim 3, characterized in that the spatial noise compression comprises a directional filter, in particular a median filter. 18. A method according to any one of claims 4 to 17, characterized in that the temporal noise compression comprises a motion-compensated predictor of each pixel from the previous field. 19. The iterative temporal noise compression is spatially noise compressed current field and spatial and / or temporal noise compression obtained from the predictor,
19. The method according to claim 18, characterized in that a non-linear combination of pixels of the last motion-compensated source field is performed. 20. Zero of the main block controlling the type of one of which is used to construct the third output field among at least two distinct interpolation filter types containing temporal components. A method according to any one of claims 8 to 19, characterized in that the vector is calculated from the block matching error obtained by the motion estimation. 21. In order to construct said second and fourth output fields which are respectively controlled by said reliability and chrominance reliability, vertical temporal interpolation in both said second and fourth output fields. , Or a combination of vertically interpolated pixels and a fallback pixel is used, in which case the fallback pixel is used without motion compensation. The method according to claim 1. 22. The predictor uses chrominance pixels that are sub-sampled horizontally by a factor of 4 with respect to the luminance pixels.
The method according to any one of 1. 23. To construct said second and fourth output fields which are respectively controlled by said reliability and chrominance reliability: interpolated only in the vertical direction in both said second and fourth output fields. 23. Method according to claim 21 or 22, characterized in that a combination of the following pixels is used: fallback pixels, and the fallback pixels are used without motion compensation. 24. An apparatus for motion-compensated up-conversion of an interlaced video signal, the method comprising: calculating a motion vector of a main block and a motion vector of a sub-block smaller than the main block for one input. Second storage means for receiving an input field, the input and output of which are connected to a motion estimator for performing a motion evaluation from field to field, the line storage means storing pixels used for the purpose; A first subsequent storage means for converting the output field of the second storage means into a field of double field frequency; and-receiving a motion vector of the sub-block from the motion estimator and storing it in the next output field. Field-to-field motion compensation is performed on the output signal obtained from the first storage means to obtain motion-compensated upcoding. Device according to any one of claims 1 to 23, which comprises a motion-compensated interpolator for generating an inverted output signal, and a third storage means connected to its input and output. . 25. A device for motion-compensated upconversion of an interlaced video signal, comprising: first storage means used for converting a source field into a field of double field frequency; A motion estimation for performing a motion estimation from field to field, including line storage means for storing pixels used for calculating a motion vector of a main block and a motion vector of a sub-block smaller than the main block for one input field. A second subsequent storage means having its input and output connected to the input device; and its input and output receiving the motion vector of the sub-block from the motion estimator and the second output field in the next output field. Motion compensated from field to field on the output signal obtained from the output of the storage means of Device according to any one of claims 1 to 23, comprising a third storage means connected to a motion compensated interpolator for generating an upconversion output signal. 26. A device for motion-compensated upconversion of an interlaced video signal, comprising: first storage means used for converting a source field into a field of double field frequency; The motion vector of the main block is calculated for one output field and stored in the storage means, and then the motion vector of the sub block smaller than the main block is calculated from the motion vector of the main block for the next output field. A subsequent second storage means, the inputs and outputs of which are connected to a motion estimator that performs field-to-field motion estimation; and-the next output field that receives the motion vector of the sub-block from the motion evaluator. Motion compensation is performed from field to field on the output signal obtained from the output of the second storage means, 24. A method as claimed in any one of claims 1 to 23, comprising a motion compensated interpolator for generating the motion compensated upconversion output signal and a third storage means having its input and output connected. Equipment related to. 27. A device for motion-compensated up-conversion of an interlaced video signal, comprising: first storage means used for converting a source field into a field of double field frequency; The input signal of the second storage means is obtained from the output signal of the first storage means, receives the motion information from the motion evaluator, and is the field on the output signal obtained from the output of the second storage means. To a first input of a motion-compensated interpolator for performing motion compensation in the field to produce a motion-compensated up-conversion output signal, the signal at the second input of the motion-compensated interpolator being A subsequent second storage means obtained from the output signal of the second storage means; and-receiving an output signal of the second storage means at a second input,
The signal at the first input is obtained from the output signal of the first storage means, calculates the motion vector of the main block for one output field, stores it in the storage means and then for the next output field. 3. A motion estimator for performing motion compensation from field to field by calculating a motion vector of a sub-block smaller than the main block from the motion vector of the main block.
An apparatus related to the method according to any one of 3.