JPH0562877B2 - - Google Patents

Description

[Detailed Description of the Invention] [Industrial Application Field] This invention detects a motion vector indicating the amount and direction of movement of a television camera when the entire shooting screen moves due to panning or the like of the television camera. The present invention relates to a motion detection device for television signals applied to.

[Conventional technology]

One method of narrowing the transmission band of high-definition television signals is to shift the subsampling position every 4 fields so that the amount of data is reduced by 1/2.
A motion vector indicating the movement of the television camera is transmitted together with the image data reduced to 4, and on the receiving side, the data of the period of 4 fields is used to reproduce the screen of 1 field for the still image.
When the entire screen moves in the same direction, such as when a television camera pans, some devices perform motion correction by shifting the coordinate axes according to the motion vector and reading the previous frame data from memory.

The number of motion vectors used for such motion correction may be one per field. Conventionally, in order to improve estimation accuracy, the screen was divided into 4
The motion vectors for the entire screen are obtained by dividing the screen and subjecting the plurality of motion vectors obtained in each region to a majority vote or simple arithmetic averaging.

A block matching method is used to obtain motion vectors for each area in which one screen is divided into multiple areas. Furthermore, the screen is divided into a large number of blocks for simplification using representative points, frame differences for motion vector detection are determined for each block, and the strength of correlation is investigated from the absolute value of this frame difference. The frame difference is, for example, the difference between the spatially central pixel (representative point) of the block of interest in the previous frame and each pixel included in the block of interest in the current frame. By integrating the absolute value of this difference for each of the four regions of one screen, a table of frame difference integral data for each region is obtained. The position of the minimum value in this frame difference integral table is detected as the motion vector of that area.

[Problem that the invention seeks to solve]

Conventional motion vector detection devices for television cameras have the following drawbacks.

First, the reliability of the motion vector determined from the frame difference integral table for each area differs depending on the picture in that area. For example, in a region with many low frequency components such as the sky or the sea, the distribution near the minimum value in the frame difference integral table is gentle, and the reliability of the motion vector is poor. On the other hand, in areas with many high-frequency components of fine patterns, the distribution is steep and the reliability of the motion vector is high. Conventional devices did not take into account such differences in reliability depending on the pattern.

Second, since each region is affected differently by noise, moving objects, etc., the motion vector detection accuracy for each region is different. In other words, in a region with little noise or moving objects, the frame difference is minimum at the position of the motion vector for most pixels, and a distribution with a single extreme value is obtained. On the other hand, in areas where there is a lot of noise or moving objects, the distribution of frame differences has multiple extreme values, and the accuracy of motion vectors detected in such areas is low. Conventional devices do not take into account the effects of such noise and moving objects.

The purpose of the present invention is to eliminate the drawbacks of conventional motion vector detection devices, and to determine the reliability of motion vectors determined in each region and their reliability, and to determine a motion vector that is appropriate as a motion vector for the entire screen. An object of the present invention is to provide a motion vector detection device that can detect motion vectors.

[Means for solving problems]

This invention divides the screen 56 of one field to form a plurality of areas 57A to 57D.
Divide each of A to 57D into a plurality of blocks, and calculate the absolute value of the frame difference between the representative point pixel data in the block of the previous frame, the representative point pixel data in the block of the current frame, and the corresponding pixel data in a predetermined range. means for calculating the absolute value of the frame difference for each predetermined range in each region, and detecting the motion vector of each of the regions 57A to 57D from the frame difference integral data; and distribution of the frame difference integral data. The reliability of each of the motion vectors V1 to V4 in the regions 57A to 57D is determined based on the assumption that the steeper the slope between the minimum value and its neighborhood, the higher the reliability, and the first weight corresponding to this determination is determined. Means for calculating coefficients α ₁ to _{α 4} 6-10, 16-20, 26-30, 36-4
0, 49 to 52, and the respective motion vectors V of the plurality of regions 57A to 57D from the minimum values H1 to H4 of the frame integral data.
1 to V4, the degree of residual deviation is determined, and the second
Means 75 to 78, 8 for calculating weighting coefficients β ₁ to β ₄ of
1 to 87, first weighting coefficients α ₁ to _{α 4} and second weighting coefficients β ₁ to
Means 61 to 64, 91 to 64 for calculating the third weighting coefficients γ1 _{to γ4} actually used from _β4 , and the third weighting coefficients _γ1 to _γ4 are used in each region 57A to γ4.
95-98, 105-107 for calculating a motion vector V of one field by weighted averaging motion vectors V1-V4 for each 57D;

[Effect]

Regarding the distribution of frame difference integral data for each of the regions 57A to 57D, when the pattern in that region has a low frequency, the slope between the minimum value and its vicinity is gentle; on the other hand, when the pattern in that region has a high frequency In the case of , the slope between the minimum value and its vicinity becomes steep. The steeper the slope of the minimum value corresponding to the motion vector, the higher the detection accuracy. Therefore, the first weighting coefficient α ₁ ~ whose weight increases as the slope becomes steeper.
_α4 is formed.

Further, since the magnitude of the minimum value becomes the residual deviation after motion correction, a second weighting coefficient is formed in which the weight increases as the minimum value decreases, and the influence of noise, moving objects, etc. is taken into account.

Motion vectors V1 to V of each area 57A to 57D
The third weighting coefficients γ ₁ to _{γ 4} that are actually used for weighted averaging of 4 are obtained by combining the first weighting coefficient and the second weighting coefficient by weighted averaging.

〔Example〕

An embodiment of the present invention will be described below with reference to the drawings. In this embodiment, the motion vector for the entire field is formed by weighted averaging of the motion vectors obtained for each area obtained by dividing the screen of one field into four equal parts. The weighting coefficient γ for this weighted average is formed by simply averaging the first weighting coefficient α and the second coefficient β. The first weighting coefficient is determined by determining the reliability of the motion vector of each region from the type of distribution of frame difference integral data in the table of frame difference integral data of each of the four regions. The second weighting coefficient is determined in accordance with the degree of fitting of the motion vectors of the four regions.

FIG. 1 shows a configuration for forming the above-mentioned first weighting coefficient and second weighting coefficient.
The minimum value H1 of the frame difference integral data included in the first region of one field is supplied to the first field. Input terminals 2, 3, 4, and 5 are each supplied with frame difference integral data H2, H3, H4, and H5 of the first region adjacent above, below, and to the left and right of the minimum value of the first region.

Each of the input terminals 11, 12, 13, 14, and 15 receives the minimum value H11 of the frame difference integral data included in the second region of one field, and the surrounding frame difference integral data H12, H13, H14, and H15. is supplied. Input terminals 21, 22, 23, 24,
25, the minimum value H21 of the frame difference integral data included in the third region of one field and the surrounding frame difference integral data H22, H23, H24,
H25 is supplied. Input terminals 31, 32, 3
3, 34, and 35 each have the 4th field of 1 field.
Minimum value of frame integral data included in the area of
H31 and surrounding frame difference integral data H32,
H33, H34, and H35 are supplied.

Figure 2 shows the frame difference integral data table for one block expressed in X-Y coordinates (x, y).
is the coordinate of the minimum value of the frame difference integral data.
Since the vertical spacing between pixels of a digital television signal is twice the horizontal spacing, in order to equalize the spacing between the pixels on the top, bottom, left and right, (x, y+1) and ( x,y
The vertically adjacent frame difference integral data of each coordinate position of -1) and the frame difference integral data of the left and right sides of each coordinate position of (x-2, y) and (x+2, y) that are one space apart are selected. That is, frame difference integral data
H1, H11, H21, H31 are the coordinates (x, y) of the minimum value of each area, and H2, H12, H22, H32 are the frame difference integral data at the position (x, y + 1).
H3, H13, H23, H33 are frame difference integral data at position (x, y-1), H4, H14, H24,
H34 is the frame difference integral data at the position (x+2, y), H5, H15, H25, H35 are (x-2, y)
This is frame difference integral data at the position.

The position (x, 1) of the minimum value of the frame difference integral data in each area is usually not far apart.

Here, the formation of the above-mentioned frame difference integral data and detection of a motion vector will be explained with reference to FIGS. 3, 4, and 5.

In FIG. 3, reference numeral 110 indicates an input terminal for a digital television signal captured by a television camera and digitized. This digital television signal is supplied to a representative point extraction circuit 111 and a block data extraction circuit 112, and the pixel data of the representative point obtained from the output of this representative point extraction circuit 111 is supplied to a representative point memory 113. The signal is supplied to this representative point memory 113. The pixel data of the representative point read from the representative point memory 113 is that of the previous frame.

The block data extraction circuit 112 extracts pixel data for each block included in the current frame and supplies it to the subtraction circuit 114. This subtraction circuit 114 calculates the difference between each pixel in the block and the representative point of the previous frame, that is, frame difference data. This frame difference data is supplied to a conversion circuit 115 and converted into an absolute value.

As an example, the size of one block is 32 samples in the horizontal direction and 8 lines in the vertical direction, as shown in FIG. Therefore, one block contains 256 pixels. The representative point is 1
This is pixel data located at the spatial center of the block.

The absolute value of the frame difference data from the conversion circuit 115 is supplied to the integration circuit 116, and the integrated value of the frame difference data is supplied to the switch circuit 117.
As shown in FIG. 5, the frame difference data is an area 57 formed by dividing the screen 56 of one field into four parts.
It is obtained for each of A, 57B, 57C, and 57D. The integrated value of the frame difference data from the integrating circuit 116 is distributed by the switch circuit 117 to the graph memories 121, 122, 123, and 124 for each area. The graph memory 121 has a capacity of one block and stores the absolute value of the frame difference data (frame difference integral data) of each block included in the area 57A. Gram memory 122, 12
Similarly, areas 57B and 57
Store frame difference integral data of C and 57D.

The frame difference integral data stored in each of these graph memories 121 to 124 is supplied to minimum value detection circuits 131, 132, 133, and 134, respectively. Minimum value detection circuits 131 to 134
is to detect the position of the minimum value in the frame difference integral data of the corresponding graph memories 121 to 124, that is, the motion vector of each of the regions 57A to 57D. Since the input digital television signal is input continuously, the graph memory 121
-124 each have two memory banks, and in a field in which frame difference integral data is written in one of them, a motion vector is detected from the frame difference integral data in the other field.

In FIG. 5, the case where the four motion vectors are the same, as shown by the solid line, is an ideal case. Actually, the pattern of the image (low frequency pattern,
The degree of reliability of the motion vector determined for each region differs due to the influence of the moving object and the difference in whether the image is a high-frequency image or a moving object. Also, in Figure 5,
The motion vectors shown by broken lines are those when the television camera zooms up, and at this time, four motion vectors spread out radially from the center.

Weighting coefficients γ ₁ to _{γ 4} that reflect the above-mentioned differences in the degree of reliability of motion vectors for each region are generated by the weighting coefficient generation circuit 120. The weighting coefficients γ ₁ to _{γ 4} are supplied to each of the multiplication circuits 95, 96, 97, and 98, and are multiplied by the motion vectors from the minimum value detection circuits 131 to 134, respectively. These multiplier circuits 9
The outputs of 5 to 98 are added to the adder circuits 105, 106, 10.
7, and the motion vector for one field is taken out at the output terminal 108.

FIG. 1 shows the configuration of the weighting coefficient generation circuit 120, which has input terminals 1 to 5, 11 to 15, and 2.
Frame difference integral data supplied to graph memories 121 and 12, respectively.
2,123,124. The slope near the minimum value of each region is calculated from these frame difference integral data.

The slope ΔH1 near the minimum value regarding the area 57A is:
It corresponds to the sum of differences in four directions.

ΔH1=(H2-H1)+(H3-H1) +(H4-H1)+(H5-H1) =H2+H3+H4+H5-4H1 Sum and shift of frame difference integral data H2 to H5 formed by adder circuits 6, 7, and 8 4 by circuit 9
The multiplied minimum value H1 of the frame difference integral data is supplied to the subtraction circuit 10, and the slope ΔH1 is obtained at the output of the subtraction circuit 10.

Slope ΔH2 near the minimum value regarding area 57B
are adder circuits 16, 17, 18, shift circuit 19
and is generated by the subtraction circuit 20. Area 57C
The slope ΔH3 near the minimum value for the region 57D is generated by the addition circuits 26, 27, 28, the shift circuit 29, and the subtraction circuit 30, and the slope ΔH4 near the minimum value for the area 57D is generated by the addition circuits 36, 37, 38,
It is generated by a shift circuit 39 and a subtraction circuit 40.

The larger the slope near this minimum value, that is, the steeper the slope, the higher the reliability of the motion vector.
Therefore, the weighting coefficient α ₁ to _{α 4} is proportional to the magnitude of the slope.
is calculated. In this case, subtraction circuits 10, 20,
30, 40, are supplied to adder circuits 41, 42, 43, 44, and terminal 45.
An offset signal OFS1 from is added to each slope.

The outputs of the adder circuits 41 to 44 are the adder circuit 4
6, 47, and 48 to form the sum total of slopes ΣΔH of the following equation.

ΣΔH=(ΔH1+OFS1)+(ΔH2+OFS1) +(ΔH3+OFS1)+(ΔH4+OFS1) The sum of these slopes ΣΔH is the dividing circuit 49, 50,
It is supplied as the denominator input of each of 51 and 52.
The outputs of the adder circuits 41 to 44 are supplied as numerator inputs to the divider circuits 49 to 52, respectively.
Weighting coefficients α ₁ to _{α 4} of the following equations are obtained from 9 to 52, respectively. The offset signal OFS1 is added to prevent the numerator from becoming zero in the following equation.

α ₁ = (ΔH1+OFS1)/ΣΔH α ₂ = (ΔH2+, FS1)/ΣΔH α ₃ = (ΔH3+OFS1)/ΣΔH α ₄ = (ΔH4+OFS1)/ΣΔH On the other hand, the minimum value of each frame difference integral data of areas 57A to 57D H1, H11, H21, and H31 are supplied to adder circuits 71, 72, 73, and 74, and the offset signal from terminal 70 is applied to their minimum value.
OFS2 is added. The outputs of the adder circuits 71 to 74 are supplied to reciprocal circuits 75, 76, 77, and 78, respectively, and are converted into reciprocal values. offset signal
OFS2 is added to prevent the denominator term from becoming zero. The minimum value of frame difference integral data is the residual deviation after correction when noise or moving objects exist in each region, and its reciprocal is:
It means the degree of accuracy when performing motion correction for that area.

The respective outputs of the reciprocal circuits 75 to 78 are sent to the adder circuit 8.
5, 86, 87 are added, and the sum of the reciprocals is Σ1/H
is formed. This sum Σ1/H is calculated by the division circuit 81,
It is supplied to 82, 83, and 84 as a denominator input.
The division circuits 81 to 84 are used to perform proportional allocation, and each of the division circuits 81 to 84 receives the reciprocal circuit 7 as a numerator input.
5 to 78 outputs are provided. Division circuit 81-8
4, second weighting coefficients β ₁ to β ₄ expressed by the following equations are obtained.

β ₁ = {1/(H1+OFS2)}/(Σ1/H) β ₂ = {1/(H11+OFS2)}/(Σ1/H) β ₃ = {1/(H21+OFS2)}/(Σ1/H) β ₄ = {1/(H31+OFS2)}/(Σ1/H) The weighting coefficients γ 1 to γ ₄ to be actually used are the weighted average of the first weighting coefficient and the second weighting coefficient calculated as _above . Desired. In this example,
Weighting coefficients γ ₁ to γ ₄ are determined by simple averaging.

That is, the weighting coefficients α ₁ and β ₁ are supplied to the adding circuit 61, the weighting coefficients α ₂ and β _{2 are supplied to the adding circuit 62, the weighting coefficients α 3 and β 3} are supplied to the adding circuit 63, and the adding circuit 64 is supplied with weighting coefficients α ₃ and β ₃ . are supplied with weighting factors α ₄ and β ₄ . The outputs of these adder circuits 61 to 64 are halved by shift circuits 91, 92, 93, and 94, respectively.
Then, the following weighting coefficients γ ₁ to γ ₄ are formed.

γ ₁ = (α ₁ + β ₁ )/2, γ ₂ = (α ₂ + β ₂ )/2, γ ₃ = (α ₃ + β ₃ )/2, γ ₄ = (α ₄ + β ₄ )/2, This weight Coefficients γ ₁ to _{γ 4} are supplied to multiplication circuits 95 to 98, respectively, and are multiplied by motion vectors V1, V2, V3, and V4 of each region from input terminals 101 to 104, respectively. Using this weighted average, the motion vector V for the entire field is determined at the output terminal 108 as shown in the following equation.

V=γ ₁ ·V1+γ ₂ ·V2+γ ₃ ·V3+γ ₄ ·V4 This motion vector indicates the movement of the television camera. When compressing and transmitting a high-definition television signal to improve image quality, a motion vector V is transmitted and motion correction is performed on the receiving side.

Note that instead of simply averaging the first weighting coefficient and the second weighting coefficient, the third weighting coefficient to be used may be determined by performing a weighted averaging in consideration of the amount of noise and the like.

〔Effect of the invention〕

In each of the plurality of regions formed by dividing the screen of one fiddle, the present invention provides a first method in which the weight increases as the slope of the minimum value of frame difference integral data and the surrounding frame difference integral data becomes steeper.
Since the motion vectors of each region are weighted and averaged using the weighting coefficient, estimation accuracy can be improved without being affected by the picture on the screen.

In addition, the present invention weights the motion vectors of each region using a second weighting coefficient that increases the weight as the residual deviation of the motion vectors of each region is smaller, so that the motion vectors of each region are weighted and averaged without being affected by noise or moving objects. , the estimation accuracy can be improved.

[Brief explanation of the drawing]

FIG. 1 is a block diagram of a weighting coefficient generation circuit and a weighted average circuit in one embodiment of the present invention, FIG. 2 is a schematic diagram used to explain the weighting coefficient generation circuit, and FIG.
The figure is an overall block diagram of an embodiment of the present invention, and FIGS. 4 and 5 are schematic diagrams used to explain motion vector detection. 1-15, 11-15, 21-25, 31-3
5: Input terminals to which frame difference integral data of each of a plurality of regions is supplied, 49-52, 81-84:
Dividing circuit for proportional distribution, 95-98: Multiplication circuit for weighted average, 108: Output terminal for motion vector of entire field, 110: Input terminal for digital television signal, 121-124:
Graph memory, 56:1 field screen.

Claims (1)

[Scope of Claims] 1. The screen of one field is divided to form a plurality of regions, each of the regions is divided into a plurality of blocks, and the representative point pixel data in the block of the previous frame and the above of the current frame are divided. determining the absolute value of the frame difference between the representative point pixel data in the block and the corresponding pixel data in a predetermined range, and integrating the absolute value of the frame difference for each of the predetermined ranges in each of the regions; A means for detecting a motion vector for each of the above regions from this frame difference integral data; means for determining the reliability of each motion vector of the region and calculating a first weighting coefficient corresponding to this determination; means for determining the degree of residual deviation when motion correction is performed by and calculating a second weighting coefficient corresponding to the degree of this residual deviation; means for calculating a third weighting coefficient to be used; and means for calculating a motion vector of the first field by weighted averaging of the motion vectors of each of the regions using the third weighting coefficient. A television signal motion detection device characterized by: