JPH0813146B2 - Motion vector detector - Google Patents

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion vector detecting device for detecting a motion amount of an image.

2. Description of the Related Art As an example of a conventional image motion vector detecting device, there is one shown in Japanese Patent Application Laid-Open No. 61-269475.

FIG. 7 is a schematic diagram thereof, in which 1 is a first latch circuit, 2 is a representative point storage memory, 3 is a second latch circuit, 4 is a correlator, 5 is an address controller, and 6 is an address switching circuit. , 7 is a cumulative adder, and 8 is a correlation search circuit, which form a motion vector detection unit 11. Reference numeral 9 is a correlation validity / invalidity determination circuit, 10 is a determination circuit, and these constitute a motion vector determination unit 12.

An image motion vector detection device using the conventional correlation calculation device configured as described above will be described.

First, the motion vector of the image will be described with reference to FIGS. 6 (a) to 6 (c). FIG. 11A shows an image at a certain time. Then, FIG. 6B shows an image after one field or one frame. As described above, when the image is translated by the movement of the image pickup apparatus or the like, a vector indicating the amount of translation of the image as indicated by an arrow in FIG. FIG. 8 shows the state of representative points and pixels around them in the representative point matching method, which is the most general method of detecting the motion vector of such an image. The motion vector detection is performed by detecting where the image data at the position of the representative point in a certain field has moved among the surrounding pixels in the next field.

Next, an image motion vector detecting device using a conventional correlation calculating device will be described with reference to FIGS. 7 and 8.

The image data at each representative point on the screen is fetched by the first latch circuit 1 and is written at an address corresponding to each representative point in the representative point storage memory 2 at a timing. And next field (or next frame)
At, the image data in the motion vector detection area around the position of each representative point and the representative point of the previous field (frame) stored in the representative point storage memory 2 are correlated and input to the cumulative adder 7. The cumulative adder 7 cumulatively adds the data obtained by correlating the positions of the coordinates with respect to the representative point. Then, when the cumulative addition around all the representative points is completed, the correlation search circuit 8 determines the place having the highest correlation value among the cumulative addition values held in the cumulative adder 7. That is, the position (address) having the highest value of correlation when the position of the representative point is used as the reference is the motion vector. Further, based on the distribution of the correlation values around the representative point (average value, minimum value, maximum value, gradient, etc.), the correlation validity / invalidity determination circuit 9 determines whether the motion vector obtained by the correlation calculation is valid or invalid. Determine whether. The above operation is performed for each area when the screen is divided into a plurality of areas.

Then, the determination circuit 10 determines the motion vector of the entire screen based on the motion vector obtained from each area of the screen and the validity determination information.

Since the operation up to this point is performed for each field (frame), the next field (frame) is calculated while performing the correlation calculation.
There is a first latch circuit 1 for storing the image data at the representative point for the correlation calculation of. Further, the second latch circuit 3 holds the image data of the representative point when the image data of a certain representative point and the image data of its surroundings are correlated.

The motion vector detection unit 11 detects the motion vector by the above correlation calculation, and the vector determination unit 12 determines the motion vector of the entire screen from the motion vector of each area of the screen obtained by the motion vector detection unit 11 and the correlation information. Ask for.

Problems to be Solved by the Invention However, as in the case where the subject is stationary and only the imaging device moves, it suffices if the motion vectors in each area are all the same (FIG. 9 (a)). When a part of the subject is not stationary and moves differently from the entire screen (FIG. 9 (b)), the motion vectors in each area are not the same.

Therefore, as in the above-mentioned conventional example, when the motion vector of the entire screen is determined by taking the average of the motion vectors of each area,
There is a problem that a detection error occurs due to a motion vector of a region that is different from the motion of the entire screen.

In view of the above point, the present invention does not cause a detection error of the motion vector of the entire screen even when a part of the subject is not stationary, and a motion vector detection device capable of extracting the motion of the background of the image without interruption. The purpose is to provide.

Means for Solving the Problems The present invention divides a screen into a plurality of areas, and uses a motion vector detection unit that detects a motion vector of an image in each area and an output of the motion vector detection unit to disperse the motion vector of each area. A divergence calculation unit that obtains a divergence degree indicating the condition, and a vector invalidity determination unit that determines that a motion vector in a region having a large difference between the past integrated motion vector of the entire screen when the divergence degree of the motion vector is large If the motion vector of the entire screen is determined by the motion vector of the area other than that determined, and when the divergence of the motion vector is small, by determining the motion vector of the entire screen from the motion vector of each area It is characterized by comprising an entire screen vector determination unit that operates so as not to stop the output of the motion vector of the entire screen as much as possible.

In the above configuration, an integrator that obtains an integrated motion vector of each area may be provided, and the divergence calculation unit may use the output of the integrator to obtain the divergence of the motion vector.

Effect The present invention calculates the divergence of the motion vector of each area of the screen by the above configuration, and when this divergence is large, the motion vector of the area having a large difference from the past motion vector of the entire screen is invalid, and other If there is at least one motion vector detection area that seems to have detected a correct value using the motion vector of, the motion vector of the entire screen is obtained using these. When the divergence is small, the above operation is not performed and the motion vector detected from each area is used as it is to obtain the motion vector of the entire screen, so that even if the detection device malfunctions, the output is performed as much as possible. Realizes an operation that does not stop.

When the divergence is calculated using the integrated motion vector that has been subjected to the integration process, it can be detected even when the moving speed of the subject in a part of the screen is small, so that the detection error of the motion vector of the entire screen is reduced. be able to.

Practical Example FIG. 1 is a block diagram of a motion vector detecting device in a first embodiment of the present invention, in which 11 is a motion vector detecting unit,
Reference numeral 13 is a motion vector reliability determining unit, 15 is a divergence calculating unit, 16 is a divergence determining unit, 18 is a vector invalidity determining unit, 19 is a whole screen vector determining unit, and 21 is a low-pass filter (LPF).

The operation of the motion vector detecting device configured as described above will be described below.

First, in FIG. 1, the motion vector detecting unit 11 detects the motion vector between fields or frames of the input image signal for each area of the screen in the same manner as in the conventional example.
Next, in the motion vector reliability determination unit 13, the distribution of correlation values around the representative points in that area (average value, minimum value, maximum value,
Based on the gradient, etc., it is determined whether or not the motion vector in that area is reliable, and the motion vector determination information (a signal indicating whether or not the motion vector in each area is correctly detected) is output. This is the same operation as the correlation validity / invalidity determination circuit 9 (see FIG. 7) of the conventional example.

Next, the divergence calculation unit 15 calculates the divergence of the motion vector by using the motion vector of the area in which the motion vector determination information is valid. The calculation method is shown in the following formula.

Divergence (DIV1) = [Σ | (average value of motion vector of each area) − (motion vector of each area) |] ÷ (number of areas for which motion vector judgment information is OK) where Σ is motion vector judgment information Indicates that the sum is calculated for the area in which is valid, and | represents absolute value. Also, since the vector is two-dimensional,
For the divergence, the average value, the root mean square value, or the added value of the values obtained for each component is used as the actual divergence.

The larger the value of the divergence DIV1 obtained by this calculation, the greater the degree to which the respective motion vectors detected from the respective areas point in different directions, that is, the greater the possibility that a part of the subject is moving. Indicates. If the degree of divergence is zero, all motion vectors point in the same direction, and it can be determined that the subject is a stationary object.

Next, the divergence determination unit 16 determines the divergence degree obtained by the above calculation.
Divergence by comparing the size of DIV1 with a certain threshold
When DIV1 exceeds the threshold value, the divergence condition flag is set to the divergent state. The fact that the degree of divergence is large means that a part of the subject is moving and the motion vector in that region is in a state that should be excluded from the candidates for determining the motion vector of the entire screen.

The vector invalidity determination unit 18 is a case where the output of the divergence determination unit 16 is in the state of divergence, and the motion vector of each area in which the motion vector determination information is valid and the output of the low-pass filter 21 When the difference in the delay components of the entire motion vector is not within a predetermined error range, invalid information that invalidates the motion vector detected in that region is output, assuming that the subject in that region has moved.

Here, the reason why the low-pass filter 21 adds a delay element to the integrated motion vector information will be described. If part of the screen begins to move while the subject is stationary, the divergence DIV1
However, there is a time delay until it is determined that a part of the screen has moved above a certain threshold. Therefore, after the determination, the motion vector of the entire screen has a value after a part of the screen has moved. In order to compensate for this and use a value close to the motion vector of the entire screen when a part of the screen starts to move, low-pass filtering processing is performed.

Here, when the vector invalidity determination unit 18 makes the determination, since the motion vector of the entire screen in the current field (frame) has not yet been determined, the delay component is 1 field (in addition to the low-pass filtering process). There is a (frame) delay.

Using the invalid information obtained as described above, the motion vector of each area, and the motion vector determination information, the overall screen vector determination unit 19 determines the overall motion vector of the screen. As the determination method, a motion vector detected in an area excluding an area determined to be unreliable in the motion vector based on the motion vector determination information and an area determined to be in motion of the subject based on the invalid information is used. Determine the motion vector for the entire screen. The determination method is the median (median value) or the average value of the motion vectors of the areas determined to be valid.

As described above, according to the present embodiment, it is possible to provide a motion vector detection device that does not cause a detection error of the motion vector of the entire screen even when a part of the subject is not stationary.

FIG. 2 is a block diagram of a motion vector detecting device in the second embodiment of the present invention.

In the figure, 11 is a motion vector detection unit, 13 is a motion vector reliability determination unit, 14 is a first integrator, 15 is a divergence calculation unit, 16 is a divergence determination unit, 17 is a timer, 18 is a vector invalidity determination unit, Reference numeral 19 is an entire screen vector determination unit, 20 is a second integrator, and 21 is a low-pass filter (LPF). The difference from the first embodiment (FIG. 1) is that the timer 17, the first integrator 14, the second
This is the addition of the integrator 20.

The operation of the motion vector detecting device configured as described above will be described below.

First, in FIG. 2, the motion vector detecting unit 11 detects a motion vector between fields or frames of an input image signal for each area of the screen, as in the first embodiment. Next, the motion vector reliability determination unit 13 can trust the motion vector in the area based on the distribution of correlation values around the representative point in the area (average value, minimum value, maximum value, gradient, etc.). It is determined whether or not the motion vector determination information (a signal indicating whether or not the motion vector in each area is correctly detected) is output. Up to this point, the operation is the same as in the first embodiment.

Next, the first integrator 14 integrates the motion vector of each area. In order to prevent the divergence of the value after the integration, the integration is not a perfect integration, but an integration with a leak is performed. At this time, if the motion vector of the area is determined to be incorrect by the motion vector determination information and the motion vector is invalid, the integration operation is not performed, and only the integration leakage operation is performed. This is shown in FIG. That is, while the integration proceeds and the integrated motion vector increases during the period A, the motion vector determination information becomes invalid during the period B, and the motion vector in that area is considered unreliable, so the integral operation is not performed. Is attenuated due to integral leakage. Then, in the period C, the motion vector determination information becomes valid again, and the integration operation is restarted.

 The operation of integration is expressed by the following equation.

(1) When motion vector determination information is valid s_vect (n) = s_vect (n−1) × m + vect (n) (2) When motion vector determination information is invalid s_vect (n) = s_vect (n−1) × m where s_vect (n): integrated motion vector of current field (frame) s_vect (n-1): integrated motion vector of previous field (frame) vect (n): (difference) motion vector of current field (frame) m : Integrated damping coefficient (≦ 1).

Here, if the value of m is 1, it is a perfect integration. If it is less than 1, it is incomplete integration, and the integration error increases as it becomes smaller. Also, since the vector is two-dimensional, it is necessary to perform integration on each of the two independent axis components in the horizontal and vertical directions.

In addition, the timer 17 operates such that the FLGB information is not set to OK until a predetermined time elapses after it is determined that the motion vector of the region is invalid based on the motion vector determination information and is determined to be valid for each region. This will be described with reference to FIG. It is assumed that the motion vector determination information is valid first (period A). Although the integrated motion vector increases by this, the motion vector determination information becomes invalid during the period B, and the value of the integrated motion vector that should normally be a broken line graph is attenuated. Then, in the period of C, the motion vector determination information becomes valid again, and the integration operation is restarted.

Here, the motion vector determination information becomes invalid, and a certain predetermined time elapses after it is determined that the motion vector is correct again until the output of the first integrator 14 returns to the original value (the value of the characteristic of the broken line). FLG information is invalid). As a result, the divergence calculation unit 15 and the vector invalidity determination unit 18 realize an operation that does not use the information of the motion vector of the area until the integrated motion vector returns to near the original value. FL after motion vector judgment information becomes valid
The period until the G information becomes valid takes a period in which the integral value can be sufficiently restored by using the time constant of the integral calculation as a guide.

Next, the divergence calculator calculates the divergence of the motion vector using the integral motion vector corresponding to the area in which the FLG information is valid and the motion vector determination information is valid. The calculation method is shown in the following formula.

Divergence (DIV2) = [Σ | (average value of integrated motion vector of each area)-(integrated motion vector of each area) |] ÷ (FLG
(The number of areas where both the information and motion vector judgment information are valid) where Σ indicates that the total sum is calculated for the areas where the FLG information is valid and the motion vector judgment information is valid, and | | is an absolute value Indicates to take. Also, since the vector is two-dimensional, the divergence value is the average value, the root mean square value, or the added value of the values obtained for each component, and is used as the actual divergence value.

The larger the value of the divergence DIV2 obtained by this calculation, the greater the degree to which the respective motion vectors detected from each area point in different directions, that is, the greater the possibility that a part of the subject is moving. Indicates. If the degree of divergence is zero, all motion vectors point in the same direction, and it can be determined that the subject is a stationary object.

The difference from the first embodiment is that in the first embodiment, the divergence is calculated using the motion vector for each field (frame), but in the present embodiment, the integrated motion vector subjected to integration processing is used. Is the point that is used.

The divergence determination unit 16 determines the divergence degree DIV2 obtained by the above calculation.
Is compared with a predetermined value, and when the divergence DIV2 is larger, the divergence condition flag is set to the divergence present state. The fact that the degree of divergence is large means that a part of the subject is moving and the motion vector in that region is in a state that should be excluded from the candidates for determining the motion vector of the entire screen. The vector invalidity determination unit 18 searches for a region that should be excluded from the motion vector candidates of the entire screen.

The vector invalidity determination unit 18 is a case where the output of the divergence determination unit 16 is in the state of divergence, and the FLG information is valid, and the motion vector determination information is effective. If the difference between the delay component of the integral value of the motion vector of the entire screen that is the output of the wave filter 21 is not within a certain predetermined error range, the motion vector detected in that area is regarded as moving in the area. Output invalid information to be invalidated.

Here, the low-pass filter 21 used in the vector invalidity determination unit 18
The output signal of will be described. Whole screen vector determination section
The motion vector of the entire screen obtained by 19 is integrated by the second integrator 20. At this time, the integration characteristic is the same as that of the first integrator 14, and incomplete integration is performed in the states other than the state in which the motion vector cannot be detected, and only the integration leakage operation is performed in the state in which the motion vector cannot be detected. Next low-pass filter
21 adds a delay element to the integrated motion vector information.
This signal becomes the delay component of the integral value of the motion vector of the entire screen. Here, when the determination is made by the vector invalidity determination unit 18, since the motion vector of the entire screen in the current field (frame) has not yet been determined, the delay component is other than the integration process and the low-pass filtering process. There is a delay of one field (frame).

The reason why the delay element is further added to the integral value of the motion vector in this way will be described below. As shown in Fig. 5, when a part of the screen starts to move from the state in which the subject is stationary (the origin of the graph), the divergence DIV2 is calculated using the integrated motion vector. The degree DIV2 gradually increases, but the judgment that a part of the subject has moved must be that the divergence degree DIV2 is above a certain threshold,
There is a time lag between when a part of the subject starts moving and when it is determined that there is movement. Therefore, when the determination is made, the integral value of the motion vector of the entire screen has changed, and the motion vector of the entire screen at the time when a part of the screen has moved is a past one. Therefore, in order to compensate for the time difference, the low-pass filter 21 is provided with a time delay, and a value close to the motion vector of the entire screen when a part of the subject actually moves is used.

Using the invalid information obtained as described above, the motion vector of each area, and the motion vector determination information, the overall screen vector determination unit 19 determines the overall motion vector of the screen. As the determination method, a motion vector detected in an area excluding an area determined to be unreliable in the motion vector based on the motion vector determination information and an area determined to be in motion of the subject based on the invalid information is used. Determine the motion vector for the entire screen. The determination method is the median (median value) or the average value of the motion vectors of the areas determined to be valid.

As described above, according to the present embodiment, since the divergence is evaluated by using the integration result of the motion vector, this can be detected even when the moving speed of the subject in a part of the screen is small, and the motion of the entire screen can be detected. It is possible to provide a motion vector detection device that does not cause a vector detection error.

Although the average value of the motion vector of each area is used in the calculation of the divergence calculation unit 15 in the above embodiment, a median may be used. Further, in the calculation of the numerator term in the calculation of the divergence calculation unit 15, the ratio between the average value of the integrated motion vector of each area and the integrated motion vector of each area may be taken. Further, although the calculation of the divergence calculation unit 15 performed division by the number of areas in which the FLG information or the motion vector determination information is valid, this may not be performed. The LPF 21 may be omitted.

Further, in the first and second embodiments, the signal processing is digitized, and the output of the motion vector detection unit 11 (distribution of correlation values around the motion vector / representative point (average value, minimum value, maximum value , Gradient, etc.) is input to the microcomputer, and the parts other than the motion vector detection unit 11, namely, the motion vector reliability determination unit 13, the timer 17, the first and second integrators 14, 20, the divergence calculation unit 15, the divergence calculation unit, The operations of the determination unit 16, the vector invalidity determination unit 18, the entire screen vector determination unit 19, and the low-pass filter 21 may all be realized by software of a microcomputer.

Effect of the Invention In order to continuously extract the movement of the background of the screen as little as possible without malfunction, when the motion vector differs depending on the detection area and the motion vector becomes disordered, it is determined based on the motion vector determined to be correct so far. It realizes the operation of using close motion vectors effectively. Further, even if a malfunction occurs, the correct motion vector can be adopted and output as quickly as possible, which has a great practical effect.

[Brief description of drawings]

FIG. 1 is a block diagram of a motion vector detecting device in a first embodiment of the present invention, FIG. 2 is a block diagram of a motion vector detecting device in a second embodiment of the present invention, and FIG. 3 is a block diagram of the same embodiment. FIG. 4 is a diagram showing a temporal change of the integrated motion vector according to the motion vector determination information, FIG. 4 is a relationship diagram of the integrated motion vector, FLG information, and motion vector determination information of the same embodiment, and FIG. 5 is a divergence of the embodiment. Change over time, Fig. 6 (a) ~
FIG. 7C is an explanatory diagram of a motion vector of an image, FIG. 7 is a configuration diagram of a conventional motion vector detection device, FIG. 8 is an explanatory diagram of states of representative points and surrounding pixels in a representative point matching method, and FIG. FIGS. 9A and 9B are diagrams showing the state of the detected motion vector in each area. 11 …… Motion vector detector, 14 …… First integrator, 15 ……
Divergence calculation part, 18 ... Vector invalidity judgment part, 19 ... Whole screen vector judgment part.

Claims (2)

[Claims] 1. A motion vector detecting section for dividing a screen into a plurality of areas and detecting a motion vector of an image in each area,
There is a large difference between the divergence calculation unit that calculates the divergence degree indicating the degree of dispersion of the motion vector of each area using the output of the motion vector detection unit and the past motion vector of the entire screen when the divergence degree of the motion vector is large. The motion vector of the region is determined to be invalid, and the motion vector of the region other than that determined to be invalid is used so long as the motion vector of the region estimated to be correct is used. Determines the vector and when the divergence of the motion vector is small, determines the motion vector of the entire screen from the motion vector of each area so that the motion vector output of the entire screen does not stop as much as possible when a malfunction occurs. A motion vector detecting device, comprising: 2. A motion vector detecting section for dividing a screen into a plurality of areas and detecting a motion vector of an image in each area,
An integrator that obtains an integrated motion vector of each area, a divergence calculation unit that calculates the divergence degree indicating the degree of dispersion of the motion vector of each area using the output of the integrator, and a divergence calculator that calculates when the divergence of the motion vector is large It is determined to be invalid so that the motion vector of the area having a large difference from the integrated motion vector of the entire screen is invalid, and this is used as long as the motion vector of the area estimated to be correct exists. The motion vector of the entire screen is determined by the motion vector of the area other than the above.When the divergence of the motion vector is small, the motion vector of the entire screen is determined from the motion vector of each area. A motion vector detection device comprising an entire screen vector determination unit that operates so as not to stop vector output as much as possible.