JPH036779A - Processing system for extraction of three-dimensional position information - Google Patents

Abstract

PURPOSE:To easily extract the 3-dimensional position information on an object by providing a shift form discriminating means which discriminates whether or not the object is shifted based on one of N different shift forms stored previously. CONSTITUTION:An image input part 101, feature point extracting part 102, a shift form discriminating part 103, a reference point distance supply part 104, a position information extracting part 105 are provided. In such a constitution, one of N different shift forms stored previously is decided. When an object is shifted in the decided shift form, at least >=2 feature points are used. Then a 3-dimensional position is extracted based the coordinates set on the two frame images of those feature points, the distance between the reference characteristic point and an image input device, and the distance between each reference characteristic point and the image input device. Thus it is possible to extract the 3-dimensional position information on the feature points of a moving object just by observing the object.

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Application Field] The present invention relates to a three-dimensional position information extraction processing method performed by observing an object with one image input device.

[Conventional technology]

In the conventional method of extracting three-dimensional positional information about feature points on an object from a series of frame images obtained by observation with a single image input device, a predetermined movement of the object is determined. A stereo image is constructed from two input images observed before and after the movement of the object being moved by the object movement means, and a stereo
Three-dimensional position information was extracted by dimensional position measurement.

[Problem to be solved by the invention]

Therefore, it is necessary to apply a predetermined motion to the object by a predetermined amount of movement, or to manually measure the amount of movement, which takes time and effort to extract three-dimensional position information. There was a problem.

In order to solve such conventional problems, the present invention has the following features:
By observing an object moving in one of N predetermined movement modes without setting any restrictions on the amount of movement of the object, it is possible to extract N number of three-dimensional position information of the object. The objective is to provide a dimensional position information extraction processing power formula.

[Means to solve the problem]

In the three-dimensional position information extraction processing power formula of the present invention, the three-dimensional movement of an object between two frame images in a series of frame images obtained by observation with one image input device is stored in advance. The present invention has a moving form determining means that uses the coordinates of feature points on these two frame images to determine whether the object is in one of N specific moving forms. It has characteristics.

The three-dimensional position information extraction processing power formula of the present invention has a movement mode determining means for determining whether or not the object has moved according to one movement form among N movement forms stored in advance. It differs from the conventional method in that it

[Example] Below, eight examples of the present invention will be described in detail based on one drawing. FIG. 1 shows a block diagram of the three-dimensional position information extraction processing power formula in this embodiment.

This is an example in which one of the feature points is used as a reference point and the distance of the reference point to the image input device is given. In the figure, 101 is an image input section, 102 is a feature point extraction section, and 103 is a moving form discrimination section.

104 is a reference point distance adding section, and 105 is a position information extraction section.

The image input unit 101 observes a moving object using an image input device such as a television camera, and outputs two frame images of a series of frame images obtained by observation to the feature point extraction unit 102. The feature point extraction unit 102 extracts at least two or more feature points from the two frame images output from the two-image input unit 101, and
The coordinates on the two frame images are output to the moving form discriminator 103 and the position information extractor 05. Extraction of feature points from two frame images involves, for example, extracting points where the shape or surface color of an object locally changes abruptly using image processing, and then matching the points that are the same on the object. .

The associated points are defined as feature points. Movement mode discrimination unit +0
3 memorizes N specific movement forms of an object,
Using the coordinates of the feature points on the two frame images output from the feature point extraction unit 102, the three-dimensional movement of the object between these two frame images is determined by the stored N of the object.
It is determined whether or not it is one of the specific movement modes on the street, and a control signal indicating whether or not to extract position information is output to the position information extraction unit +05. 1 out of N ways of transportation
If it is determined that one movement is a j-t, it outputs a control signal to execute the extraction of one position information, and also extracts the position information from a signal indicating which movement form among N ways of movement. 105, and when it is determined that the movement is not included in the N ways of movement, the position information extraction unit 105 outputs a control signal to the effect that extraction of 1 position information is not executed.
Output to. The reference point distance giving unit 104 sets one point among the feature points as a reference point, and gives the distance from the reference point to the image input device as a reference for the distance of each feature point to the image input device. If only the positional relationship between the two is required, an appropriate value may be given as the distance from the reference point to the image input device.
When the control signal output from 3 is a command to execute position information extraction, the position of the feature point output from the feature point extraction unit 102 on the two frame images and the position output from the movement form determination unit 103 Movement mode and reference point distance adding unit 1
Using the distance of the reference point output from 04 to the image input device, the three-dimensional position information of the feature point at the time of inputting the two frame images is extracted and output.

One embodiment of the movement mode determination unit 103 will be described below.

In a series of frame images obtained by observing an object moving according to the Nth specific movement mode out of N pre-stored specific movement modes,
From the coordinates on the frame image of the feature points extracted from two frame images in a series of frame images, the rotation angle of the rotational motion component of the object and the translation amount of the translational motion component of the object between these two frame images are calculated in three dimensions. Let n5 be the number of feature points required to extract target movement information.

FIG. 2 shows the processing procedure of the discrimination method in the moving form discrimination unit 103.
2. The number of extracted feature points is counted.

Let M be the number of extracted feature points. In process 202.

Assuming that the movement form of the object between the two frame images output from the image input unit 101 is the movement form of the N movement forms stored in advance, n0 feature points are selected from the M feature points. The movement information between these two frame images is extracted by the method described later for the number of combinations that take the feature points, lcn++. In process 203, process 202
If the HCnk pieces of movement information extracted in step 12 agree with each other, it is determined that the form of movement between the two frame images is the 5th movement form. As shown in FIG. 2, the processes 202 and 203 for determining whether or not it is the 2nd movement mode are as follows: Do each. In process 204, the number of cases where it is determined to be the i-th movement mode in process 203 from k·1 to ii=lJ is checked. If the number checked in process 204 is 0, which of the pre-stored movement forms is the three-dimensional movement form of the object between the two frame images output from the one-image input unit 101? Since this is not applicable, a control signal indicating that extraction of the 19th position information is not executed is outputted to the position information extraction unit 105 in step 2 205. When the number checked in process 204 is 1 or more, a control signal to execute extraction of 9 position information is outputted in process 205, and
The position information extraction unit 105 processes a signal representing a movement form corresponding to the movement of an object between two frame images through processing 206.
0 When the three-dimensional movement form of the object between two frame images output from the image input unit 101 corresponds to two or more movement forms among the movement forms stored in advance. It may be any one of these forms of movement.1 For example, by assigning numbers to the movement forms that are stored in advance, and selecting one of the two or more corresponding movement forms. The moving form with the smallest number is determined as one corresponding moving form as the three-dimensional moving form of the object between the two frame images.

Next, a method for extracting movement information of an object between two frame images in process 202 will be described. In the case where the second pre-stored movement form is "°°rotational movement around the Y-axis and translational movement in the X direction and the We will show how to extract the amount of movement.

If the camera model is represented by the central projection shown in Figure 3, the center of the camera lens is the origin and the optical axis of the camera is the Z axis.

In an orthogonal coordinate system with the X axis in the direction perpendicular to the image plane of the camera, the three-dimensional position (Xk,
Yk, Zk) L;! , Pk Noah L, -
4.The position on the image is (Llk, Vk), and the distance from the center of the camera lens to the image plane of the camera is r, then Zk-Z
k (1
) can be expressed as

As shown in FIG. 4, the first and jth feature points Pk
Pik and Pjk are the points when inputting two frame images.
(k-0,1,-,N-1,N: number of feature points,
Pie, PjO are reference points), and the coordinate values of these feature points on the two frame images output from the feature point extraction unit 102 are (Uik, Vik), (Ujk,
Vjk).

In Fig. 4, when the object between the i-th and j-th frame images is performing a rotational movement around the Y-axis by a rotation angle β and a translational movement in the X direction and the X direction, the following equation holds true. .

1 -〉-1--÷ -1-→0Pjk
-OPjO= R0Pik-OPiO (K≠0) (2) Translational movement vector t= 0PjO0PiO (3) However, the two points of feature point PO that can be obtained and use the amount of translational movement of the object between two frame images as one reference Coordinate values on the frame image (IJ, ., ■, .), (Ui.

■,. ) can be extracted using equation (5).

The rotation angle β of the object between two frame images is expressed by the following equation (
Substitute 11 into equation (2) and eliminate Zjk using equation 1 (4).

Since the Y coordinate values of the feature points P ik and P jk are the same when the i-th and j-th two frame images are input, the following equation holds true.

ZikVik = ZjkVjk
(4) By substituting equation (1) into equation (3) and eliminating ZjO using equation (4).

By deriving and eliminating Zik from equation (6).

The rotation angle β is determined by the coordinate values (Ul, ,
V. ), (Uj.,■yo.), (Ui,.

V;k), CUik, Viw), (U
It can be obtained from equation (7) using rp, Vil) (Uit=Vjj).

auralOa*oal+ awesome+ ab+al.

here.

a. = ((Ujm−UiO) VjOViI1
1+ (Uim-UjO) ViOVjml ra1=
(ViOVjm-VjOVim) f 2+ UjO
LIimViQVjm-UiOUjmVjOVima,
, -1(UiOUim) VjOVjm+ ([J
jO-Ujm) ViOViml r Thus,
Using the coordinates on the frame image of three feature points output from the feature point extraction unit 102 shown in FIG. By doing so, it is possible to realize the extraction of movement information for the movement form of ``rotational movement around the Y axis and translational movement in the X direction and Z direction'' in the process 202 shown in FIG. An example was explained.

Next, an example of the 1-position information extraction unit 105 will be described in the case where the movement form output from the movement form discrimination unit 103 is "rotation i! movement around the Y axis and translational movement in the X direction and the Z direction."

Using the coordinates of the feature points on the two frame images output from the feature point extracting unit 102 in FIG. 1, the rotation angle β is determined from Ni (7). Obtained rotation angle β, 2 (4) and equation (6)
By using , the Z components of the special points P ik and P jk at the time of inputting the j-th and j-th frame images are as follows.

1k- ulkV, lICO3β+vJ* f 5irlβ-u
ikVIk (k・1, 2..., N-1) (
8), and the X component and Y component of the three-dimensional position of the feature point are obtained by substituting equations (8) and (2) (9) into equation (11). If the form of movement to be performed is "rotational movement around the Y-axis and translational movement in the X and Z directions," then the two frames i-th and The three-dimensional positions of feature points at the time of image input can be determined.

Therefore, the 2 output from the feature point extraction unit 102 in FIG.
Coordinate values U ik of feature points on two frame images.

1 Movement output from the movement mode determination unit 103 using Vik, Ujk, Vjk, and the Z coordinate value ZiO of the reference point that can be calculated from the image of the reference point output from the reference point distance adding unit 104 and the distance to the human powered device. By using the position information extraction method corresponding to the form, it is possible to obtain the three-dimensional position of the feature point when the i-th and first frame images are input.

〔Effect of the invention〕

As explained above, according to the second aspect of the present invention, it is determined which of the N pre-stored movement forms the movement was performed, and a certain movement form among the N movement forms is determined. When moving, each feature point is calculated using at least two feature points, the coordinates of these feature points on the two frame images, and the distance of the reference point among the feature points to the image input device. This is because it is possible to extract the three-dimensional position based on the distance of the reference point of the feature point to the image input device.

There is no need to move an object in a predetermined movement mode and amount of movement, or manually measure the movement form and amount of movement, and just observe the object in motion. Dimensional position information can be extracted.

Furthermore, according to the present invention, by increasing the number N of specific movement modes stored in advance, even if the restrictions on the movement of the object are weakened, the One of the N ways of movement that I remember
It is now possible to obtain two frame images in which movement corresponds to , and three-dimensional position information can be extracted more efficiently in terms of the effort and time required for extraction.

[Brief explanation of drawings]

Fig. 1 is a block diagram of a three-dimensional position information extraction processing power formula showing an embodiment of the present invention, Fig. 2 is a diagram showing the processing procedure of the movement mode discrimination section of Fig. 1, and Fig. 3 is a camera model. FIG. 4 is a diagram showing the correspondence between a three-dimensional positional change of a feature point due to the movement of an object and a two-dimensional positional change of a feature point on a frame image. 101... Image input section, 102... Feature point extraction section. 1.03...Movement mode discrimination unit 104...Reference point distance giving unit 105...Position information extraction unit. 201...Counting process of the number M of feature points. 202...Movement information extraction process 203...Discrimination process to determine whether the six MCn pieces of movement information match each other 204...Counting process for the number of cases in which the extraction results in 203 match 205...Control 21 Signal output processing 206: Output processing of a signal representing the movement mode.

Claims (1)

[Claims] In a three-dimensional position information extraction processing method for observing an object using an image input device and extracting three-dimensional position information, two of a series of frame images obtained by observing the object are provided.
Feature points are extracted from two frame images, and the coordinates of the extracted feature points on the two frame images are used to move the object three-dimensionally between the two frame images in N ways stored in advance. a moving form discriminating means for discriminating whether the movement is one of specific moving forms, and the moving form discriminating means stores in advance the three-dimensional movement of the object between the two frame images. If it is determined that the movement is one of the N movement modes set, three of the feature points based on one of the feature points are determined from the coordinates of the feature points on the two frame images. A three-dimensional position information extraction processing method characterized by extracting dimensional positions.