JPH10320558A - Calibration method, corresponding point search method and device therefor, focus distance detection method and device therefor, three-dimensional position information detection method and device therefor, and recording medium - Google Patents

Abstract

(57) [Problem] To obtain position information of a three-dimensional object when a focal length changes, a special device must not be attached to a zoom lens in order to detect a current focal length. I didn't. A stereo camera (8) includes a first camera (4) having a fixed focal length lens and a second camera (5) having a variable focal length lens. Image data storage unit 10
Accumulates image data from the first camera 4 and the second camera 5. The calculation unit 10 calculates three-dimensional position information of the object 9 from each image data from the data storage unit 10. The calibration data storage unit 12 stores calibration data obtained by the calibration device.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a calibration method suitable for a so-called stereo camera system using a plurality of CCD cameras or the like. A corresponding point searching method and apparatus for photographing and searching for a corresponding point on a two-dimensional coordinate system of another imaging apparatus that matches a projected image on a two-dimensional coordinate system of a reference imaging apparatus, and a lens having a fixed focal length The same image of the same object is taken by the first image pickup device provided with the second image pickup device provided with the lens having a variable focal length, and the first image pickup device is used to take an image of the object on the two-dimensional coordinate system. A focal length detection method and apparatus for determining the focal length of the second imaging device based on the one projected image, and a first imaging device having a fixed focal length lens and a variable focal length lens. A three-dimensional coordinate position detecting method and apparatus for photographing the same object with the second imaging device and detecting the position of the object in a three-dimensional coordinate system, and a recording recording the processing procedure of each of the above methods Regarding the medium.

[0002]

2. Description of the Related Art A so-called stereo camera is constituted by using a plurality of video cameras (hereinafter, simply referred to as cameras), for example, two cameras, and each camera images an object in a three-dimensional space. As a three-dimensional position information detection device that obtains position information of an object in a three-dimensional space from a two-dimensional image obtained as a result of imaging, a stereo camera system includes:
Conventionally known.

[0003] For the stereo camera system, for example, Koichiro Exit, "Geometry for Computer Vision: 2. Unraveling the Device of Stereo"
ol. 37, no. 7, pp. 662-670,199
Details are disclosed, for example, in July 2006.

In a stereo camera system, a target object (target object) for obtaining position information in a three-dimensional space is
When shooting with a plurality of cameras, the light receiving surface of a photoelectric conversion element such as a CCCD (hereinafter, appropriately referred to as
From the position information of the target object projected on the screen), the position information of the target object in the three-dimensional space can be specified. Therefore, when the position information of the object existing at a certain position in the three-dimensional space and the target object exists at that position,
It is necessary to determine in advance the correspondence relationship (position information correspondence relationship) between the target object and the position information projected on the screen of each camera. Obtaining this positional information correspondence is called calibration.

First, the calibration will be described. FIG. 14 shows a calibration device that performs calibration.

In FIG. 14, pipes 101 and 102
Are arranged so as to be included in the same plane and to be parallel in a three-dimensional space, and the carriage 103 is installed so as to be able to move smoothly along the pipes 101 and 102. . The cart 103 has cameras 104 and 105 constituting a stereo camera.
Is attached.

Scales are marked on the pipes 101 and 102 so that the amount by which the carriage 103 slides can be measured. A flat plate 106 on which a square lattice pattern is written is installed perpendicular to the direction in which the carriage 103 slides. As shown in FIG. 14, the horizontal direction of the square lattice is the x axis, the vertical direction is the y axis, and the sliding direction, that is, the direction perpendicular to the square lattice is the z axis. On the camera side with respect to the flat plate 106, z> 0. Such three-dimensional coordinates including the x-axis, the y-axis, and the z-axis are defined as world coordinates.

The measurement of the calibration is performed by the camera 10
This is performed by shifting the position of the carriage 103 on which the plates 4 and 105 are placed and photographing the flat plate 106 from two places. FIG.
FIG. 15 is an explanatory diagram in a case where a photograph is taken from a location, and is a diagram when the device in FIG. 14 is viewed from directly above.

First, the camera 104 and the camera 105 are
Fixed in a position P _1, the flat plate 106, a square lattice is taken as reflected, then move the camera 104 and 105 to the other position P ₂ by the sliding of the carriage 103 by a distance L, again, flat 106 To shoot. Note that FIG.
In the embodiment, the cameras 104 and 105 are slid in a direction away from the flat plate 106, but the sliding directions may be reversed.

As described above, the camera 104 and 105 are slid and the flat plate 106 is photographed.
As shown in FIG. 16, the two-dimensional images are cameras 104 and 1
05 can be obtained by fixing the plate 05 and sliding the flat plate 106.

That is, as shown in FIG.
04 and 105, and fixed to a position P _1, the flat plate 10
6, the plate 106 is slid along the z-axis by a distance L in a direction away from the cameras 104 and 105, and the plate 106 is photographed again at that position. By doing so, a similar two-dimensional image can be obtained.

In FIG. 16, the lower left corner of the square lattice (first square lattice Q ₁ ) drawn on the flat plate 106 before being separated by the distance L is set as the origin and the origin of the world coordinates is defined as In the first square lattice Q ₁ ,
The position of (i, j) is (i, j,
0). Further, the position of (i, j) in the square lattice (second square lattice Q ₂ ) of the flat plate 106 after sliding by the length L is (i, j, −) in the world coordinates.
L).

FIG. 17 is a diagram showing the camera 104 and the first square lattice Q ₁ and the second square lattice Q ₂ on the flat plate 106. The optical center of the camera 104 is O ₁ , for example, CC
Position information of the target object is projected on a screen serving as a light receiving surface such as D111. For example, the coordinate position on the CCD 111 (h, k), the first coordinate positions in a square lattice Q ₁ of (p, q) is projected, and a second square lattice Q
_It is assumed that the coordinate position (r, s) in 2 is projected.
In addition, the place other than the intersection of each vertical line and horizontal line of each square lattice,
The coordinates can be obtained by interpolation.

To explain this again using the world coordinates, the coordinate positions (h, k) on the CCD 111 are three-dimensional coordinate positions (p, q, 0) and (r, s, -L). ) Is projected. That is, two-dimensional coordinate position (h, k), three-dimensional coordinate position (p, q, 0) and (r, s, -L)
Assuming that a straight line connecting the two is represented by l, it can be seen that all points on the straight line l are displayed at the coordinate position (h, k) on the CCD 111.

Therefore, the straight line 1 is composed of the position information of the object in the three-dimensional space (here, the coordinates in the world coordinate system) and the position information of the two-dimensional image obtained by imaging the object (here, the position on the CCD 111). It represents the correspondence (positional information correspondence) with the two-dimensional coordinate system.

This straight line 1 is represented by (xr) / (pr) = (ys) / (qs) =
(Z + L) / L.

In the same manner as when the straight line 1 is obtained, the CCD 1
With respect to other coordinate positions in the two-dimensional coordinate system 11 as well, a straight line as a set of points in the three-dimensional space projected thereon is obtained. Further, the same is performed for the camera 105.

As described above, the cameras 104 and 10
The calibration of the stereo camera system is completed by finding all the straight lines for 5.

In the stereo camera system thus calibrated, 3
Position information of an object in a dimensional space can be obtained.

First, an object is photographed with a stereo camera.
Then, it is assumed that, for example, as shown in FIG. 18, an object is projected on the coordinate position (a, b) of the screen (CCD) 111 of the camera 104. In the calibration, since the straight line l ₁ in the world coordinate system corresponding to the coordinate position (a, b) has been obtained, the object exists on this straight line l ₁ in the world coordinate system.

The screen of the camera 105 (CC
D) Straight lines in the world coordinate system corresponding to the respective positions of 112 are also obtained in the calibration. Therefore, of these straight lines, with the straight line l ₁ and the intersection, it is possible to pick out straight. The selected straight lines are 113 ₁ , 113 ₂ , 113 _{i... In} the figure.

The point on the screen 112 of the camera 105 corresponding to the straight line 113 ₁ is 114 ₁ . Similarly, 1
The point corresponding to 13 ₂ is 114 _2, the point corresponding to 113 _i is 114 _i. Others are the same. Therefore, as can be seen, all these straight lines 113 ₁ , 11 ₁
Collecting the points 114 ₁ , 114 ₂ , 114 _{i... Of} the screen 112 of the camera 105 corresponding to 3 ₂ , 113 _i . This straight line 115
Is generally called an epipolar line.

As described above, the object shown at the coordinate position (a, b) of the screen 111 of the camera 104 is a straight line l.
Somewhere on the _1. Therefore, on the screen 112 of the camera 105, the object should be projected somewhere on the epipolar line 115. Therefore, the projected image of the object may be searched on the epipolar line 115. For example, the point (c,
If the projection image exists in d), the straight line l ₂ corresponding to the point (c, d) is determined. This straight line l ₂ is a straight line 113 ₁ , 1
13 ₂ , 113 _i... After that, a straight line l
₁ and, by finding an intersection of the straight line l ₂ in the world coordinate system, it is possible to know the position 116 in the three-dimensional object.

Here, a method of searching for a corresponding point on the epipolar line 115 on the screen 112 corresponding to the coordinate position (a, b) on the screen 111 will be described. Usually, the search is performed by a method called block matching. For example, a block of 4 pixels × 4 pixels around the coordinate position (a, b) of the screen 111 of the camera 104 is extracted. Next, the screen 1 of the camera 105
Focusing on a certain point on the 12 epipolar lines 115,
A block of 4 pixels × 4 pixels is extracted centering on the point. Then, the correlation between the two blocks is calculated. As the calculation of the correlation, for example, a method of calculating the sum of absolute differences of each pixel can be used.

The sum of absolute differences is calculated for all points on the epipolar line 115, and the point having the highest correlation is determined as the projection point of the object. Because the object is projected on the screen 111 of the camera 104,
That is, 4 pixels × 4 pixels centered on the coordinate position (a, b) and an image projected on the screen 112 of the camera 105, that is, 4 pixels × 4 pixel centered on the coordinate position (c, d)
This is because the image is similar to the four pixels.

Incidentally, the focal lengths of the two cameras used in the stereo camera system described so far have always been fixed. That is, there has been no use of a zoom lens that changes the focal length.

In recent years, for example, a virtual technique has been widely used in which a real person is photographed by a camera, and a robot made by CG (computer graphic) is synthesized with the photographed image so as to be positioned beside the person. Became known. At this time, unless the position of the human in three dimensions and the position of the robot in virtual three dimensions are accurately matched, the synthesized image looks unnatural. Therefore, the position of the person must be accurately measured in three dimensions.
Measure using the stereo camera system described above. Then, the image data of the human being used for the synthesis is stored in one camera of the stereo camera system (for example,
Image of the second camera). That is, if a stereo camera is used, image data of a person and the position of the person in three dimensions can be obtained at a time. After that, the projection image of the robot that appears in the camera when the robot is assumed to be next to a human in three dimensions may be combined with the image of the second camera. In such a virtual studio that performs photographing using virtual technology, a person is naturally photographed by zooming in or out. Therefore, a zoom lens is required for the second camera among the stereo cameras. Thus, a stereo camera using a zoom lens is required.

However, when a zoom lens is used, the focal length changes, and naturally, a straight line (1 in FIG. 18) corresponding to each point on the screen in the world coordinate system.
13 ₁ , 113 ₂ , 113 _i... ) Change. In other words, since the epipolar line 115 depends on the focal length, it is unclear how to search for the corresponding point.

Of course, if a special device is attached to the zoom lens so that the current focal length can be detected, this can be dealt with. In other words, if you know the focal length at that time,
What is necessary is just to find the epipolar line corresponding to the focal length, and to search on that line.

[0030]

However, in order to obtain three-dimensional object position information when the focal length changes, as described above, a special zoom lens is required to detect the current focal length. I had to put on a special device.

The present invention has been made in view of the above-described circumstances, and has as its object to provide a calibration method that enables calibration of an imaging apparatus that changes the focal length.

Further, the present invention has been made in view of the above-mentioned circumstances, and a corresponding point search method and a corresponding point search method capable of detecting coincidence of a projected image of an object on a two-dimensional coordinate system between imaging devices having different focal lengths. The purpose is to provide the device.

Further, the present invention has been made in view of the above-mentioned circumstances, and the first image pickup apparatus having a fixed focal length lens and the second image pickup apparatus having a variable focal length lens have the same object. And a focal length for obtaining the focal length of the second imaging device based on a first projection image on a two-dimensional coordinate system obtained by imaging the object with the first imaging device. It is intended to provide a detection method and device.

Further, the present invention has been made in view of the above circumstances, and is the same between a first image pickup apparatus having a fixed focal length lens and a second image pickup apparatus having a variable focal length lens. It is an object of the present invention to provide a three-dimensional coordinate position detection method and apparatus capable of photographing an object and detecting the position of the object in a three-dimensional coordinate system.

Further, the present invention has been made in view of the above-mentioned circumstances, and records processing procedures for realizing the corresponding point searching method, the focal length detecting method, and the three-dimensional position information detecting method. The purpose is to provide a recording medium.

[0036]

According to the present invention, there is provided a calibration method for imaging an object at a plurality of specific focal lengths by an imaging device for imaging an object with a variable focal length. The position information correspondence between each position of the projection image of the two-dimensional coordinate system obtained in this way and each position of the object in the three-dimensional coordinate system is obtained. Specifically, perform calibration at several focal lengths,
Calibration at the remaining focal length is obtained by interpolation.

Further, in order to solve the above-mentioned problems, the method and apparatus for searching for a corresponding point according to the present invention provide a method for determining the size of a cutout block centered on a corresponding point candidate on a two-dimensional coordinate system of another imaging apparatus. Is changed according to the ratio of the focal length between the reference imaging device and the other imaging device, and then has the same size as a cutout block centered on a projected image of the reference imaging device on a two-dimensional coordinate system. The corresponding points are searched for by adjusting the degree of correspondence and calculating the degree of correspondence between the two blocks.

According to another aspect of the present invention, there is provided a method and apparatus for detecting a focal length, wherein a second projected image on a two-dimensional coordinate system obtained by imaging an object by a second imaging device is provided. Is obtained by imaging the object with the first imaging device.
The correlation with the first projection image on the dimensional coordinate system is calculated and searched, and the focal length of the second imaging device is obtained from the search result.

Further, in order to solve the above-mentioned problems, a method and apparatus for detecting three-dimensional position information according to the present invention provide a method for detecting a second object on a two-dimensional coordinate system obtained by imaging an object using a second imaging device. The projection image is searched by calculating a correlation with a first projection image on a two-dimensional coordinate system obtained by imaging the object by the first imaging device, and from the search result, the second imaging device Is obtained, and the three-dimensional position of the object is detected from the focal length.

In the recording medium according to the present invention, in order to solve the above-mentioned problem, the size of a cutout block centered on a corresponding point candidate on a two-dimensional coordinate system of another imaging apparatus is used as a reference. A step of changing the focal length of the imaging device and the other imaging device according to a ratio of the focal length, and a clipping block centered on the projected image on the two-dimensional coordinate system of the imaging device serving as the reference and the variable clipping block. A processing procedure is recorded that includes a step of adjusting the size to the same size and a step of determining the degree of correspondence between the adjustment block and the cutout block centered on the projection image.

Further, in order to solve the above-mentioned problems, the recording medium according to the present invention converts a second projected image on a two-dimensional coordinate system obtained by imaging an object with a second imaging device into a first image. The first of a two-dimensional coordinate system obtained by imaging an object with an imaging device
The processing procedure includes a step of calculating and searching for a correlation with respect to the projection image, and a step of calculating the focal length of the second imaging device from the search result.

Further, in order to solve the above-mentioned problems, the recording medium according to the present invention converts a second projected image on a two-dimensional coordinate system obtained by imaging an object by a second imaging device into a first image. Calculating and searching for a correlation with respect to a second projected image on a two-dimensional coordinate system obtained by imaging the object with the imaging device;
A processing procedure including a step of obtaining a focal length of the second imaging device from the search result and a step of detecting a three-dimensional position of the object from the focal length is recorded.

[0043]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a calibration method according to the present invention will be described below. This embodiment is a calibration apparatus that performs calibration at several focal lengths using the calibration method according to the present invention.

As shown in FIG. 1, this calibration apparatus is composed of a camera 4 having one fixed focal length (hereinafter referred to as appropriate).
This is a device for calibrating a stereo camera including a first camera 5) and a camera 5 with a zoom function (hereinafter, appropriately referred to as a second camera) 5.

The calibration device includes pipes 1 and 2 which are included in the same plane and are arranged in parallel in a three-dimensional space, a truck 3 which moves smoothly along the pipes 1 and 2, and a truck 3 Is provided perpendicular to the sliding direction, and has a flat plate 6 on which a square lattice pattern is written.

Scales are marked on pipes 1 and 2,
The amount by which the carriage 3 slides can be measured. The cart 3 slides the pipes 1 and 2 with the stereo camera mounted thereon.

The horizontal direction of the square lattice written on the flat plate 6 is the x-axis, the vertical direction is the y-axis, and the sliding direction, that is, the direction perpendicular to the square lattice is the z-axis. Z> 0 on the camera side from the flat plate 6.

The calibration apparatus shown in FIG. 1 having such a configuration is similar to the calibration apparatus shown in FIG. 14 except that a second camera 5 having a zoom function is mounted. , How to slide,
The same applies to how to obtain three-dimensional coordinates.

The actual calibration measurement is based on the first
The position of the carriage 3 on which the camera 4 and the second camera 5 are mounted is shifted, and the photographing is performed while photographing a square lattice of the flat plate 6.

Here, as in the case of FIGS. 14 to 16, in the calibration apparatus of FIG.
Is moved by the distance L, and the first square lattice Q ₁ and the second square lattice Q ₂ on the flat plate 6 shown in FIG. 2 are photographed with reference to the first camera 4 and the second camera 5. I can do it. Also, the method of obtaining world coordinates is the same.

Screen of the first camera 4 (CCD surface)
The three-dimensional object that appears in each coordinate position of, on which straight line in the world coordinate system, can be determined as described with reference to FIG. That is, the first
The correspondence between each point on the screen of the camera 4 and the straight line in the world coordinate system is obtained.

Next, a second camera having a zoom function will be described.
Focal length specific focal length of the camera 5 (an f ₁₎
And measure the calibration. That is,
After placing the focal length f _1, sliding the carriage 3 by a distance L without moving the focal length. Thus for the second camera 5 when the focal length f _1, as in the prior art, the calibration can be performed. That is, each point of the screen of the second camera 5 when the focal length f _1, the correspondence between the lines in the world coordinate system determined. However, this correspondence is the relationship under the condition that the focal length is f _1.

Next, the carriage 3 is slid back to the original position. Then, by setting the focal length of the second camera 5 to another particular focal length (the f _2), to measure the calibration. In other words, if you set the focal length f _2,
The carriage 3 is slid by the distance L without moving the focal length. Thus, calibration can be performed. In other words, each point of the screen of the second camera 5 when the focal length f _2, the correspondence between the lines in the world coordinate system determined. However, this correspondence relationship indicates that the focal length is f
It is a relationship under the condition that it is ₂ . Similarly, calibration is performed at some focal lengths (f ₃ , f ₄ ,...) Of the second camera 5 using the calibration device.

As described above, the correspondence between each point on the screen of the first camera 4 and the straight line in the world coordinate system is obtained. The focal length of the second camera 5 is f ₁ , f ₂ ,
f ₃ , f ₄ ,. . . In this case, the correspondence between each point on the screen of the second camera 5 and a straight line in the world coordinate system is obtained.

[0055] _{Further, f 1, f 2, f} 3, f 4 the calibration at the focal length of the non _... is the focal length f _1,
It is obtained by interpolation from the correspondence between each point on the screen at f ₂ , f ₃ , f _4... and a straight line in the world coordinate system.

That is, according to the above-described calibration apparatus to which the calibration method according to the present invention is applied, calibration of a stereo camera having one camera using a zoom lens can be performed without adding a special device. Can be done.

The application of the calibration method according to the present invention to the calibration apparatus shown in FIG. 1 of a stereo camera comprising one camera 4 having one fixed focal length and one camera 5 having a zoom function is limited. Instead, as shown in FIG. 3, the camera comprises two cameras (first camera) 4 and (third camera) 7 having a fixed focal length, and one camera (second camera) 5 having a zoom function. It may be applied to a calibration device of a stereo camera.

The measurement of the actual calibration by the calibration apparatus shown in FIG. 3 is performed by the first camera 4 and the third camera 7, which are cameras having a fixed focal length, and the first camera 4 and the third camera This is performed while shifting the position of the carriage 3 on which the second camera 5 which is a camera with a zoom function, which is positioned between the cameras 7, is photographed and the square lattice of the flat plate 6 is photographed.

Here, as in the case of FIGS. 14 to 16, in the calibration apparatus of FIG.
Is moved by the distance L, so that the first camera 4,
With reference to the second camera 5 and the third camera 7, FIG.
The first square lattice Q ₁ and the second square lattice Q ₂ on the flat plate 6 shown in FIG. Also, the method of obtaining world coordinates is the same.

FIG. 4 shows the relationship between each point on the screen of each of the first camera 4 and the second camera 5 described with reference to FIG. 2 and a straight line in the world coordinate system. Thus, the relationship between each point on the screen of the third camera 7 and a straight line in world coordinates may be added.

Thus, the calibration at several focal lengths by the calibration device to which the calibration method according to the present invention is applied is completed.

Using the stereo camera calibrated in this way, an actual object is photographed. In addition,
Although the description is omitted in FIGS. 1 and 3, the plurality of cameras in the figures are integrated, and even when the camera is removed from the calibration device and an actual object is photographed, the positional relationship between the cameras is not changed. Can be kept.

Next, an embodiment of a three-dimensional position information detecting method and a three-dimensional position information detecting method for obtaining the position of an object in a three-dimensional space by using the calibration obtained by a calibration apparatus to which the above-described calibration method is applied will be described.

This embodiment has a configuration as shown in FIG. 5, and comprises a first camera 4 which is one camera having a fixed focal length and a second camera 5 which is one camera with a zoom function. The distance to the photographed object 9 is measured from the image data photographed by the stereo camera 8, the three-dimensional position information is outputted from the position output terminal, and the image data from the second camera 5 is outputted from the image output terminal. It is a three-dimensional position information detecting device. In the process in which the three-dimensional position information detecting device obtains the three-dimensional position information by applying the three-dimensional position information detecting method, the technique of the corresponding point searching method and the focal length detecting method according to the present invention is used.

The three-dimensional position information detecting device is provided with the first
Camera 8 consisting of camera 4 and second camera 5
An image data storage unit 10 for storing image data from the first camera 4 and the second camera 5; and three-dimensional position information from the image data from the data storage unit 10 to the object 9 finally. An arithmetic unit 11 that performs an operation, a calibration data storage unit 12 that stores calibration data obtained by the calibration device illustrated in FIG. 1, and a control unit 13 are provided.

The calibration data stored in the calibration data storage unit 12 is stored in advance by the calibration device shown in FIG. 1 by associating each point of the screen of the first camera 4 with a straight line in the world coordinate system. Relationship, for each focal length of the second camera 5,
This is calibration data obtained by determining the correspondence between each point on the screen of the second camera 5 and a straight line in the world coordinate system. Since the calibration data includes data indicating the correspondence between the focal lengths of the second camera 5 and the straight lines in the world coordinate system, if the focal lengths of the second camera 5 are known, The three-dimensional position information up to the object can be obtained.

However, when several objects are photographed using the stereo camera 8, the focal length of the second camera 5 is unknown. Conventionally, a special device is attached to a lens to mechanically measure the focal length to obtain the focal length. However, this three-dimensional position information detecting device uses the focal length detecting method according to the present invention. Thus, the focal length is obtained by image processing.

Hereinafter, the operation of the three-dimensional position information detecting device will be described with reference to the flowchart of FIG. 6. First, the focus according to the present invention for obtaining the focal length of the second camera 5 by image processing will be described. The distance detection method will be described. This focal length detection method is executed by the calculation unit 11 under the control of the control unit 13.

This method of detecting the focal length uses the first camera 4
And the second image obtained by the second camera are taken out, and a corresponding point in the second image corresponding to a specific point in the first image is searched for. The focal length of the second camera is obtained from the search result.

That is, the same object is photographed by the first camera 4 having the fixed focal length lens and the second camera 5 having the variable focal length lens, and the object is photographed by the first camera 4. Is a focal length detection method for obtaining the focal length of the second camera 5 based on the first projection image on the two-dimensional coordinate system obtained by the above. The focal length detection method is obtained by imaging an object with the second camera 5. The second projection image on the two-dimensional coordinate system is searched for by calculating the correlation with the first projection image, and the focal length of the second camera 5 is obtained from the search result.

Here, the specific point in the first image is
It is necessary to select from outside a predetermined area connecting the center of the screen of the first camera 4 and the center of the screen of the second camera 5. That is, as shown in FIG. 7, of the projection images of the object 9 projected on the screen 14 of the first camera 4,
A part other than the direction 15 connecting the camera 4 and the second camera 5 is selected. Direction 1 connecting first camera 4 and second camera 5
The portion shown in FIG. 5 is a hatched portion 16 shown in FIG.

If the positions of the two cameras are horizontal, the first
A horizontal portion passing through the center of the screen 14 of the camera 4 is a direction connecting the first camera 4 and the second camera 5.

Here, the reason for selecting from a portion other than the portion 16 projected in the direction 15 connecting the first camera 4 and the second camera 5 will be described later.

According to the above conditions, the control unit 13 operates the arithmetic unit 1
In step S1 of the flowchart in FIG. 6, the coordinate position of the object 9 on the screen 14 of the first camera 4 is obtained as (a, b) as shown in FIG. The coordinate position (a, b) is located at a position other than the hatched portion 16 in FIG.

Next, at step S2, the coordinate position (a,
A straight line l ₁ in the world coordinate system corresponding to b) is obtained with reference to the data in the calibration storage unit 12.

Next, in step S3, the second camera 5
To set the focal length of, for example, the f _1. The straight line in the world coordinate system corresponding to each position of the screen 17 of the second camera 5 when the focal length is assumed to be f ₁ is also obtained by the calibration device and stored in the calibration data storage unit 12. I have.

Therefore, it is possible to select a straight line having an intersection with the straight line l ₁ among these straight lines. The selected straight lines are 18 ₁ , 18 ₂ , and 18 _i in FIG. The point on the screen 17 of the second camera 5 corresponding to the straight line 18 ₁ is 19 ₁ . Similarly, the point corresponding to the straight line 18 ₂ is 19 ₂ , and the point corresponding to the straight line 18 _i is 19
_i . Others are the same. Thus, as can be seen, points 19 ₁ , 19 ₂ , 17 b on the screen 17 of the second camera 5 corresponding to all these straight lines 18 ₁ , 18 ₂ , 18 _i .
When 19 ₂ and 19 _i are collected, a straight line 20f ₁ is obtained. The straight line 20f ₁ is a epipolar line when the focal length was to be f _1.

As described above, under the control of the control unit 13, the arithmetic unit 11 sets the epipolar line assuming the focal length set in step S3 as the focal length of the second camera 5 in step S4. Ask.

Next, in step S5, the coordinate positions (a, b) on the epipolar line obtained in step S4
Is calculated.

Next, it is determined in step S6 whether or not the correlation has been checked for all the focal lengths. If not, steps S4 and S5 are repeated while setting another focal length in step S9, and The correlation with the coordinate position (a, b) on each epipolar line assuming the focal length is calculated.

Then, the position on the screen 17 of the second camera 5 when the correlation is the strongest in step S7 is obtained, and the three-dimensional position of the object is calculated from the focal length at that time.

Finally, in step S8, the position of the object is output from the position output terminal, and the image data from the second camera 5 is output from the image output terminal via the image data storage unit 10.

The focal lengths on the screen 17 of the second camera 5 corresponding to the positions of the points (a, b) on the screen 14 of the first camera 4 are f ₂ , f ₃ , f _{4. -} is a and when the in from seeking epipolar line first may take the correlation on each epipolar line.

FIG. 9 shows the screen 1 of the second camera 5.
On 7, the epipolar line 20f ₁ when the focal length was to be f _1, the epipolar line 20f ₂ when the focal length was to be f _2, epipolar when the focal length was to be f ₃ shows a state in which the line f ₃ is obtained.

Then, the same image data as the point (a, b) on the screen 14 of the first camera 4 is searched for at all points on all the straight lines shown in FIG. This is a search in two dimensions. For example, when the image data of the position of the point on the screen 17 of the second camera 5 (c, d) are the same, it can be seen the focal length of the second camera 5 is f _2. This is because the point (c, d) is not supposed to be on the epipolar line, the epipolar line 20f ₂ including the point (c, d) is that the thing when the focal length of f _2.

Here, the reason why “the part other than the direction 15 connecting the first camera 4 and the second camera 5 in the projection image of the object 9 projected on the screen 14 of the first camera 4, that is, the area The point (a, b) is selected as shown in FIG. 8 from the portion excluding 16 ".
If a point in the hatched portion 16 in FIG. 7 is selected, the corresponding epipolar line on the screen 17 of the second camera 5 becomes the same regardless of the focal length. That is, the epipolar lines 20f ₁ , 20f ₂ and 20f ₃ shown in FIG. 9 become the same straight line. In this case, the screen 1 of the second camera 5 corresponding to the point (a, b) of the screen 14 of the first camera 4
This is because even if the point on 7 is searched, the focal length cannot be obtained from the result.

Next, the screen 17 of the second camera 5
Each of the above epipolar lines 20f ₁ , 20f ₂ , 20f ₃
Above, a point (a,
A method for searching for the position of the image data corresponding to b) will be described in detail. This method is applied to the processing in step S5 in FIG.

This search method uses block matching. However, if the focal length differs between the first camera 4 and the second camera 5, the size of the corresponding block also changes. For example, if an object is shown in 4 pixels × 4 pixels of the screen 14 of the first camera 4, the screen 17 of the second camera 5 may be 4 pixels × 4 pixels depending on the focal length. Or 8 pixels × 8 pixels.

Then, using the corresponding point searching method according to the present invention, the corresponding points between the two images on the screen 14 and the screen 17 taken by the first camera 4 and the second camera 5 having different focal lengths. Ask for.

That is, in this corresponding point search method, the same object is photographed by the first camera 4 serving as a reference and the other second camera 5 and coincides with the projected image on the screen 14 of the first camera 4. Is a corresponding point search method for searching for a corresponding point on the screen 17 of the second camera 5. The size of a cutout block centered on a corresponding point candidate on the screen 17 of the second camera 5 is After changing according to the ratio of the focal length between the first camera 4 and the second camera 5, the size is adjusted to the same size as the cutout block centered on the projected image of the first camera 4 on the screen 14. And
By finding the degree of correspondence between the two blocks, a corresponding point on the screen 17 is searched.

Finally, each epipolar line 2 shown in FIG.
0f _1, 20f _2, 20f ₃ on _..., searches for a part which is the same image data as the point on the screen 14 of the first camera 4 (a, b). That is, for example, a block of 4 pixels × 4 pixels is extracted from a point (a, b) on the screen 14 of the first camera 4. The same part as the image data shown in this block is
f ₁ , 20f ₂ , 20f _3... One is selected from each of the above points.
This is a search.

Here, for example, among these epipolar lines 20f ₁ , 20f ₂ , 20f _3... , Let us consider the epipolar line 20f ₂ when the focal length is f ₂ . Assuming that the focal length of the first camera 4 is f, the object projected on the 4 × 4 pixels on the screen 14 of the first camera 4 is (4 × 4) on the screen 17 of the second camera 5.
The image is projected onto (f ₂ / f) pixels × (4 × f ₂ / f) pixels. Therefore, when searching on the epipolar line 20f ₂ , (4 × f ₂ / f) pixels × (4 × f ₂ / f) pixels are extracted and matching is performed.

This is generalized as follows. The position (a, b) of the screen 14 of the first camera 4 on the epipolar line 20f _i when the focal length is f _i
It is checked whether there is a part having a strong correlation with a block of 4 × 4 pixels centered at. At this time, the epipolar line 20
around the individual points on the _{f i, (4 × f i} / f) pixels × (4
× f _i / f) A pixel is cut out and reduced or enlarged to create a block of 4 × 4 pixels. Then, the correlation between the created block and a block of 4 × 4 pixels centered on the point (a, b) of the screen 14 of the first camera 4 is checked. If the correlation is high, that is the point to seek. If there is no point having a high correlation, block matching is similarly performed at each point on the epipolar line corresponding to another focal length, and a point having a high correlation can be finally obtained.

Referring to FIG. 10, a method for creating a block of 4 pixels × 4 pixels will be described again. 4 pixels × 4 centering on a point (a, b) of the screen 14 of the first camera 4 in the figure
For the block of pixels 21, (4 × f _i / f) pixels × (4 × f _i / f) pixels centered on a point on the epipolar line 20 f _i on the screen 17 of the second camera 5. Is enlarged or reduced to create a block 23 of 4 pixels × 4 pixels. This enlargement or reduction is an enlargement or reduction of an image, and is well known, and the details are omitted here.

In the above description, the size of the block is 4 pixels × 4 pixels, but the size may be another size. Also, besides block matching, for example,
It is conceivable to search for matching points by checking the shape matching, but in this case, it is sufficient to search by expanding or reducing to f _i / f.

Now, the point corresponding to the point (a, b) on the screen 14 of the first camera 4 is obtained on the screen 17 of the second camera 5 to obtain the focal length of the second camera 5. However, the same operation can be performed for another point on the screen 14 of the first camera 4 to obtain the focal length of the second camera 5. When the focal length is calculated from a plurality of points on the screen 14 of the first camera 4 as described above, for example, the most probable focal length of the first camera 5 may be obtained from the calculation result by the least square method or the like. I can do it.

Thus, the focal length of the second camera 5 can be obtained by applying the focal length detecting method and the corresponding point searching method according to the present invention.

By the way, the calibration data storage unit 12 of the three-dimensional position information detecting device already stores the calibration data obtained by the calibration device shown in FIG. When the focal length is obtained by applying the corresponding point searching method and the focal length detecting method, each point on the screen 17 of the second camera 5 in the case of the focal length is obtained from the calibration data storage unit 12. And a straight line in the world coordinate system, the three-dimensional position to each object can be measured by considering the camera as a stereo camera having a fixed focal length.

The three-dimensional position information detecting device shown in FIG. 5 includes a first camera 4 which is a camera having a fixed focal length and a second camera 5 which is a camera having a zoom function. Although the stereo camera 8 is used, the method and apparatus for detecting three-dimensional position information according to the present invention may use a stereo camera including a plurality of cameras having a fixed focal length and one camera with a zoom function.

FIG. 11 shows a configuration of a three-dimensional position information detecting device using a stereo camera 25 comprising, for example, two cameras having a fixed focal length and one camera having a zoom function. The two fixed focal length cameras are referred to as a first camera 4 and a third camera 7. One camera with a zoom function sandwiched between the first camera 4 and the third camera 7 is referred to as a second camera 5.

The three-dimensional position information detecting device includes a stereo camera 25 including a first camera 4, a second camera 5, and a third camera 7, and an image storing image data from the stereo camera 25. A data storage unit 26, a calculation unit 27 that finally calculates three-dimensional position information from the image data from the image data storage unit 26 to the object 9, and a calibration obtained by the calibration device shown in FIG. A calibration data storage unit 28 for storing the calibration data and a control unit 29 are provided.

The calibration data stored in the calibration data storage unit 28 is stored in advance by the calibration device shown in FIG. 3 by associating each point of the screen of the first camera 4 with a straight line in the world coordinate system. Relationship, for each focal length of the second camera 5,
Calibration obtained by finding the correspondence between each point on the screen of the second camera 5 and a straight line in the world coordinate system, and the correspondence between each point on the screen of the third camera 7 and a straight line in the world coordinate system. Data. Since the calibration data includes data indicating the correspondence between the focal lengths of the second camera 5 and the straight lines in the world coordinate system, if the focal lengths of the second camera 5 are known, The three-dimensional position information up to the object can be obtained.

However, when several objects are photographed using the stereo camera 25, the focal length of the second camera 5 is unknown. In order to obtain the focal length, the three-dimensional position information detecting device also obtains the focal length by image processing using the focal length detecting method according to the present invention.

First, the first camera 4 and the third camera 7 are regarded as one stereo camera, and the three-dimensional positions of the objects shown on the screens of the first camera 4 and the third camera 7 are measured. . It is assumed that a point (a, b) appears on the screen of the first camera 4. The position of this object 9 is
Since the positions can be known from the first camera 4 and the third camera 7, their positions are indicated by (e, f, g) in FIG. Next, the position of the projected image of the object 9 on the screen 17 of the second camera 5 is checked.

[0105] when the focal length is to be f _1, a second
The straight line in the world coordinate system corresponding to each position of the screen 17 of the camera 5 is also obtained by the calibration by the calibration device shown in FIG. Therefore, it is possible to select a straight line 30 passing through the coordinate position (e, f, g) in the world coordinates from these straight lines. The position of the screen 17 of the second camera 5 corresponding to the selected straight line 30 is represented by (c
f ₁ , df ₁ ).

Similarly, the focal lengths of the second camera 5 on the screen 17 are f ₂ , f ₃ , f ₄ ,. . . Is selected, a straight line passing through the coordinate position (e, f, g) is selected, and a second line corresponding to the straight line is selected.
Of the screen 17 of the camera 5 is determined. FIG. 13 shows the obtained result. 13, when the focal length is assumed to be f _1, the position of the object 9 is projected is (cf _1, df _1). Assuming that the focal length is f _2, the position where the object 9 is projected is (cf.
₂ , df ₂ ). When the focal length is assumed to be f _3, the position where the object is projected is (cf _3, df _3). Others are the same.

That is, according to the focal length of the second camera 5, (cf ₁ , df ₁ ), (cf ₂ , df ₂ ),
(Cf ₃ , df ₃ ). . . Should be projected to any of the positions. Therefore, the same image data as the position (a, b) on the screen of the first camera 4 is transferred to the (cf ₁ , df ₁ ), (cf ₂ , d) in the screen of the second camera 5.
f ₂ ), (cf ₃ , df ₃ ). . . Need only be searched for on the line 31 containing. In the three-dimensional position information detecting device shown in FIG. 5, the two-dimensional search had to be performed.
The three-dimensional position information detecting device shown in FIG. 11 needs to search only one dimension.

This search is performed by applying the block matching detection method according to the present invention and changing the size of the block.
Explore. That is, when examining whether there is a corresponding point at the position (cf ₁ , df ₁ ) obtained when the focal length is assumed to be f ₁ on FIG. 13, (4 × An (f _i / f) pixel × (4 × f _i / f) pixel is cut out and reduced or enlarged to create a block of 4 × 4 pixels. Then, the created block and a block 4 centered on the position (a, b) of the screen of the first camera 4 are set.
The correlation with a block of pixels × 4 pixels is examined. If the correlation is high, that is the point to seek. If the points are not highly correlated, points (cf ₂ , df ₂ ) corresponding to different focal lengths,
(Cf ₃ , df ₃ ),... Finally, a point having a high correlation can be obtained. The focal length corresponding to the strong correlation point is determined by the second camera 5 to be determined.
Is the focal length.

Now, the focal length has been obtained by obtaining a point corresponding to the position (a, b) of the screen of the first camera 4, but the same applies to another point of the screen of the first camera 4. To determine the focal length. When the focal length is calculated from a plurality of points in this way, for example, the most probable focal length can be obtained from the calculation results by the least square method or the like.

Thus, the three-dimensional position information detecting device shown in FIG. 11 can determine the focal length of the second camera. If the focal length is obtained, the correspondence between each point on the screen 17 of the second camera 5 and the straight line in the world coordinate system when the focal length is obtained is obtained at the calibration stage. Similarly to the above, it is possible to measure a three-dimensional position up to each object by considering a stereo camera having a fixed focal length.

According to the three-dimensional position information detecting device shown in FIGS. 5 and 11, the information of the three-dimensional position of the object shown in the camera is obtained and the image data of the second camera is output. Thus, at the same time, it is possible to obtain image data obtained by photographing those objects at their intended focal length.

Further, the applicant of the present invention
Japanese Patent Application No. 09-73202, filed on June 6,
According to “Calibration method and calibration device, and correction method and correction device”, a technology relating to a method for performing correction when a flat plate on which a square lattice pattern is written cannot be installed perpendicularly to a sliding direction is disclosed. However, in the present invention, calibration can be performed more accurately using this technique.

The above-described corresponding point search method includes the operation unit 11
And 27 under the control of the control units 13 and 29. Therefore, the first and fourth cameras 4 and 7 having a fixed focal length are used as the operation units 11 and 27 as corresponding point searching devices.
May be used only for searching for a corresponding point on the two-dimensional coordinate system of the second camera having a variable focal length that matches the projected image on the two-dimensional coordinate system.

The focal length detecting method is also the same as that of the arithmetic unit 1
This is executed under the control of the control units 13 and 29 in 1 and 27. Therefore, using the calculation units 11 and 27 as focal length detection devices, the second projection images on the two-dimensional coordinate system obtained by imaging the object with the second camera 5 having a variable focal length can be used as the first camera 4 and The correlation may be calculated with respect to the projection image obtained from the third camera 7 and searched, and the search result may be used only for calculating the focal length of the second imaging device from the search result.

In order to execute the corresponding point search method, the operation units 11 and 27 sequentially take out the processing procedure from the recording medium and execute the processing procedure under the control of the control units 13 and 29. Here, the recording medium is inside or outside the arithmetic unit 11, and the size of the cut-out block centered on the candidate of the corresponding point on the two-dimensional coordinate system of the second camera 5 is set to be equal to the size of the first camera 4. A step of changing the ratio of the focal length to the second camera 5 and the size of the variable cutout block as the size of the cutout block centered on the projected image of the first camera 4 on the two-dimensional coordinate system. , And a process of obtaining a degree of correspondence between the adjustment block and the cutout block centered on the projection image.

In order to execute the above-mentioned focal length detecting method, the arithmetic units 11 and 27 sequentially take out the processing procedures from the recording medium and execute them under the control of the control means 13 and 29. Here, the recording medium is inside or outside of the calculation units 11 and 27, and the second projection image on the two-dimensional coordinate system obtained by imaging the object with the second camera 5 is stored in the first camera. The processing procedure includes a step of calculating and searching for a correlation with respect to the first projection image obtained in Step 4 and a step of calculating the focal length of the second imaging device from the search result. .

In order to execute the above-described three-dimensional coordinate position detecting method, the operation units 11 and 27 sequentially take out the processing procedure from the recording medium and execute it under the control of the control means 13 and 29. Here, the recording medium is inside or outside of the calculation units 11 and 27, and the second projection image on the two-dimensional coordinate system obtained by imaging the object with the second camera 5 is stored in the first camera. A step of calculating and searching for a correlation with respect to the first projection image obtained in step 4; a step of obtaining a focal length of the second imaging device from the search result; And a step of detecting a dimensional position.

[0118]

According to the calibration method of the present invention, it is possible to realize the calibration of an imaging device having a variable focal length.

Further, the corresponding point searching method and apparatus according to the present invention can detect the coincidence of the projected images of the object on the two-dimensional coordinate system between the imaging devices having different focal lengths.

Further, the method and apparatus for detecting a focal length according to the present invention capture an image of the same object by using a first imaging device having a fixed focal length lens and a second imaging device having a variable focal length lens. The focal length of the second imaging device can be obtained based on a first projection image on a two-dimensional coordinate system obtained by imaging the object with the first imaging device.

Further, according to the method and the apparatus for detecting three-dimensional position information according to the present invention, the first image pickup device having a fixed focal length lens and the second image pickup device having a variable focal length lens are provided. The same object can be photographed, and the position of this object in the three-dimensional coordinate system can be detected.

Further, according to the three-dimensional position information detecting method and apparatus, the present focal length can be detected without attaching a special device to the zoom lens, thereby obtaining three-dimensional information of the target object, Image data captured at the intended focal length can be obtained at the same time.

[Brief description of the drawings]

FIG. 1 is a perspective view of a calibration device as an embodiment of a calibration method according to the present invention.

FIG. 2 is a view showing a state where a flat plate is slid by the calibration device shown in FIG. 1;

FIG. 3 is a perspective view of a calibration device according to another embodiment.

FIG. 4 is a view showing a state where a flat plate is slid by the calibration device shown in FIG. 3;

FIG. 5 is a block diagram of a three-dimensional position information detecting device which is an embodiment of the three-dimensional position information detecting method and device according to the present invention.

FIG. 6 is a flowchart for explaining the operation of the three-dimensional position information detecting device shown in FIG. 5;

FIG. 7 is a diagram for explaining selection of a corresponding point for obtaining a focal length in the three-dimensional position information detection device shown in FIG. 5;

FIG. 8 is a diagram for explaining a method of obtaining a focal length by the three-dimensional position information detecting device shown in FIG. 5;

FIG. 9 is a diagram showing epipolar lines required for obtaining a focal length by the three-dimensional position information detecting device shown in FIG. 5;

FIG. 10 is a diagram for explaining block matching performed by the three-dimensional position information detecting device shown in FIG. 5;

FIG. 11 is a block diagram of a three-dimensional position information detection device according to another embodiment of the three-dimensional position information detection method and device according to the present invention.

FIG. 12 is a diagram for explaining a principle for obtaining a focal length by the three-dimensional position information detecting device shown in FIG. 11;

FIG. 13 is a diagram for explaining a method of obtaining a focal length by the three-dimensional position information detecting device shown in FIG. 11;

FIG. 14 is an external perspective view of a stereo camera system serving as a conventional three-dimensional position information detecting device.

FIG. 15 is a diagram showing a state where two cameras are slid in the stereo camera system shown in FIG. 14;

FIG. 16 is a diagram showing a state where a flat plate is slid by the stereo camera system shown in FIG. 14;

FIG. 17 is a diagram illustrating a state where a point on a predetermined straight line is projected at a predetermined position on a screen.

FIG. 18 is a diagram for explaining a method of obtaining a three-dimensional position of an object using the stereo camera system shown in FIG. 14;

[Explanation of symbols]

4 Camera with fixed focal length lens, 3 carriages,
Reference Signs List 5 Camera with variable focal length lens, 6 Flat plate with square lattice pattern, 8 Stereo camera, 10 Image data storage unit, 11 Operation unit, 12 Calibration data storage unit, 13 Control unit

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Naosuke Asari 3-14-13 Higashi-Gotanda, Shinagawa-ku, Tokyo Inside Sony Kihara Laboratory Co., Ltd. (72) Inventor Junji Horikawa 3--14 Higashi-Gotanda, Shinagawa-ku, Tokyo No. 13 Inside Sony Kihara Research Laboratories (72) Inventor Hidetoshi Nagano 3-chome, Higashi Gotanda 3-chome, Shinagawa-ku, Tokyo (72) Inventor Takahiro Ishii 3-chome Higashi Gotanda, Shinagawa-ku, Tokyo No. 14-13, Sony Kihara Research Laboratories (72) Inventor Takashi Nozaki, 3-14-13 Higashi Gotanda, Shinagawa-ku, Tokyo Within Sony Kihara Research Laboratories

Claims (23)

[Claims] 1. An image pickup apparatus for imaging an object with a variable focal length, each position of a projected image of a two-dimensional coordinate system obtained by imaging the object at a plurality of specific focal lengths, and a three-dimensional image of the object. A calibration method comprising: obtaining a positional information correspondence with each position in a coordinate system. 2. The calibration method according to claim 1, wherein the positional information correspondence at a plurality of other focal lengths other than the plurality of specific focal lengths is obtained by interpolation. 3. An image of the same object is taken by a reference imaging device and another imaging device, and the projected image on the two-dimensional coordinate system of the other imaging device coincides with the projected image of the reference imaging device on the two-dimensional coordinate system. A corresponding point search method for searching for a corresponding point, comprising: determining a size of a cutout block centered on a candidate of a corresponding point on a two-dimensional coordinate system of the another imaging apparatus; And then adjust the size to the same size as the cutout block centered on the projected image of the reference imaging device on the two-dimensional coordinate system, and adjust the correspondence between the two blocks. A corresponding point search method characterized by searching for the corresponding point by calculating the degree of the corresponding point. 4. The corresponding point searching method according to claim 3, wherein the degree of correspondence between the two blocks is obtained by calculating a correlation between the two blocks. 5. The corresponding point searching method according to claim 3, wherein a plurality of said other imaging devices are provided. 6. An image of the same object is taken by a reference imaging device and another imaging device, and the projected image on the two-dimensional coordinate system of the other imaging device coincides with the projected image of the reference imaging device on the two-dimensional coordinate system. A corresponding point searching apparatus for searching for a corresponding point, wherein a size of a cutout block centered on a corresponding point candidate on a two-dimensional coordinate system of the other imaging apparatus is determined by using the reference imaging apparatus and the other And then adjust the size to the same size as the cutout block centered on the projected image of the reference imaging device on the two-dimensional coordinate system, and adjust the correspondence between the two blocks. A corresponding point searching device, comprising a corresponding degree calculating means for calculating the degree of the corresponding point. 7. The corresponding point searching device according to claim 6, wherein the correspondence degree calculating means calculates a degree of correspondence between the two blocks by calculating a correlation between the two blocks. 8. The corresponding point searching device according to claim 6, wherein a plurality of said other imaging devices are provided. 9. An image of the same object is taken by a reference imaging device and another imaging device, and the projected image on the two-dimensional coordinate system of the other imaging device coincides with the projected image of the reference imaging device on the two-dimensional coordinate system. A recording medium which records a corresponding point search processing procedure for searching for a corresponding point, wherein a size of a cutout block centered on a candidate of a corresponding point on a two-dimensional coordinate system of the other imaging apparatus is determined by the reference A step of changing the ratio of the focal length between the image pickup device to be used and the other image pickup device, and cutting out the variable cutout block around the projected image on the two-dimensional coordinate system of the reference image pickup device A recording medium recording a processing procedure comprising: a step of adjusting the size to the same size as a block; and a step of calculating a degree of correspondence between the adjustment block and the cutout block centered on the projection image. body. 10. An image of the same object is photographed by a first imaging device having a fixed focal length lens and a second imaging device having a variable focal length lens, and the object is imaged by the first imaging device. A focal length detection method for obtaining a focal length of the second imaging device based on the first projection image on the two-dimensional coordinate system obtained by the second imaging device. The second projection image on the obtained two-dimensional coordinate system is searched by calculating a correlation with the first projection image, and the focal length of the second imaging device is obtained from the search result. Characteristic focal length detection method. 11. The method according to claim 1, wherein the first projection image is selected from outside a predetermined region connecting a center of the two-dimensional coordinate system of the first imaging device and a center of the two-dimensional coordinate system of the second imaging device. The focal length detecting method according to claim 10, wherein: 12. A candidate for a second projected image on a two-dimensional coordinate system of the second imaging device is centered on a ratio of a focal length of the first imaging device to a focal length of the second imaging device. After changing the size of the cut-out block, the size of the cut-out block is adjusted to the same size as the cut-out block centered on the first projection image.
The second projection image is searched, and the second search image is obtained from the search result.
11. The focal length detecting method according to claim 10, wherein the focal length of the imaging device is obtained. 13. An image of the same object is photographed by a first imaging device having a fixed focal length lens and a second imaging device having a variable focal length lens, and the object is imaged by said first imaging device. A focal length detection device for obtaining a focal length of the second imaging device on the basis of the first projection image on the two-dimensional coordinate system obtained by the second imaging device. The second projection image on the obtained two-dimensional coordinate system is searched by calculating a correlation with the first projection image, and the focal point for calculating the focal length of the second imaging device from the search result. A focal length detecting device comprising a distance calculating means. 14. The method according to claim 1, wherein the first projection image is selected from outside a predetermined area connecting a center of the two-dimensional coordinate system of the first imaging device and a center of the two-dimensional coordinate system of the second imaging device. 14. The focal length detecting device according to claim 13, wherein: 15. The method according to claim 15, wherein the focal length calculating unit calculates a second focal length of the second imaging device on a two-dimensional coordinate system based on a ratio of a focal length of the first imaging device to a focal length of the second imaging device.
The size of the cutout block centered on the candidate is changed, and then adjusted to the same size as the cutout block centered on the first projection image, and the second block is set according to the degree of correspondence between the two blocks. 14. The focal length detecting device according to claim 13, wherein a projected image is searched, and a focal length of the second imaging device is calculated from the search result. 16. An image of the same object is photographed by a first imaging device having a fixed focal length lens and a second imaging device having a variable focal length lens, and the object is photographed by the first imaging device. A recording medium that records a focal length detection processing procedure for obtaining a focal length of the second imaging device based on the first projection image on the two-dimensional coordinate system obtained in the second step. Searching for a second projected image on a two-dimensional coordinate system obtained by imaging the object with the imaging device by calculating a correlation with the first projected image; Recording a focal length of the image pickup apparatus. 17. An image of the same object is taken by a first image pickup device having a fixed focal length lens and a second image pickup device having a variable focal length lens, and the object is photographed in a three-dimensional coordinate system. A three-dimensional coordinate position detecting method for detecting a position, wherein a second projection image on a two-dimensional coordinate system obtained by imaging an object by the second imaging device is used for the first projection image. A three-dimensional position information detecting method, wherein a three-dimensional position of the object is detected from the focal length of the second imaging device. 18. The three-dimensional position information according to claim 17, wherein a position of the object in a three-dimensional coordinate system is detected by using a plurality of first imaging devices having the fixed focal length lens. Detection method. 19. A candidate for a second projected image on a two-dimensional coordinate system of the second imaging device is centered on a ratio of the focal length of the first imaging device to the focal length of the second imaging device. After changing the size of the cut-out block, the size of the cut-out block is adjusted to the same size as the cut-out block centered on the first projection image.
The second projection image is searched, and the second search image is obtained from the search result.
18. The three-dimensional position of the object is detected from the focal length of the imaging device according to the above.
The three-dimensional position information detecting method according to the above. 20. The same object is photographed by a first image pickup device having a fixed focal length lens and a second image pickup device having a variable focal length lens, and the object is photographed in a three-dimensional coordinate system. A three-dimensional coordinate position detection device for detecting a position, wherein a second projection image on a two-dimensional coordinate system obtained by imaging an object by the second imaging device is used for the first projection image. And a search for the focal length of the second imaging device from the search result, and a three-dimensional position information calculating means for detecting a three-dimensional position of the object from the focal length. Three-dimensional position information detecting device. 21. The three-dimensional position information according to claim 20, wherein the position of the object in a three-dimensional coordinate system is detected by using a plurality of first imaging devices having the fixed focal length lens. Detection device. 22. The three-dimensional position information calculating means, on the basis of the ratio of the focal length of the first imaging device to the focal length of the second imaging device, on the two-dimensional coordinate system of the second imaging device. After changing the size of the cut-out block centering on the candidate of the second projection image, the size is adjusted to the same size as the cut-out block centering on the first projection image, and depending on the degree of correspondence between the two blocks. 21. The method according to claim 20, further comprising: searching for the second projected image; obtaining a focal length of the second imaging device from the search result; and calculating a three-dimensional position of the object from the focal length. A three-dimensional position information detection device. 23. The same object is photographed by a first image pickup device having a fixed focal length lens and a second image pickup device having a variable focal length lens, and the object is photographed in a three-dimensional coordinate system. A recording medium recording a three-dimensional coordinate position detection procedure for detecting a position, wherein a second projected image on a two-dimensional coordinate system obtained by imaging an object by the second imaging device is A step of calculating and searching for a correlation with the first projection image; a step of obtaining a focal length of the second imaging device from the search result; and detecting a three-dimensional position of the object from the focal length. A recording medium which records a processing procedure having steps.