JP2002170102A - Method and apparatus for automatically acquiring and restoring a photographing object - Google Patents

Abstract

(57) [Summary] [Problem] Accurate elevation data around a block and elevation data of a road with respect to three-dimensional coordinate values (elevation data) of the ground obtained or restored from space survey, aerial survey, etc. Etc. are obtained or restored with high precision. SOLUTION: While capturing video in a time series from an image input unit (camera), position information of the image input unit at the time of capturing a corresponding image is acquired, and a physical scale of an object in the image is determined. Then, the moving amount of the image input unit is sequentially calculated in accordance with the time series, and the three-dimensional model is sequentially obtained while acquiring the three-dimensional coordinate value of the object while corresponding to the calculated moving amount of the image input unit. This is a method for automatically acquiring and restoring the photographing object that outputs the image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for automatically acquiring and restoring a surface to be photographed such as a ground, an elevation surface, a road surface and the like. The present invention relates to a technique which is effective when applied to a surveying field for measuring three-dimensional coordinate values (elevation data) of the ground.

[0002]

2. Description of the Related Art Conventionally, altitude data is obtained by space survey using a satellite or the like at intervals of meshes of N meters (for example, 50 meters) on the ground (for example, altitude data of 10 meters is obtained from sample points at intervals of 50 meters). Calculated by calculation), and are measured as three-dimensional coordinate values of each sample point.

[0003]

However, as shown in FIG. 16, the sampling points are coarse and it is impossible to accurately measure the elevation at the block level. Also, with the development of remote sensing technology in recent years, it has become possible to mount a laser measuring instrument on an airplane and measure altitude data more accurately than space surveying. However, as shown in FIG. 17, since obstacles such as trees and cars are also measured by laser,
It has been difficult to accurately measure elevation data around roads and city blocks. An object of the present invention is to provide a technique capable of acquiring and restoring accurate elevation data around a block, elevation data of a road, and the like with high accuracy. The above and other objects and novel features of the present invention will become apparent from the description of the present specification and the accompanying drawings.

[0004]

The outline of the invention disclosed in the present application is briefly described as follows. (1) While capturing images in time series from an image input unit (camera), obtain the position information of the image input unit when the corresponding image was captured, and determine the physical scale of the object in the image. In accordance with the time series, the moving amount of the image input unit is sequentially calculated, and the three-dimensional model is sequentially obtained while acquiring the three-dimensional coordinate values of the object while corresponding to the calculated moving amount of the image input unit. This is a method for automatically acquiring and restoring a shooting target to be output.

(2) An image input unit (camera) that captures an image of an object and captures images in chronological order, a sensing unit that obtains position information of the image input unit when a corresponding image is captured, and A registration unit that determines the physical scale of the target object, a movement distance calculation unit that sequentially calculates the movement amount of the image input unit according to the time series, and an image input unit that is calculated by the movement distance calculation unit And a model output unit that generates and outputs a three-dimensional model based on the three-dimensional coordinate values of the geometric information acquisition unit while corresponding to the movement amount of the object. And a device for automatically acquiring and restoring a photographing object.

(3) A ground that automatically represents the ground from time-series image data obtained by photographing the ground as an object, or automatically acquires and restores three-dimensional geometric information or a surface shape to be modeled; In the method for automatically acquiring and restoring an elevation surface, model data on the ground registered in a database is read out, and camera parameters of image input means are automatically obtained from the model information and the shape of the model reflected in the image of the object. Adjust, associating the model data with the coordinates detected from the target object image, setting a slit surface in the associated image, and setting a physical width of the slit surface in the image along the horizontal or horizontal direction. Distance obtained when the image input means (camera) moves during the next time n + Δn with respect to the image taken at the predetermined time n, Temporal variation to high, the optical system of the visual line information of the image input means, as well, from the position information of the image input means in the time n, the time m (m from the time n
Until ≧ n + Δn), the time at which the image input means passes through the ground portion shown in the image at time n is calculated, and the time t (n <t ≦ m) when the image input means passes is calculated. Time information of the image input means, time n and time n + Δn
From the distance moved up to and the slit width set as described above, while acquiring three-dimensional coordinate values at each vertex of the slit surface of the ground part set at time n, sequentially acquiring and restoring three-dimensional ground, Or to reproduce.

(4) Ground which automatically represents the ground from time-series image data obtained by photographing the ground as an object, or automatically acquires and restores three-dimensional geometric information or surface shape to be modeled. In the altitude plane automatic acquisition / restoration device, the model data on the ground registered in the database is read out, and the camera parameters of the image input means are obtained from the model information and the shape of the model reflected in the image of the object. Camera parameter automatic adjustment means of image input means for automatic adjustment, registration means for detecting coordinates to be associated in the model data and the object image, and uniquely associating the coordinates, and coordinates of the model data and the object image are Slit surface setting means for setting a slit surface in the associated image; and a horizontal surface of the slit surface in the image. Slit width obtaining means for obtaining the physical length of the width along the horizontal or horizontal direction, and when the image input means moves during the next time n + Δn with respect to the image taken at the predetermined time n From the distance of the distance, the amount of temporal change in altitude therebetween, the line-of-sight information of the optical system of the image input means, and the position information of the image input means at the time n, the time n
From time to time m (m ≧ n + Δn), ground portion passing time calculating means for calculating the time at which the image input means passes through the ground portion reflected in the image at time n;
The position information of the image input means at the time t (n <t ≦ m) at the time of passing, the distance moved up to the time n and the time n + Δn, and the slit width set above are set at the time n. Optical information extracting means for extracting optical information corresponding to the slit surface of the ground portion from the video while acquiring three-dimensional coordinate values at each vertex of the slit surface of the ground portion, and a three-dimensional shape of the slit surface. Means for correcting geometrical shape distortion from the positional relationship of the image input means, conversion optical information obtaining means for obtaining optical information deformed or converted based on the geometrical shape distortion, the three-dimensional geometric information, Means for sequentially acquiring, restoring, or reproducing a three-dimensional model of the ground by combining information.

(5) A road surface for automatically expressing and reconstructing three-dimensional geometric information or a surface shape for automatically representing the ground or modeling from time-series image data taken of a road. In the automatic acquisition / restoration method, the model data on the ground registered in the database is read out, and the camera parameters of the image input unit are automatically adjusted based on the model information and the shape of the model reflected in the image of the object. The model data is uniquely associated with image coordinates, the model data is detected in an image in which image coordinates are associated, a slit surface is set in the uniquely associated image, and the slit surface in the image is set. To obtain the physical length of the width along the horizontal or horizontal direction,
For the image taken at the predetermined time n, the next time n
The distance from the predetermined time n to the time from the distance when the image input means moved during + Δn, the amount of temporal change in elevation during that time, the line-of-sight information of the optical system of the image input means, and the position information of the image input means at time n Until m (m ≧ n + Δn), the time at which the image input means passes through the ground portion shown in the image at time n is calculated, and the time t (n <t ≦ From the position information of the image input means at the time of m), the distance moved up to the time n and the time n + Δn, and the set slit width, each vertex of the slit surface of the ground portion set at the time n is three-dimensionally. It acquires, restores, or reproduces a three-dimensional ground sequentially while acquiring coordinate values.

(6) Automatic acquisition of a road surface that automatically acquires or restores three-dimensional geometric information or a surface shape to be automatically expressed from time-series image data taken of a road. In the restoration device, model data on the ground registered in the database is read from the database, and camera parameters for automatically adjusting image input means (camera) parameters based on the model information and the shape of the model reflected in the image of the object. Automatic adjusting means, detecting coordinates to associate the model data with the image coordinates, and registering means for uniquely associating the coordinates; and setting a slit plane in the image in which the model data and the image coordinates are uniquely associated. Slit surface setting means, and an object having a width along a horizontal direction or a horizontal direction of the slit surface in the image. A slit width acquiring means for acquiring lengths, the image captured at the time of the predetermined time n,
From the distance when the image input means moves during the next time n + Δn, the amount of temporal change in altitude during that time, the line-of-sight information of the optical system of the image input means, and the position information of the image input means at time n, the predetermined time n to time m (m ≧ n + Δ
Until the time n, the image input means calculates the moment when the image input means passes through the ground portion shown in the image at the time n, and the time t (n <T ≦ m), the position information of the image input means,
While acquiring three-dimensional coordinate values at each vertex of the slit surface of the ground portion set at the time n from the distance moved to the time n and the time n + Δn and the set slit width,
An optical information extracting means for extracting optical information corresponding to the slit surface of the ground portion from an image,
Means for correcting geometrical shape distortion based on the dimensional shape and the positional relationship between the image input means, transformed optical information acquiring means for acquiring deformed or transformed optical information based on the geometrical shape distortion, and the three-dimensional geometric information And means for sequentially acquiring, restoring, or reproducing a three-dimensional model of the ground by combining optical information.

The point of the present invention is that a time-series image obtained by photographing an elevation surface while moving a camera of an image input device and sensor information synchronized with the image are used for the elevation obtained by the image input device. Calculate the three-dimensional coordinate value of the surface,
The purpose is to acquire and restore an elevation surface by sequentially synthesizing it with the corresponding texture data. This makes it possible to easily and accurately acquire and restore the shape of the elevation surface, as compared with elevation data from space surveys and aerial surveys.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the drawings together with embodiments (examples) according to the present invention. In all the drawings for describing the embodiments, components having the same function are denoted by the same reference numerals, and their repeated description will be omitted.

(Embodiment 1) FIG. 1 is a block diagram showing a schematic configuration of a road surface acquisition / restoration apparatus according to Embodiment 1 of the present invention, and FIG. It is a schematic diagram showing a schematic configuration with a moving means.

1 and 2, 1 is an image input device (image input unit), 2 is a sensing unit, 3 is a registration unit, 4 is a moving distance calculation unit, 5 is a geometric information acquisition unit,
6 is a model construction unit, 7 is a model output unit, 9 is an image input device moving means, 10 is a road surface, Xi, Yi, Zi and X
j, Yj, Zj are position coordinates on the road surface, and P1 is time t
i, the camera viewpoint (position) at time i, P2 is the camera viewpoint (position) at time tj, Q is the position coordinate of the road surface corresponding to the camera viewpoint (position) at time tj, and RO is the camera of the image input unit 1. The optical axis of incident light passing through the viewpoint, ti and tj are time, Hc is the height from the road surface to the viewpoint of the camera, Vi
Is the moving speed of the image input device moving means, and Li is the moving amount.

The road surface acquisition / restoration device according to the first embodiment includes:
As shown in FIG. 1, an image input device (image input unit) 1, a sensing unit 2, a registration unit 3, a movement distance calculation unit 4, a movement distance calculation unit 4, a geometric information acquisition unit 5, a model construction unit 6, and It comprises a model output unit 7.

In the image input device (image input unit) 1,
Images are fetched in a time series, or stored according to uses such as post-processing. The sensing unit 2 acquires the position information of the camera of the image input device (image input unit) 1 at the time of capturing the image. The registration unit 3 acquires the basic model on the registered road surface by projecting it on an image, and determines the physical scale of the target object in the image.

The moving distance calculating unit 4 sequentially calculates the moving amount of the camera according to the time series, and the geometric information obtaining unit 5 obtains the three-dimensional coordinate value of the road surface while corresponding to the calculated amount. The model construction unit 6 synthesizes, processes, and shapes the model according to the three-dimensional viewer, and outputs the model in the model output unit 7.

The road surface acquisition / restoration device according to the first embodiment includes:
As shown in FIG. 2, an image input device (camera) 1 is mounted on a vehicle and is mounted on a vehicle, and at the time ti, a target portion on a road surface which is projected from a camera viewpoint (Xi, Yi, Zi). Three-dimensional structure or shape (Xj, Yj,
An example of acquiring and restoring (Zj-Hc) will be described. Further, it is assumed that the viewpoint of the camera is equivalent to the sensing position.

As described above, according to the present invention, the camera viewpoint acquired at each time ti and the three-dimensional coordinate value of the road surface portion corresponding to the image center of the image at that time are sequentially obtained, and the shape is obtained as the road surface. It is to restore.

FIG. 3 is a block diagram showing a functional configuration of the registration unit 3. As shown in FIG. 3, reference numeral 30 denotes a basic model data storage unit, 31 denotes a model selection unit for selecting predetermined basic model data from data registered in the basic model data storage unit 30, and 32 denotes a time-series image data storage unit (storage device). ), 33 is a corresponding image loading unit, 34 is a corresponding shape obtaining unit, 35 is a projection unit for basic model data, 3
Reference numeral 6 denotes a matching shape detection unit, reference numeral 37 denotes an allowable error value detection unit (registration detection unit), reference numeral 38 denotes a camera posture parameter adjustment unit, reference numeral 39 denotes a camera position parameter adjustment unit, and reference numeral 40 denotes a registration success signal output unit. is there.

In the registration unit 3, as shown in FIG. 3, the basic model data is obtained by selecting a basic model on the road surface registered in the model data storage unit 30 and projecting it on an image. get. Further, the correspondence image is read from the time-series image data registered in the time-series image data storage unit 32 by the corresponding image loading unit 33,
The corresponding shape of the correspondence image is obtained by the corresponding shape obtaining unit 34. The selected basic model data is projected onto the obtained correspondence image by the basic model data projection unit 35.

The consistency between the corresponding shape of the acquired correspondence image and the basic model data is detected by a consistency detecting unit 36, and the detected error value is used as an allowable error value detecting unit (registration success detection). Unit) 37 detects whether or not the error is equal to or less than the allowable error value ε. If the error is not equal to or less than the allowable error value ε, the camera attitude parameter adjusting unit 38 and the camera position parameter adjusting unit 39 adjust the error to the allowable error value ε or less. And when the error becomes equal to or smaller than the allowable error value ε, a registration success signal is output to the registration output unit 4.
Output from 0.

FIG. 4 is a flowchart showing the procedure of the processing operation of the road surface acquisition / restoration device according to the first embodiment. The processing operation of the road surface acquisition / restoration device according to the first embodiment will be described with reference to FIG. As shown in FIG. 4, the image input data 101 is a unique image managed by a time code or the like, and corresponding thereto, camera sensing information at that time, that is, camera viewpoint information, Information (Xi, Yi, Zi) is acquired in time series (FIG. 2). These data are stored in the video data storage 10
2. In the sensing information storage unit 111, data is stored as appropriate.

The following processing does not change the basic processing concept in online processing or offline processing.
For convenience, a processing method and apparatus of an offline type, that is, an image data and sensing data storage type will be described.

From the stored images, first, a physical value is given to the width of the road for physical scale adjustment. This means that the basic model data 103 representing physical dimensions such as a sign on a road, a pedestrian crossing, or a sign near an intersection registered in a database of basic model data in advance is read, and default camera parameters (internal , External) and superimpose it on the image (105).

Next, in the superimposed image, a corresponding portion (edge, line segment, etc.) in the projected shape and the image is detected (104), and when the corresponding shape does not match (106), , Successively adjusting the external parameters of the camera, that is, the posture and position parameters (107, 10)
8). This adjustment amount is adjusted until the projected shape matches the corresponding shape in the image (until a certain degree of error is reached) (106) (see FIG. 4). Thereby, in the horizontal direction of the image, a unique correspondence between the coordinate value (xd, yd) of the image and the physical dimension (for example, the distance m from the vicinity of the center of the image to xd, yd) is obtained. In, when a certain pixel length is designated in the horizontal direction from the center of the image, the physical length of the line is obtained (109).

Next, from the sensing information storage unit 110,
Camera position information (Xi, Yi, Z) at time ti
i) is obtained (112). Further, the position information of the camera at the time ti + Δt is obtained (113). At this time, the movement distance during the time Δt is calculated (115). At time ti, the road of the object that the camera was shooting,
To be more precise, the moving means (vehicle) 9 of the camera senses a physical line acquired in the horizontal direction as the center of the image when a certain time (time tj) has elapsed from this time ti.

The geometric information acquiring unit 5 acquires three-dimensional coordinate values of the road surface to be acquired from the relationship between the movement distance Li from time ti to time tj and the line of sight of the camera (see FIG. 2) (115). In addition, since the road surface 10 is not necessarily horizontal, and the angle between the road surface and the horizontal surface changes with time, in the present invention, the gradient of the surface where the ground 10 contacts the vehicle is determined from the sensed data 110. Is calculated (114). This angle Δθ
Is used to obtain the three-dimensional coordinate value of the target road surface, and calculates the moving distance s when the following equation 1 is minimized (116).

From this movement distance s, a time tk from the time ti until the predetermined distance is reached is calculated, and the three-dimensional coordinate values (Xk, Xk,
Yk, Zk) are obtained.

[0029]

| F (s) + s / tan (θ + Δθ) −h | <ε In Equation 1, s is the moving distance, f (s) is the sensed altitude function, and θ is the horizontal position of the camera at the horizontal position. The angle to be performed, Δθ is the variation angle from time ti to ti + Δt,
h is the height from the ground to the sensor, and ε is the tolerance.
The acquired road width is given to the three-dimensional coordinate value corresponding to the center of the image, and the road surface is restored by interpolating between adjacent acquisition points of the three-dimensional coordinate value Zk at the center of the image.

The above sequential processing is performed according to a time series, and three-dimensional geometric information is sequentially obtained. Finally, model the road according to the specified 3D model format,
The road model data obtained by three-dimensional modeling is output.

FIG. 5 is a flowchart showing a processing procedure of the road surface restoring method for each slit plane. As shown in FIG. 6 (illustration of the XY coordinate directions is omitted for simplicity of the drawing), the road surface restoration method in units of slit planes uses a three-dimensional coordinate value ( Z
(i-1), Z (i), Z (i + 1)) are obtained (201). The road length is acquired from the obtained three-dimensional coordinate values (Z (i-1), Z (i), Z (i + 1)) (202) and input to the coordinate conversion unit (203) for the slit point.

The three-dimensional coordinate values (Z (i-1), Z (i), Z (i
+1)), the method of acquiring the road length (vertical direction of the slit plane or the traveling direction) is, as shown in FIG. 7, a method of acquiring the three-dimensional coordinate point Z (i) adjacent to the acquired three-dimensional coordinate point Z (i) on the road plane. The three-dimensional coordinate point Z (i-1) obtained and the midpoint (zd (i) and zu (i)) of the obtained three-dimensional coordinate point Z (i + 1) are obtained, and the Z coordinate axis direction (road length) Calculate the distance Li between the midpoints with respect to the direction). In FIG. 7, this midpoint is represented by zd (i) and zu on the image.
Corresponds to (i).

On the other hand, the physical width (X coordinate direction) of the road surface is obtained (204), and the azimuth on the XY plane is obtained (205), and the coordinate conversion unit for the slit point (203).
To enter. In the coordinate conversion unit 203 for the slit point,
The road length and the physical width of the road surface obtained from the three-dimensional coordinate values (Z (i-1), Z (i), Z (i-1)) are represented by three-dimensional coordinate values (Z
(i-1), Z (i), Z (i + 1)), and interpolates between adjacent acquisition points of the three-dimensional coordinate value Z (i) at the center of the image,
Restore the road surface in slit units.

FIG. 8A is a flowchart showing the processing procedure of the road surface model output processing according to the first embodiment, and FIG. 8B is a diagram for explaining the processing procedure. In the output processing of the road surface model according to the first embodiment, as shown in FIG. 8, the azimuth angle φ at that time is calculated from the coordinate values (X ′, Y ′) at the time of obtaining the road surface, and the azimuth φ is obtained at time t. By rotating the azimuth angle φ around the X and Y coordinate values of the center of the slit plane, (X, Y) coordinate values of the end points of the slit plane are acquired (401). FIG. 9 shows the XY locus of the X and Y coordinate values of the center of the slit plane acquired at time t.
Shown in Next, the Z coordinate of the slit surface from the Z coordinate value near time t is allocated (402). FIG. 10 shows the obtained Z locus of the Z coordinate value of the center of the slit plane.

Next, based on the X, Y, and Z coordinate values, 3
A shape for a dimensional model is described (403). The shape description method for a three-dimensional model is as shown in FIG.
An end point A1 of the slit surface is obtained from the Z coordinate value at the time t and the Z coordinate value at the time t-1. Similarly, the end point A2 of the slit surface is obtained from the Z coordinate value at the time t-1 and the Z coordinate value at the time t. Further, an end point A3 of the slit surface is obtained from the Z coordinate value at time t and the Z coordinate value at time t + 1. Similarly, the end point A4 of the slit surface is obtained from the Z coordinate value at the time t + 1 and the Z coordinate value at the time t. The determined end points A1, A2, A
3 and A4 are connected by a straight line to describe the shape of the slit plane for the three-dimensional model, and an output from the 3D viewer is obtained (40
4).

(Embodiment 2) FIG. 12 shows Embodiment 2 of the present invention.
FIG. 1 is a block diagram showing a schematic configuration of a road surface acquisition / restoration device, and 8 is an optical information acquisition unit. As shown in FIG. 12, the road surface acquisition / restoration device according to the second embodiment is different from the first embodiment in that an optical information acquisition unit 8 is added to the configuration of the first embodiment.
Here, only the difference will be described.

FIG. 13 is a flowchart showing the procedure of the processing operation of the road surface acquisition / restoration device according to the second embodiment.
Position coordinates Z of slit surface 10A forming each road surface 10
(i) are sequentially obtained (see FIG. 6), and each slit plane 10A is projected on the image separately from the restored processing. At this time, the end of the slit (xLu, zLu)-(xLd, zL
d) and (xRu, zRu)-(xRd, zR
d) is calculated as the inclination of the straight line after the projection. For example, if the camera optical axis is perpendicular to the road surface (in FIG. 2, the road surface is viewed directly below), the straight line is a straight line (x = 0) along the traveling direction. In actual in-vehicle photographing, when the angle formed with the horizontal plane is θ, the reciprocal of the slope a of the straight line z = a * x + b shown in FIG. 14 is a value for generating an orthographic image.

In the process of acquiring the orthogonally projected texture according to the second embodiment, the shape of the slit surface on the road surface is acquired (60).
1) The obtained slit plane shape is projected on an image plane (602). Next, as shown in FIG. 14, coordinate values P (xL, z) at the end points P, Q of the slit plane shape on the image plane.
L) and Q (xK, zK) are acquired (603), and a straight line in the length direction of the road surface is detected by connecting the end points P and Q (6).
04). The slit plane is divided into minute line segments using the detected straight line and a straight line z = a * x + b in the length direction of the road surface in the slit plane (605). The minute line segments on the divided slit surface are sequentially converted into orthographic coordinates (60
6), and store it as texture data in the texture database (608) (607).

FIG. 15 is a flowchart showing a processing procedure for outputting a three-dimensional model combining geometric information and optical information according to the second embodiment. As shown in FIG. 15, the output of the three-dimensional model obtained by combining the geometric information and the optical information according to the second embodiment is, with the center coordinates of the slit plane as the center, the road width and the road length, and Is given a three-dimensional coordinate value (701). Optical information is also acquired together with the start point and end point coordinate values so as to be mapped on this slit plane.

When a model is output from a certain time k, the coordinate value of the center of the corresponding slit plane, the road edge, and the road length are loaded from the memory, and the geometrical information of the road plane at the time k-1 and the time k + 1 is obtained. In order to maintain the consistency, the lower side (time k-1 direction) and the upper side (time k-1) of the slit surface at time k are maintained.
The (+1 direction) Z value is averaged with an adjacent Z value to obtain a common Z value (702). Other coordinate values,
That is, the X and Y coordinates are obtained in consideration of the three-dimensional coordinate value of the acquired image center of the slit plane, the road width, and the road length (703).

The geometric information determined at this time is stored in an image on which optical information is accumulated and in a texture database 704.
Unique with texture data read from
In correspondence with the dimensional coordinate values, texture mapping is performed on the texture data read from the texture database 704 (705), and a three-dimensional model is sequentially generated and output (706).

As described above, the invention made by the present inventor is:
Although specifically described based on the embodiment, the present invention
It is needless to say that the present invention is not limited to the above-described embodiment, but can be variously modified without departing from the scope of the invention.

[0043]

The effects obtained by the invention disclosed in the present application will be briefly described as follows.
According to the present invention, three-dimensional coordinate values (elevation data) of the ground obtained or restored from space survey, aerial survey, etc.
In contrast, accurate elevation data around the block, elevation data of roads, and the like can be obtained or restored with high accuracy.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a schematic configuration of a road surface acquisition / restoration device according to a first embodiment of the present invention.

FIG. 2 is a schematic diagram illustrating a schematic configuration of a road and a moving unit of the image input apparatus according to the first embodiment.

FIG. 3 is a block diagram illustrating a functional configuration of a registration unit 3 according to the first embodiment.

FIG. 4 is a flowchart illustrating a procedure of a processing operation of the road surface acquisition / restoration device according to the first embodiment;

FIG. 5 is a flowchart illustrating a processing procedure of a road surface restoration method for each slit surface according to the first embodiment;

FIG. 6 is a diagram for explaining a road surface restoration method for each slit surface according to the first embodiment.

FIG. 7 is a diagram illustrating a method of acquiring a road length from three-dimensional coordinate values according to the first embodiment.

FIG. 8 is a flowchart illustrating a processing procedure of a road surface model output process according to the first embodiment;

FIG. 9 is a diagram showing an XY locus of X and Y coordinate values of the center of the slit plane acquired at time t in the first embodiment.

FIG. 10 is a diagram for explaining a method of acquiring a road length from three-dimensional coordinate values according to the first embodiment.

FIG. 11 is a diagram for calculating the end points A1, A2, A3, and A4 of the slit surface from the Z coordinate value at time t and the Z coordinate value at time t−1 according to the first embodiment; FIG.

FIG. 12 is a block diagram illustrating a schematic configuration of a road surface acquisition / restoration device according to a second embodiment of the present invention.

FIG. 13 is a flowchart illustrating a procedure of a processing operation of the road surface acquisition / restoration device according to the second embodiment;

FIG. 14 is a diagram for explaining a method of forming a slit surface shape on an image surface according to the second embodiment.

FIG. 15 is a flowchart illustrating a processing procedure for outputting a three-dimensional model obtained by combining geometric information and optical information according to the second embodiment.

FIG. 16 is a diagram for explaining a problem of the related art.

FIG. 17 is a diagram for explaining a problem of the related art.

[Explanation of symbols]

REFERENCE SIGNS LIST 1 image input device (image input unit) 2 sensing unit 3 registration unit 4 moving distance calculation unit 5 geometric information acquisition unit 6 model construction unit 7 model output unit 8 optical information acquisition unit 9 image Input device moving means 10 Road surface Xi, Yi, Zi, Xj, Yj, Zj Position coordinates of camera viewpoint P1 Camera viewpoint (position) at time ti P2 Camera viewpoint (position) at time tj Q: Time Position coordinates of the road surface viewed from the camera viewpoint (position) at tj RO: Optical axis of incident light passing through the camera viewpoint ti, tj: Time Hc: Height from the road surface to the camera viewpoint Vi: Moving speed Li: Movement amount 30: Basic model data storage unit 31: Model selection unit 32: Time-series image data storage unit (storage device) 33: Corresponding image loading unit 34: Corresponding shape obtaining unit 35: Basic model data Consistency detecting section 37 ... tolerance value detecting section of the projection portion 36 ... corresponding shape (registration success detecting means) 38 ... camera posture parameter adjuster 39 ... camera position parameter adjuster 40 ... registration success signal output unit

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G09B 29/00 G01B 11/24 AK (72) Inventor Shigeru Nagai 2-3-3 Otemachi, Chiyoda-ku, Tokyo No. 1 Inside Nippon Telegraph and Telephone Corporation (72) Tomohiko Arikawa 2-3-1 Otemachi, Chiyoda-ku, Tokyo F-term within Nippon Telegraph and Telephone Corporation (reference) 2C032 HB03 HB11 HC01 HC23 2F065 AA04 AA53 BB05 CC40 DD03 EE11 FF01 FF04 FF64 FF67 JJ03 JJ26 MM07 QQ21 QQ25 QQ39 QQ42 RR02 RR06 SS13 UU05 5B050 AA01 BA09 BA17 DA07 EA07 EA13 EA28 FA02 5B057 AA16 BA02 CA12 CA16 CB13 CB08 DC17 DC17 DC17

Claims (6)

[Claims] 1. While acquiring video in a time series from an image input unit, obtain position information of an image input unit when a corresponding image is captured, determine a physical scale of an object in the image, According to the time series, the moving amount of the image input unit is sequentially calculated, and the three-dimensional model is sequentially output while acquiring the three-dimensional coordinate value of the object while corresponding to the calculated moving amount of the image input unit. A method for automatically acquiring and restoring an object to be photographed. 2. An image input unit (camera) that captures an image of an object and captures a video in a time series; a sensing unit that acquires position information of the image input unit when a corresponding image is captured;
A registration unit that determines the physical scale of the object in the image, a moving distance calculating unit that sequentially calculates the moving amount of the image input unit according to the time series, and a moving distance calculating unit that calculates the moving amount. A geometric information acquisition unit for acquiring three-dimensional coordinate values of the object to be photographed while corresponding to the movement amount of the image input unit; and a three-dimensional model is generated and output based on the three-dimensional coordinate values of the geometric information acquisition unit. An apparatus for automatically acquiring and restoring a photographing object, comprising: a model output unit. 3. A ground, an altitude for automatically expressing or reconstructing three-dimensional geometric information or a surface shape to be modeled by automatically expressing the ground from time-series image data obtained by photographing the ground as an object. In the automatic surface acquisition / restoration method, the model data on the ground registered in the database is read out, and the camera parameters of the image input means are automatically adjusted based on the model information and the shape of the model shown in the image of the object. The model data and the coordinates detected from the target object image are associated with each other, a slit surface is set in the associated image, and the physical width of the slit surface in the image in the horizontal or horizontal direction is set. The length is acquired, and the next time n +
From the distance when the image input means (camera) has moved during Δn, the amount of temporal change in altitude between them, the line-of-sight information of the optical system of the image input means, and the position information of the image input means at time n, From time n to time m (m ≧ n +
During the time up to Δn), the time at which the image input means passes through the ground portion shown in the image at time n is calculated, and at the time t (n <t ≦ m) at that time From the position information of the image input means of the above, the time n and the distance moved to the time n + Δn, and the slit width set above, the three-dimensional coordinate value is set at each vertex of the slit surface of the ground portion set at the time n. A method for automatically acquiring and restoring a ground and an elevation surface, wherein a three-dimensional ground is acquired, restored, or reproduced sequentially while acquiring. 4. A ground for automatically representing the ground or automatically acquiring and restoring three-dimensional geometric information or a surface shape to be modeled from time-series image data obtained by photographing the ground as an object; In the altitude plane automatic acquisition / restoration device, the model data on the ground registered in the database is read out, and the camera parameters of the image input means are automatically obtained from the model information and the shape of the model shown in the image of the object. A camera parameter automatic adjusting means of the image input means to be adjusted, a registration means for detecting coordinates to associate the model data with the object image and uniquely associating the coordinates, and a coordinate between the model data and the object image Slit surface setting means for setting a slit surface in the attached image; and a horizontal direction or a slit surface in the image. Is a slit width obtaining means for obtaining the physical length of the width along the horizontal direction, and a case where the image input means moves during the next time n + Δn with respect to the image taken at the predetermined time n. From the distance, the amount of temporal change in altitude therebetween, the line-of-sight information of the optical system of the image input means, and the position information of the image input means at the time n, the image input from time n to time m (m ≧ n + Δn) A ground portion passing time calculating means for calculating a time instant when the means passes through the ground portion shown in the image at time n, and an image at time t (n <t ≦ m) when the passing time From the position information of the input means, the distance moved up to time n and time n + Δn, and the set slit width, three-dimensional coordinate values are obtained for each vertex of the slit surface of the ground portion set at time n. The ground pick-up An optical information extracting means for extracting optical information from the video corresponding to up surface,
Means for correcting geometrical shape distortion based on the three-dimensional shape of the slit surface and the positional relationship of the image input means, transformed optical information acquiring means for acquiring deformed or transformed optical information based on the geometric shape distortion, Combining the three-dimensional geometric information and the optical information to sequentially obtain and restore a three-dimensional model of the ground,
Or a means for reproducing the ground,
Elevation surface automatic acquisition and restoration device. 5. A road surface for automatically representing and reconstructing the ground or automatically acquiring and restoring three-dimensional geometric information or a surface shape to be modeled from time-series image data taken of a road as an object. In the automatic acquisition / restoration method, the model data on the ground registered in the database is read out, and the camera parameters of the image input unit are automatically adjusted based on the model information and the shape of the model reflected in the image of the object, The model data and image coordinates are uniquely associated, detected in an image in which the model data and image coordinates are associated, a slit plane is set in the uniquely associated image, and a slit plane in the image is set. The physical length of the width along the horizontal direction or the horizontal direction is obtained, and the next time n +
From the distance when the image input means moves during Δn, the amount of temporal change in altitude during that time, the line-of-sight information of the optical system of the image input means, and the position information of the image input means at time n, the time from a predetermined time n to time Until m (m ≧ n + Δn), the time at which the image input means passes through the ground portion shown in the image at time n is calculated, and the time t (n <t ≦ From the position information of the image input means at the time of m), the distance moved up to the time n and the time n + Δn, and the set slit width, each vertex of the slit surface of the ground portion set at the time n is three-dimensionally. An automatic road surface acquisition / restoration method characterized by successively acquiring / restoring or reproducing a three-dimensional ground while acquiring coordinate values. 6. Automatically representing or modeling from time-series image data taken of a road
Automatic acquisition of dimensional geometric information or surface shape
In a device for automatically acquiring and restoring a road surface to be restored, model data on the ground registered in a database is read out, and image input means (camera) is obtained from the model information and the shape of the model reflected in the image of the object. A camera parameter automatic adjusting means for automatically adjusting parameters; a registration means for detecting coordinates to be associated with the model data and the image coordinates, and uniquely associating the coordinates; and a uniquely associated model data and image coordinates. Slit surface setting means for setting a slit surface in an image, slit width obtaining means for obtaining a physical length of a width along a horizontal direction or a lateral direction of the slit surface in the image, and a predetermined time n The distance when the image input means moves during the next time n + Δn with respect to the image taken at
From the temporal change amount relating to the altitude during that time, the line-of-sight information of the optical system of the image input unit, and the position information of the image input unit at the time n, the image input unit is switched from the predetermined time n to the time m (m ≧ n + Δn). Means for calculating the moment of passing through the ground portion shown in the image at time n, and time t (n
<T ≦ m), the position information of the image input means and the time n
From the distance moved up to time n + Δn and the set slit width, three-dimensional coordinate values are obtained at the vertices of the slit surface of the ground portion set at time n while corresponding to the slit surface of the ground portion. Optical information extracting means for extracting optical information to be extracted from an image, a means for correcting a geometric shape distortion from a three-dimensional shape of the slit surface, and a positional relationship between the image input means, and a deformation or Converting optical information acquiring means for acquiring the converted optical information; combining the three-dimensional geometric information with the optical information;
An automatic road surface acquisition / restoration device comprising means for acquiring / restoring or reproducing a dimensional model.