JPH085351A - Slit beam calibration method of three-dimensional visual sensor - Google Patents

Abstract

PURPOSE:To simplify calibration by obtaining data indicating the values of plane parameters indicating each light projection direction of slit beams corresponding to a plurality of values given as a command value as data for interpolation calculation to obtain each light projection direction on measurement. CONSTITUTION:A flat surface 41 of an object 4 for calibration is mounted on a known first plane and a light source 12 is lit. A command value Vh (h=1-n: Integer) is fed to a mirror 13 for polarization, slit beams are projected, and a light band formed on the flat surface 41 is shot by a camera 11. A plurality of points on the light band are specified by a monitor CRT and a cursor, the visual line of the camera 11 is obtained, and the three-dimensional position of each point is obtained. An index value (h) is incremented by one and processing is repeated and the mirror 13 is returned to an original position when h=n is competed. The flat surface 41 is shifted to a know second plane and a similar processing is performed. Parameters indicating a plane sharing each point are obtained from the position data and are stored at a register region and then calibration is performed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional position or position and orientation of an object (hereinafter, unless otherwise distinguished,
Both are simply called “position”. ) Of the slit light projecting type three-dimensional visual sensor used for measuring (1).

[0002]

2. Description of the Related Art In an assembly work or a processing work in a factory production line, a vision system combining an automatic machine such as a robot and a visual sensor using a camera means is provided in order to automate and save the work. It's being used. For example, one assembly work to be supplied to a predetermined position on the manufacturing line is imaged by a CCD camera, and an accurate position of the work is detected by a visual sensor by analyzing the obtained image. The positional deviation between the works is corrected by a method of correcting the position of the robot holding the other work to be assembled based on the above.

As a typical three-dimensional visual sensor incorporated in such a vision system, there is a slit light projection type three-dimensional visual sensor. This is because a slit light projector equipped with a three-dimensional visual sensor emits slit light in a controlled direction to form a light band of higher brightness than the surroundings on an object, and this is used by a CCD camera or the like. The object is three-dimensionally measured based on the principle of triangulation by observing with camera means.

FIG. 1 is a conceptual diagram for explaining this principle. Reference numeral F1 represents the position of the slit light projector, and F2 represents the lens center of the camera. The slit light projector projects the slit light L1 toward the object W to be measured, and the optical band Q is formed on the object W. The light band Q is observed by the camera from a direction different from the slit light projecting direction and is captured as an image w on the image plane IP of the camera.

A point P on the light band Q (for example, an intersection of the ridge line of the light band Q and the object W) is a point p on the image plane IP.
If the image is formed at (u, v), the point P is the plane of the line of sight L2 passing through the points F2 and p in the three-dimensional space and the slit light L1 (hereinafter, this plane is also referred to as L1). Is the intersection of.
Therefore, the three-dimensional position of the point P can be obtained by obtaining the intersection of the line of sight L2 and the plane L1. Such a principle has been widely known from the past.

By the way, in order to actually perform three-dimensional measurement based on such a principle, the following two conditions (1),
(2) must be satisfied.

(1) Slit light plane L in three-dimensional space
The position and orientation of 1, that is, the equation of the plane L1 is known. (2) The line-of-sight L2 corresponding to the position p (u, v) on the image plane IP can be calculated. In order to satisfy these conditions, it is necessary to calibrate the three-dimensional visual sensor for each of (1) and (2). The present invention provides a calibration method for satisfying the condition (1), and the calibration for satisfying the condition (2) is outside the scope of the present invention.

Generally, the projection direction of the slit light L1 is not fixed, but the projection direction is changed within a certain range,
The light band Q formed on the work W at each light projection position is observed by a camera. Therefore, in the usual method, the projection direction is controlled by reflecting the laser light beam expanded in one direction by the cylindrical lens by the rotating mirror driven by the motor. Therefore, the projection direction of the slit light L1 is controlled through controlling the rotation angle of the rotating mirror. The rotation angle of the rotating mirror is determined by a command value from a control device such as an image processing device incorporated in the three-dimensional visual sensor system.

A major problem in using a three-dimensional visual sensor having a deflection mechanism using such a rotating mirror is the equation of the command value of the rotation angle of the mirror and the slit light plane L1 actually projected. That is, the calculation procedure for obtaining the relation of is very complicated.

Now, the equation of the incident light (which has a slit shape by a cylindrical lens) on the mirror expressed on the coordinate system (mirror coordinate system o-xyz) with the rotation axis of the mirror as the z axis is as follows ( Formula 1), world coordinate system (O-XYZ)
Assuming that the equation of the reflected light from the mirror is the following equation (2), the following equations (3) to (6) are established.

Ai x + bi y + ci z + di = 0 (1) AX + BY + CZ + D = 0 (2) A = (r11ai-r21bi) cos 2θ + (r11bi-r21ai) sin 2θ-r31ci ... (3) B = (R12ai-r22bi) cos2 [theta] + (r12bi-r22ai) sin2 [theta] -r32ci ... (4) C = (r13ai-r23bi) cos2 [theta] + (r13bi-r23ai) sin2 [theta] -r33ci ... (5) D = (tx ai −ty bi) cos 2θ + (tx bi −ty ai) sin 2θ + 2dm (ai cos θ + bi sin θ) −tz ci −di (6) where rjk, tx, ty and tz are mirror coordinate systems. Is a coefficient of a transformation matrix that represents the relationship between the and the work coordinate system. Further, θ is a rotation angle of the mirror, and usually has a relationship of θ = αV + β (α and β are constants) as a command value V from the image processing apparatus.

The purpose of the calibration is to set a plane parameter A, which defines the projection direction corresponding to a specific command value V,
To find B, C, and D. Conventionally, r11 to r33, tx, ty, ai, b in the above equations (3) to (6) for the values Vh (h = 1, 2, 3 ...) Of a plurality of command values V.
A method of calculating i, ci, di, dm and α, β has been used. However, this calculation is equivalent to solving the nonlinear simultaneous equations for a total of 19 unknowns, and the processing therefor was very complicated. Further, since the deflectable range of the angle θ of the mirror is usually relatively narrow, it is difficult to improve the calculation accuracy of each unknown.

[0013]

SUMMARY OF THE INVENTION An object of the present invention is to overcome the above-mentioned problems of the prior art. That is, the invention of the present application is to enable a slit light projection type three-dimensional visual sensor to perform calibration regarding the slit light projection direction by a simple calculation process.

[0014]

The invention of the present application is, as a basic technical means for solving the above technical problems, "projecting slit light in a controlled direction to form an optical band on an object. Through a light projecting means, a camera means, an image analyzing means for analyzing a video signal acquired through the camera means, and a command value for defining the light projecting direction of the slit light to the light projecting means. A slit light calibration method in a three-dimensional visual sensor, comprising: a means for controlling a projection direction of the slit light from the light projection means, under the condition that a plurality of different values are given for the command value. The data for interpolating calculation for obtaining the slit light projection direction corresponding to the command value at the time of execution of the measurement is the data representing the value of the plane parameter indicating the projection direction of the slit light corresponding to each command value. The method "is obtained by providing (configuration described in claim 1), characterized in that by comprising the step of determining by.

Further, in the above method, "data representing the value of a plane parameter representing the projection direction of the slit light corresponding to each command value is measured under the condition that a plurality of different command values are given. As a means of embodying this "step of obtaining as interpolation calculation data for obtaining the slit light projection direction corresponding to the command value at the time of execution", "an object having a flat surface for projecting slit light is known. Calculating the position for one or more points on the light band formed under each condition arranged at different positions, and based on the data representing the position, an equation representing the corresponding projection plane. It is proposed to be carried out by a process that includes the step of calculating the parameters to be defined.

[0016]

In the present invention, in order to calibrate the slit light, the unknowns r11 to r33, tx, ty, ai, bi, c in the above equations (3) to (6) are not always required.
It is not necessary to perform a calculation for obtaining all of i, di, dm and α, β, and the command value V and the plane parameter A within the deflectable range of the mirror angle θ (which is usually relatively narrow). This is based on the recognition that it is sufficient to approximately calculate the correspondence relationship of D by interpolation calculation.

In the present invention, a plurality of different values Vh (h = 1, 2, 3 ... N) are given for the command value V, and the slit light projection direction corresponding to each command value is given under each condition. N sets of data representing the values of the plane parameters A to D are obtained, and the data are stored as they are or after being processed into a suitable form in a suitable storage means of the three-dimensional visual sensor system. As data processing, for example, it is conceivable to find the relationship between the command value V and the plane parameter in finer steps by fitting calculation to a linear function, a polygonal line function, or the like. The data obtained in this way is used to obtain the slit light projection direction during measurement.

[0018]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 2 shows the outline of the system configuration of a three-dimensional visual sensor to which the slit light calibration method of the present invention is applied. The whole system is
Image processing device 1, sensor unit controller 2, sensor unit 1
It consists of zero. The image processing apparatus 1 is of a commonly used type and includes a microprocessor (hereinafter, referred to as “CPU”) that controls the image processing apparatus 1 as a whole.

This CPU has a ROM memory in which a program for image analysis, a program for controlling the sensor controller 2 and the like are stored, and an image taken by a CCD camera 11 mounted on the sensor unit 10 is displayed in gray scale. A frame memory that stores as a grayscale signal, and
An image processing processor for image analysis and a RAM memory used for storing various setting values and temporarily storing data during calculation are connected.

Further, in addition to a manual operation section equipped with a keyboard, a sensor section controller 2 and a CR for an image monitor are provided.
A general-purpose signal interface serving as an input / output device for T or an external computer or robot controller is connected to the CPU (individual elements are not shown). The keyboard of the manual operation unit has a function of moving a cursor on a CRT for image monitor and inputting a manual command.

The projector 12 of the sensor unit 10 converts the laser light into slit-shaped light through a cylindrical lens, further reflects it by the deflecting mirror 13 and projects it toward the object to be measured. . The deflection mirror 13 is rotationally controlled by a motor 15 about its rotation axis 14, and the direction in which the slit light is projected is a command value signal given from the image processing apparatus 1 via the sensor unit controller 2. Can be controlled by.

The ON / OFF of the projector 12 is also controlled by the image processing apparatus 1 via the sensor section controller 2. Therefore, the image processing device 1
The ON / OFF state of the projector 12 and the projection direction of the slit light can be constantly monitored and controlled.

When a necessary light projection command is sent from the image processing apparatus 1 to the sensor controller 2, the light projector 12 is turned on and slit light is projected in a direction corresponding to the rotation angle θ of the deflecting mirror 13. The slit light projected on the object forms a bright light band on the surface. In this state,
When a shooting command is sent from the image processing apparatus 1 to the sensor unit controller 2, the CCD camera 11 operates and a light band image is shot. The image signal is taken into the image processing apparatus 1 via the general-purpose interface and stored in the frame memory.

Although the above-mentioned configuration and function are basically the same as those of the ordinary three-dimensional visual sensor system, in this embodiment, the calculation necessary for executing the calibration is executed by the procedure described below. A program and setting values for doing so are stored in advance in the ROM or RAM of the image processing apparatus 1, and a register area for storing the acquired calibration data is set in the RAM of the image processing apparatus 1. There is.

A procedure for carrying out the slit light calibration method according to the present invention using the system having the above-described structure and function will be described below with reference to FIGS. 3 and 4. FIG. 3 conceptually shows the arrangement for executing the calibration, and the elements shown in FIG. 2 are designated by the same reference numerals. Further, reference numeral 4 is an object for calibration, and is provided with a flat surface 41 having an appropriate light diffusive reflectivity. When the slit light is incident on the flat surface 41 of the calibration object 4 from the light projector 12 according to the rotation angle θ of the deflection mirror 13, an optical band is formed, and the CCD camera 11 captures the optical band.

FIG. 4 is a flowchart showing an example of a process for executing calibration using the arrangement shown in FIG. The outline of each step is as follows. The camera 1 mentioned in the description of the prior art
It is assumed that the calibration of 1 has been completed by a known appropriate method.

Step S1; the calibration object 4 is flattened within the projection range of the projector 12 by its flat surface 41.
Are placed on the known first plane Γ on the world coordinate system Σ w.

Step S2: The projector 12 is turned on by operating the keyboard of the manual operation unit of the image processing apparatus 1. Step S3: Command value Vh = Vmin for the deflecting mirror 13.
+ HΔV (h is an integer index having an initial value of 0 representing the rotation angle of the deflection mirror 13; h = 0, 1, 2, ... N), the slit light is projected onto the calibration object 4, and its flat surface A light band is formed on 41.

In the first step S3, h = 0 is set,
Send the command value Vh = Vmin. Here, Vmin is a command value corresponding to the minimum rotation angle θmin within the operation range of the deflection mirror 13. Further, ΔV is an interpolation step amount set for the command value V, and ΔV = (Vmin to Vmax, where Vmax is the command value that gives the maximum rotation angle θmax within the operating range.
) / N. The value of n is a positive integer value determined in consideration of the magnitude of the operating range θmin to θmax of the rotation angle θ, the degree of linearity of the V-θ characteristic, the required calibration accuracy, and the like. As a practical value of n, for example, about 5 to 20 can be considered. In FIG. 3, L11, L12, L13 are shown as the light projecting surfaces and the light bands corresponding to different values of h.
And G11, G12, G13 are illustrated.

Step S4: The optical band formed on the flat surface 41 of the calibration object 4 is photographed by the camera 11. Step S5: Display the light band image on the monitor CRT,
By moving the cursor, a plurality of points on the optical band image are appropriately selected and designated, and a calibration calculation program is activated to cause the CPU to calculate an equation representing the line of sight of the camera 11. In FIG. 3, P1-1 and P1- are designated points on the optical band G11.
2, P2-1 and P2-2 are illustrated as designated points on the light band G12.

Step S6: The three-dimensional position of the point selected in step S5 is obtained as the position of the intersection of the line of sight corresponding to each point and the plane on which the flat surface 41 lies. In the example of FIG. 3, two points P1-1, P1-2 or two points P2-1 on Γ1,
Calculate the position of P2-2. The data representing the calculation result is
The image is stored in a predetermined area prepared on the RAM of the image processing apparatus 1.

Step S7: The index value h is incremented by 1. Step S8; Step S unless h> n
Return to 3. Steps S3 to S8 are repeated until h becomes n + 1. As a result, the command value Vmin corresponding to the rotation angle distributed over the entire rotation angle range of the rotating mirror 13.
, Vmin + ΔV, Vmin + 2ΔV ... Vmin + nΔ
For V (= Vmax), a point Ph- on each projection surface L1h
Position data such as 1 and Ph-2 are acquired.

Step S9: When the above process is completed, it is confirmed that the predetermined number of calibration object placements at different positions (here, two times, generally, any number of times of two or more) have not been reached. Confirm, step S1
After 0, the process proceeds to step S11. Step S10: The index h indicating the rotation angle of the deflection mirror 13 is reset to 0.

Step S11: The calibration object 4 is placed on the flat surface 4 within the projection range of the projector 12.
1 is placed so as to ride on the known second plane Γ ′ on the world coordinate system Σw, and the process returns to step S3. Hereinafter, with respect to the plane Γ ′, the same procedure as the above steps S3 to S8 is repeated. In FIG. 3, P1-3 and P1-4 are illustrated as the position calculation designation points on the second plane Γ ′.

In this way, by repeating the processing cycle (steps S3 to S8) corresponding to each of the planes Γ and Γ ′, the rotation angles distributed over the entire rotation angle range of the rotating mirror 13 are eventually obtained. Corresponding command value Vmi
n, Vmin + ΔV, Vmin + 2ΔV ... Vmin + nΔ
For V (= Vmax), a point Ph- on each projection surface L1h
Position data such as 1, Ph-2, Ph-3, Ph-4, etc. are acquired.

Then, in the second step S9, it is confirmed that the calibration object arrangement has been executed a predetermined number of times, and the process proceeds to the next step S12. Step S12: For each h (= 0, 1, 2, ... N), the points Ph-1, Ph-2, P calculated in Step S6 are calculated.
From the position data of h-3, Ph-4, etc., for each h, a plane on which these points Ph-1, Ph-2, Ph-3, Ph-4, etc. are in common (= projection surface L1h) Find the parameters that define the equation that represents In general, the plane in 3D space is
Since it is expressed by the above-mentioned equation (2), AX + BY + CZ + D = 0, as a parameter, A that defines the equation Ah X + Bh Y + Ch Z + Dh = 0 expressing the projection plane L1h is defined.
It suffices to calculate h, Bh, Ch and Dh. The data set obtained for each value of h (0, 1, 2, ... N) is stored in a predetermined register area in the image processing apparatus 1. With the above, calibration is achieved.

In order to determine each projection plane L1h, it is sufficient to know the positions of at least three points which are on the plane L1h and do not exist on one straight line. Therefore, the position is designated corresponding to one value of h. It is sufficient that at least three points Ph-1, Ph-2, ...
Therefore, the placement position of the calibration object 4 (Γ,
Γ ') is at least once required. For example, there are three types of placement positions of the calibration object 4 (Γ, Γ ′,
Γ ”), it is also possible to select a total of 3 designated points, one for each placement position, to obtain position data, and to calculate Ah, Bh, Ch, Dh corresponding to each h based on that. .

However, in general, one calibration object 4 is used in order to avoid an error due to a variation in measurement.
The positions of two or more (for example, five) points with respect to the arrangement positions of A, Bh, and Ch are calculated from the position data of several or more (for example, ten) by the least square method or the like.
, Dh should be calculated.

Further, the method of dividing the command value V, the change of the command value V, and the change of the arrangement position of the calibration object 4 may be carried out by appropriately changing the order or the like. That is,
Calculation of three or more positions on the projection plane and parameters Ah,
Bh, Ch, and Dh may be calculated at any time.

Further, the parameters Ah, Bh, Ch, Dh
May be processed into an appropriate shape in consideration of shortening the processing time when performing measurement. For example, when a system that gives a command value by a digital value is adopted, it is conceivable that the parameter value corresponding to the digital step of the command value V is first calculated by interpolation calculation and stored in a table data format.

[0041]

According to the present invention, the slit light projecting type 3
The calibration of the slit light projection direction in the three-dimensional visual sensor can be performed by a simple calculation process.

[Brief description of drawings]

FIG. 1 is a conceptual diagram illustrating a measurement principle of a slit light projection type three-dimensional visual sensor.

FIG. 2 shows an outline of a system configuration of a three-dimensional visual sensor to which the slit light calibration method of the present invention is applied.

FIG. 3 conceptually shows an arrangement for performing calibration according to the present invention.

FIG. 4 is a flowchart showing an example of a process for executing calibration using the arrangement shown in FIG.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Image processing device 2 Sensor part controller 4 Calibration object 10 Sensor part 11 CCD camera 12 Projector 13 Deflection mirror 14 Deflection mirror rotation axis 15 Motor 41 Flat surface of calibration object G11, G12, G13, G11 ' , Q Light band IP image plane L1, L11, L12, L13 Projection plane L2 Camera line of sight P Point on light band P1-1 to P2-2 Specified point to calculate position W Target object Γ, Γ'Calibration Plane that represents the flat surface position of the object

Claims (2)

[Claims] 1. A light projecting means for projecting slit light in a controlled direction to form a light band on an object, a camera means, and an image signal acquired through the camera means. Three-dimensional vision including image analysis means and means for controlling the projection direction of the slit light from the projection means by giving a command value that defines the projection direction of the slit light to the projection means. A method for calibrating slit light in a sensor, wherein data representing the value of a plane parameter representing the projection direction of slit light corresponding to each command value under the condition of giving a plurality of different values for the command value, The method, further comprising a step of obtaining as interpolation calculation data for obtaining a slit light projection direction corresponding to the command value at the time of executing measurement. 2. Under the condition that a plurality of different values are given for the command value, data representing the value of a plane parameter representing the projection direction of the slit light corresponding to each command value is used as the command at the time of execution of measurement. The step of obtaining as the interpolation calculation data for obtaining the slit light projection direction corresponding to the value is the light formed under each condition in which the object having the flat surface for projecting the slit light is arranged at the known different position. Calculating the position for one or more points on the band,
The method for calibrating slit light of a three-dimensional visual sensor according to claim 1, further comprising the step of calculating a parameter defining an equation representing a corresponding projection plane based on the data representing the position.