JPH02171966A - Device for recognizing image - Google Patents

Abstract

PURPOSE:To eliminate the need for a mechanism for rotating cameras and a sensor for detecting the rotational amount of the cameras and to reduce the size, weight and price of the device by furthermore processing respective three-dimensional space data obtained from two cameras to obtain accurate three-dimensional space data. CONSTITUTION:The device is provided with a left camera operation processing part 5 for forming three-dimensional image data based upon two-dimensional image data recorded in left camera image storing parts 3a, 3b, a right camera operation processing part 6 for forming three-dimensional image data based upon two-dimensional image data recorded in right camera image storing parts 4a, 4b and a solid image recognition processing part 9 for forming solid image data based upon the three-dimensional data outputted from a left camera three- dimensional image storing part 7 and a right camera three-dimensional image storing part 8 and the two cameras are fixed on a camera board. Consequently, the mechanism for rotating the cameras and the sensor for detecting the rotational amount of the cameras can be eliminated, the size, weight and price of the device can be reduced and control for operating the two cameras can be omitted.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention relates to an image recognition device, and in particular, it is used to recognize the external world as a vision of an autonomous mobile device, and to recognize a target product in three dimensions during product inspection. The present invention relates to an image recognition device for recognition.

[Conventional technology]

In the conventional image recognition device used for mounting on a robot, as shown in FIGS. 2 and 3, two left cameras 1 and right cameras 2 mounted on a camera stand 23 each point the target at its focal point. It was possible to change the camera's position so that it could be captured above. In other words, in order to obtain the movement of the camera housing necessary for each camera to three-dimensionally recognize the object to be recognized and perform target tracking operation, an operating mechanism with two or more degrees of freedom and its operation amount are required. Left camera direction sensor 24 as a sensor for knowing. Right camera direction sensor 25.
Left camera elevation angle sensor 27. It had a right camera elevation angle sensor 28.

In addition, the calculation control unit 26 performs calculation ratio processing so that the spatial coordinates of the target object can be stored in the memory in a three-dimensional orthogonal coordinate system, but the two-dimensional spatial image captured by one camera and the other Since the arithmetic control unit 26 performs arithmetic processing on information from a two-dimensional spatial image captured by one camera, a three-dimensional memory for processing three-dimensional data had to be used.

[Problem to be solved by the invention]

When the conventional image recognition device described above is mounted on a robot, it has a mechanism to rotate the camera and a sensor to determine the amount of rotation, so the structure is complicated and expensive, the shape and weight are large, and it is difficult to operate the camera. There was a problem in that it required control.

In addition, since processing is performed based on the information of the two-dimensional spatial image captured by one camera and the two-dimensional spatial image captured by the other camera, the shape of the object to be recognized captured by the two cameras is different from the viewpoint of each. There is a problem in that the coordinates change depending on the angle, and the coordinates that serve as the target recognition reference point are not fixed, making recognition difficult.

The purpose of the present invention is to make the device smaller, lighter, and less expensive without requiring a mechanism for rotating the camera and a sensor for determining the amount of rotation, and without having to control the operation of two cameras. In addition, since three-dimensional spatial data is generated from data indicating two-dimensional spatial coordinates obtained from two cameras and changes in the size of the recognition target, each three-dimensional space obtained from two cameras An object of the present invention is to provide an image recognition device that can further process original spatial data to obtain accurate three-dimensional spatial data.

[Means to solve the problem]

The image recognition device of the present invention includes: (A) a viewing unit equipped with a left camera and a right camera fixed on a camera stand; (B) an image storage unit for the left camera that stores image data of the left camera; (C) a right camera image storage unit that stores the image data of the right camera; (D) a left camera arithmetic processing unit that processes and outputs the image data of the left camera image storage unit; (E) the right camera processing unit that processes and outputs the image data of the left camera image storage unit; (F) a right camera arithmetic processing unit that processes and outputs the image data of the camera image storage unit; (F) a left camera three-dimensional image storage unit that stores the image data output from the left camera arithmetic processing unit; G) a three-dimensional image storage unit for the right camera that stores image data output from the arithmetic processing unit for the right camera; (H) image data of the three-dimensional image storage unit for the left camera and the three-dimensional image for the right camera. a stereoscopic image recognition processing unit that generates and outputs stereoscopic image data by subjecting the image data in the storage unit to polar coordinate system spatial transformation processing; (I) a stereoscopic image storage unit that receives and stores the stereoscopic image data from the stereoscopic image recognition processing unit; It is configured with the following.

〔Example〕

Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram of one embodiment of the present invention.

The image recognition device shown in FIG. 1 is mounted on an industrial robot, and includes a viewing unit 11 having a left camera 1 and a right camera 2 fixed on a camera stand for photographing an object to be recognized, and a mesh-like two-dimensional The left camera image storage sections 3a and 3b record the image data of the left camera, the right camera image storage sections 4a and 4b record mesh-like two-dimensional image data, and the left camera image storage sections 3a and 3b record the image data. Three-dimensional image data is generated based on the two-dimensional image data and changes in the image data of the object due to the passage of minute time between the left camera image storage section 3a and the left camera image storage section 3b. Arithmetic processing unit 5 for left camera, 3D image storage unit 7 for left camera that records generated 3D image data, and image storage unit 4 for right camera.
Based on the two-dimensional image data recorded in a and 4b and changes in the image data of the object due to the passage of minute time between the right camera image storage section 4a and the right camera image storage section 4b, three-dimensional A calculation processing unit 6 for the right camera that generates the original image data, a 3D image storage unit 8 for the right camera that records the generated 3D image data, and a 3D image storage unit 7 for the left camera.
The three-dimensional image recognition processing section 9 generates stereoscopic image data based on the three-dimensional data stored in the three-dimensional image storage section 8 for the right camera, and the stereoscopic image processing section records the generated stereoscopic image data.

Next, the operation will be explained.

Figure 4 is an explanatory diagram showing the relationship between vectors and two coordinates, Figure 5 is an explanatory diagram showing the relationship between vectors and coordinate correction values, Figure 6 is an explanatory diagram showing the three-dimensional image data structure, and Figure 7 is an explanatory diagram showing the three-dimensional image data structure. FIG. 2 is an explanatory diagram showing the structure of image storage unit data file.

First, the left camera arithmetic processing section 5 and the right camera arithmetic processing section 6 perform the following processing on their respective storage sections.

The image data in the left camera image storage section 3a and the right camera image storage section 4a are converted into a binary image of a contour and a region by the contour detection functions of the left camera arithmetic processing section 5 and the right camera arithmetic processing section 6. The coordinates of a part of the image data converted to a binary image are extracted as feature points by the feature point extraction function,
As shown in FIG. 6, the feature code 21 for the feature point
and X.

A Y coordinate value (coordinate data 19 of the feature point) is assigned and registered as a feature point.

Next, the image data of the two left camera image storage units 3a and left camera image storage unit 3b, and the right camera image storage unit 4a and right camera image storage unit 4b, which have a slight temporal shift, are respectively stored. As shown in Fig. 4, the magnitude of the vector of the same feature point is 12 and
) The two coordinates 14 of the feature point can be determined from the distance 13 from the image center (origin center) determined by t/2.

In the vector comparison Z coordinate calculation table of FIG. 4, the Z coordinate 14 can be determined by applying the distance 13 from the image center and the magnitude 12 of the vector to the vertical and horizontal axes of the table.

Next, the vectors of all the feature points intersect, and the intersection points of the plurality of vectors are registered as infinity point coordinates, and the infinity point code 18 through which the vector passes is determined for each coordinate, as shown in FIG. Assign.

Each feature point has one or more corresponding infinity point codes 18. Among them, the camera-specific infinity point is defined as the reference infinity point. The feature point coordinate data 19, which is the x, y, and z coordinates of each feature point, the feature code 21, and the infinite point code 18 are used as three-dimensional information to be stored in the three-dimensional image storage unit 7 for the left camera and the three-dimensional image storage unit for the right camera. The image is registered in the image storage unit 8 as data.

The three-dimensional image recognition processing section 9 is a three-dimensional image storage section 7 for the left camera.
And image data to be recorded in the three-dimensional image storage section 10 from the three-dimensional image storage section 8 for the right camera is generated in the following procedure.

A three-dimensional image storage unit 7 for the left camera and a three-dimensional image storage unit 8 for the right camera have one coordinate data of the feature point, a feature code 21, and an infinite point code 18 for each feature point.
With respect to the coordinate data 19 of feature points other than the feature code 21 and the infinite point code 18, correction calculations are performed based on the movement of the object to be recognized and the difference in the positions of the left camera 1 and right camera 2.

Next, the three-dimensional image recognition processing section 9 compares the same infinity point coordinates of the infinity point code 18 in the three-dimensional image storage section 7 for the left camera and the three-dimensional image storage section 8 for the right camera, and The feature points of the left camera 3D image storage unit 7 and the right camera 3D image storage unit 8 having the infinity point code 18 are compared, and two x,
The y-coordinate is corrected using the three-dimensional coordinate correction value 17 shown in FIG. 5, and one coordinate data of the feature point is stored in the three-dimensional image storage unit 10 (FIG. 6).

FIG. 5 shows the distance 15 from the average coordinate, which is the average value of the coordinates of the left camera 3D image storage unit 7 and the right camera 3D image storage unit 8, for each of the X and Y coordinates of the feature point. , by taking the difference from the vector magnitude 12 (vector difference 15) and applying each to the vertical and horizontal axes to find the coordinate correction value 17 to be added to the average value of each of the X and Y coordinates. be able to.

Feature points having the same infinity point code 18 are each assigned the same group code 20 as the same moving group, and become data in the stereoscopic image storage unit 10 (FIG. 7).

In this way, there is no need for a mechanism for rotating the two cameras and a sensor for determining the amount of rotation, making the device smaller, lighter, and less expensive. Control can also be made unnecessary.

In addition, in a device such as a robot equipped with the image recognition device of this embodiment, the processing speed of other controls is improved since camera control is no longer necessary. Furthermore, since three-dimensional spatial data is generated from data indicating the two-dimensional spatial coordinates obtained from the white camera and changes in the size of the recognition target, each three-dimensional space obtained from the two cameras The data can be further processed into accurate three-dimensional spatial data.

〔Effect of the invention〕

As explained above, by fixing two cameras to a camera stand, the present invention eliminates the need for a mechanism for rotating the cameras and a sensor for determining the amount of rotation, and the device can be made smaller, lighter, and lighter. It has the advantage of being lower in price and eliminating the need for controls for operating two cameras.

In addition, since three-dimensional spatial data is generated from data indicating two-dimensional spatial coordinates obtained from one camera and changes in the size of the recognition target, each three-dimensional space obtained from two cameras This has the advantage that the data can be further processed into accurate three-dimensional spatial data.

[Brief explanation of the drawing]

Fig. 1 is a block diagram of an embodiment of the present invention, Fig. 2 is a block diagram of a conventional image recognition device, Fig. 3 is a structural diagram of the conventional image recognition device shown in Fig. 2, and Fig. 4 is a block diagram of a conventional image recognition device. FIG. 5 is an explanatory diagram showing the relationship between Z coordinates, FIG. 5 is an explanatory diagram showing the relationship between vectors and coordinate correction values, FIG. 6 is an explanatory diagram showing the three-dimensional image data structure, and FIG. 7 is an explanatory diagram showing the three-dimensional image storage data structure. FIG. 1...Left camera, 2...Right camera, 3
a, 3b... Image storage unit for left camera, 4a, 4
b... Image storage section for right camera, 5...
Arithmetic processing unit for left camera, 6... Arithmetic processing unit for right camera, 7... Three-dimensional image recognition unit for left camera,
8...Three-dimensional image recognition unit for right camera, 9...
... Stereo image recognition processing section, 10 ... Stereo image storage section, 11 ... Viewing section, 12 ... Vector size, 13 ... Distance from the center of the image, 14...Z coordinate of the feature point, 15...
Vector difference, 16...Distance from average coordinates,
17... Coordinate correction value, 18... Infinity point code, 19... Coordinate data of feature point, 20
...Group code, 21 ...Feature code, 23 ...Camera stand, 24 ...Left camera direction sensor, 25 ...Right camera Direction sensor, 26... Arithmetic control unit, 27... Left camera elevation angle sensor, 28... Right camera elevation angle sensor.

Claims (1)

[Scope of Claims] (A) A viewing unit equipped with a left camera and a right camera fixed on a camera stand, (B) An image storage unit for the left camera that stores image data of the left camera, (C ) a right camera image storage unit that stores image data of the right camera; (D) a left camera arithmetic processing unit that processes and outputs image data of the left camera image storage unit; a right camera arithmetic processing unit that processes and outputs the image data of the image storage unit; (F) a left camera three-dimensional image storage unit that stores the image data output from the left camera arithmetic processing unit; (G) a three-dimensional image storage unit for the right camera that stores image data output from the arithmetic processing unit for the right camera; (H) image data of the three-dimensional image storage unit for the left camera and a three-dimensional image storage unit for the right camera; (I) a stereoscopic image storage unit that receives and stores the stereoscopic image data from the stereoscopic image recognition processing unit; An image recognition device comprising: