JPS6238904A - Learning method for visual system for robot - Google Patents

Abstract

PURPOSE:To easily complete the registration of component shapes in a short time by transferring design data on components to a visual device for a robot on an on-line basis and giving some margin to the design data. CONSTITUTION:A CAD device 4 and the visual device 4 are connected so as to transmit commands or data forward and backward, and component names or component numbers are transmitted, one by one, from the visual device 4 to the CAD device 5. The CAD device 5 transmit design data corresponding to the component names or numbers to the visual device 4. The design data include auxiliary data (center point and radius of peripheral part) attached to outward appearance figures and outward appearances of components. When an indication of transition to 'margin generation mode' is supplied to the visual device 4, the visual device 4 calculates the range of variance of each outer peripheral part on the basis of auxiliary data on the outer periphery and displays the margin on the display 9 of the visual device together with the reference data. The margin is corrected by using an input means such as a mouse 10 or light pen to set the range of variance in each feature quantity.

Description

Detailed Description of the Invention [Industrial Application Field] The present invention is an industrial robot, especially an assembly robot that performs material handling, etc.
This field relates to a visual system that has the function of identifying position, direction, etc., and in particular to a method for registering (teaching, learning) its standard data.

[Conventional technology]

A typical example of this type of shape recognition device is the V
There is a module. This was developed primarily as a robot's eye, and it images the shape of the target industrial part with a solid-state camera, performs binarization processing, and then extracts the outline of the shape through connectivity analysis. Shapes are recognized and classified by measuring feature quantities (area/perimeter, shape coefficient, etc.) and comparing them with standard patterns learned in advance.

Figure 9 shows the Vision Mcxrule of S 几i.
It is a functional block diagram showing. In addition, in the same figure, 1
11 is an imaging device such as a solid-state camera, 11 is a preprocessing circuit, and 12 is an imaging device such as a solid-state camera.
11:2 [EJ path, 13 is a processing device such as a microprocessor, and 14 is an external interface.

Further, the processing device 13 includes a connectivity analysis section 131, a feature amount calculation section 132) a feature amount determination section 133, a standard pattern setting section 134, and the like.

Video of the object shape captured by the solid-state camera 1
The (video) signal is subjected to processing such as A/D conversion and noise removal in the preprocessing circuit 11, and then binarized with a certain threshold in the binarization circuit 12, and the shape is black and white. converted to a value pattern. Regarding preprocessing and binarization processing, there are many methods in this field of technology, and since this part is not the main purpose of the present invention, it will not be specifically mentioned here. This binary pattern of the target shape is given to a processing device 13 such as a microcomputer, and its connectivity analysis section 131 extracts and separates the outline of the shape, and the feature amount calculation section 132 extracts the area, perimeter, and hole of the shape. Number/Large area/
Shape identification parameters such as shape coefficients are calculated, and at the same time, position/direction identification parameters such as center of gravity position and moment of inertia are calculated. The target shape is determined by comparison with a standard pattern created in advance by a teaching operation called showing (teaching) and set in the setting unit 134 using a recognition method generally called S-i algorithm. The shape determination results are transmitted in the form of component names, their positions, rotation angles, etc., to a control device such as a robot through the external interface 14 using a transmission protocol called R.8232C.

FIG. io is a reference diagram for explaining an example of the discriminant tree (W-separated tree) method, which is a shape recognition method in the SM algorithm.

As shown in FIG. xl-
Using the perimeter, X2 = total area of the hole, X3 - minimum distance from the center of gravity to the outer contour, and X4 = maximum distance from the center of gravity to the outside, a discriminant tree (binary) as shown in Figure (0) is created.
Each of the parts is determined by the decision tree.

Now, in such an SRi algorithm, in order to determine the set value for each part of each feature in the above-mentioned discriminant tree, a ``Showing''
g) It is common to perform a teaching process called teaching.

That is, the range of each characteristic amount of each part is calculated by capturing an image of the part to be recognized with the device in advance, and is stored as a standard pattern of the part. In such show winging, images are repeatedly taken assuming all possible positions, postures, and individual differences of each part, statistical processing is performed on the dispersion of feature values in each image, and each feature of each part is calculated based on the average value and its deviation. It is normal to use a set value for the amount.

[Problem that the invention seeks to solve]

By the way, a problem with such a shape recognition device is that all the setting values in the discriminant tree depend on the imaging data obtained by showing. Therefore, in order to avoid unidentified or misidentified genuine parts,
As described above, the user must repeatedly take images assuming all possible positions, postures, and individual differences of the target part. In addition, showwing must take into account all possible variations in not only the target object but also the surrounding lighting conditions and the position of the solid-state camera. This makes it difficult to introduce this type of shape recognition device into a production line, and is also a reason for its widespread use.

Due to insufficient showwing, the target shape is not identified (
The following are the reasons why it is easy to become indiscernible.

(1) In general, the user considers all positions, all angles, and all individual differences of the target shape,
It is nearly impossible to conduct multiple showwings.

(2) Generally, a target image is converted into a digital image of about 256×256, but at this time, a “quantization error” essentially occurs due to digitization. For example, as shown in FIGS. 11(a), 11(b), and 11(c), even if the shapes are the same, the contour changes slightly depending on the angle (position) relative to the scanning direction, and quantization errors appear. In other words, this quantization error causes variations in the values of each feature amount, and the positions of the end points (convex points and four points) of the image often become unstable. Such "quantization error"
In order to prevent this, one possible solution is to make the digital mesh finer, but for example, 256X
If the number of pixels is increased from 256 to 512×512 or 1024×1024, the number of pixels increases dramatically, resulting in an increase in processing time and the loss of real-time performance that is essential for this type of device.

(3) When actually introducing it into a production line, the position of the solid-state camera may shift during operation, or the lighting conditions may change, so even if normal identification is possible immediately after show-wing. , the number of unidentified cases may increase due to changes in ambient conditions during operation. In such cases, the problem may be solved by redoing the show wing again under new conditions, but as mentioned above, it is a difficult task to perform the show wing completely using the conventional method, so it may be necessary to perform the show wing again. Showwing is actually quite difficult.

(4) In recent years, the industrial structure has changed from mass production to FMS (F
lexi1) le Manufacturing
As the industry continues to evolve, the shapes of industrial parts are often subject to design changes due to the diversification of products. In such cases, with the conventional method, the show wing must be restarted from the beginning every time the design is changed.

Under the circumstances described above, there is a need for a new teaching (learning) method to replace the conventional show winging method. Therefore, it is an object of the present invention to provide a simple teaching (learning) method and to be able to flexibly deal with design changes.

[Means and effects for solving the problems] The present invention has the following features:
Conventionally, the only means to teach the part shape to the visual device for P-bot was through showwing, but now the part design data can be transmitted on-1ine (or off-1ine).
By transferring the design data to the robot vision device at the time, the conventional show-wing work is abolished, and by giving some margin to the design data, it is possible to complete the registration of part shapes easily and in a short time. It is.

In other words, if we consider the case of assembly robots in general,
There is always a groin diagram for the target parts, especially recently C
AD/CAM (CAD; Computer
Aided Design, CAM; Compute
With the spread of industrial aided manufacturing, design data is being stored on magnetic disks, magnetic tapes,
It is often stored on a 70-bit disc or an optical disc. Furthermore, even if parts are not necessarily designed using CAD, automatic drawing reading devices have become increasingly popular recently, and it has become possible to compile handwritten drawings into a database relatively easily. Therefore, the CAD device or automatic drawing reading device and the robot vision device can be connected on-line (or off-line).
By connecting the parts, design data of the designated part, especially information regarding the contour, can be input to the robot visual system in a clear form. In other words, the length of the straight line, the curvature of the curve, the position of the hole, its radius, etc. can be taught to the visual device as numerical values along with the shape, so the visual device can calculate these parameters from the conventionally shown data. There is no need to detect the length (1 curvature, 9 radius positions, etc.), and therefore instability associated with showwing can be eliminated.

Next, the design data transferred from the CAD or drawing reading device is registered in the visual device as a reference pattern for each part, but this alone is not suitable for actual identification because no margin is added. In other words, the part shape pattern actually captured by a solid-state camera, etc.
Since there are variations due to quantization errors associated with digitization, lighting conditions, etc., it is necessary to add a margin to the reference pattern input from CAD or the like to allow for these variations. In the present invention, the following two methods are used in combination as means for adding these allowable margin data.

First, quantization errors associated with digitization are essentially inherent variations regardless of surrounding conditions, so they can be predicted in advance. In other words, it is possible to determine how much quantization error margin should be added to a straight part according to its length, and also to determine the margin for a curved or circular part based on its curvature or radius. can. Therefore, by storing the margins of the straight line part and the curved part in advance in the visual device, it is possible to automatically add the margins within the visual device at the time when the design data of the part is input.

Next, if the above margin prediction cannot cover the
In other words, in the case of a part with an arbitrary shape, special ambient conditions such as non-standard lighting conditions, or when the shape of the part varies considerably due to the production process, the visual device automatically adjusts the margin of the part. is often undecidable. Even in such a case, as long as it is an industrial part, there is a blueprint, so the part designer or the engineer of the automated line using robots should be able to decide how much variation in parts is acceptable. be. Therefore, in the present invention, the visual device for the four bots is equipped with a display for displaying the part shape and a keyboard, light pen, mouse, etc. for indicating the variation in shape, so that the operator can To enable setting margins while checking the shape. That is, first, when the standard pattern of the part transferred from a device such as CAD is displayed on the display, the operator (operator) uses an input device such as a light ben or mouse to indicate the range of shape variation while looking at the display. By doing so, the margin of the part shape is determined.

By employing the above-described method, the system can easily and stably register part shapes without performing show-wing, and can also flexibly respond to design changes.

〔Example〕

Figure 1 is a system configuration diagram for explaining the present invention in detail, Figure 2 is a partial configuration diagram thereof, Figure 3 is a configuration diagram showing the configuration of a reference data file, and Figure 4 is for explaining shape margins. FIG. 5 is a flowchart showing the processing procedure according to the present invention. In FIG. 1, 1 is an imaging device such as a solid-state camera, 2 is a robot, 3 is a robot control device, 4 is a visual device, 5 is a CAD device, 6 is an auxiliary storage device, and 7 is a display.

The operation will be described below with reference to these figures.

That is, in the design drawing of an industrial part created by the CAD device 5, all the information necessary for manufacturing the part, such as the part's outline drawing, dimensions, curvature, center point, etc., is digitized and stored on a magnetic disk as a database.

The information is stored in an auxiliary storage device 6 such as a magnetic tape, a 70TB disk, or an optical disk. When the assembly robot 2 assembles parts by depalletizing (separating) a group of parts designed by the CAD device 5 or palletizing (collecting) randomly placed parts, the robot attached to the assembly robot 2 The registration of the parts group corresponding to the visual device 4 is performed in the following procedure. In addition,
A portion of the robot vision device 40 is shown in FIG.
It is composed of a keyboard 8, a display 9, a mouse 10, and the like.

The CAD device 5 and the visual device 4 are connected to a serial line (
R8232C protocol, etc.) or parallel line (G
They are connected so that commands or data can be sent to each other via PIB protocol, etc.). For the group of parts targeted by the assembly robot 2, each part name or part number is transmitted one by one from the visual device 4 to the CAD device 5 (
(See Figure 5■). The CAD device 5 transmits the design data corresponding to the sent part name or part number to the visual device 4 (see FIG. 5 (2)). The design data includes the outline drawing of the part (outer circumference, inner periphery, etc.) and auxiliary data associated with the outline (end point position of straight part and its length, end point and curvature of curved part, center point and radius of circumferential part) etc.).

At this time, since the design drawings are generally drawn according to trigonometry, it is necessary to transmit an outline drawing viewed from the same direction as the direction in which the camera 1 of the visual device actually images the part, but usually a front view and a front view are transmitted. do. However, in some cases, when the camera views the part from a side view position, for example, when the part name (or part number) is transmitted from the visual device 4 to the CAD device 5, that information is also added. shall be.

Now, the outline drawing and outline auxiliary data of the part transmitted from the CAD device 5 are stored in the parts standard data file in the visual device 4 together with the part name (part number). In this way, reference data files regarding all applicable parts are created one after another. An example of such a standard data file is shown in FIG. 3 (a).

Shown in (b), (c), and (d).

Next, when the operator instructs the visual device 4 to move to "margin creation mode," the visual device 4 calculates the range of variation of each peripheral part based on the peripheral auxiliary data.
The margin is displayed together with the reference data on the display 9 (see FIG. 2) of the visual device (see FIG. 5). An example of how the margins are displayed is shown in dotted lines in FIG. scattering(
The data on which the calculation of margin) is based is statistically determined based on the variation of the general straight m1 arc mainly due to quantization errors, and is based on the nonvolatile data in the visual device 4. It shall be set in advance as a data file in memory (however, it shall also be possible for the user to change it according to the conditions of use). Therefore, for example, the amount of variation in the circumference of a circle with a radius of 50 pixels within a field of view of 256×256 pixels can be calculated from this data file. In general, many industrial parts are composed of basic patterns such as straight lines and circles, but there are also parts with arbitrarily shaped parts, parts with large individual differences, and parts with standard surrounding conditions. There are cases where it is not. Since these cannot be predicted in a standard manner, this method provides an "operator confirmation mode" (see Figure 5, ■). In this operator confirmation mode, the operator uses an input means such as a mouse 10 (see Figure 2) or a light pen to view the reference data and margins displayed on the display and compare them with the actual part shape variations. to correct the margins. After the operator confirmation mode ends, the visual device 4 shifts to the "feature amount calculation mode" and calculates feature amounts such as area, perimeter, shape coefficient, etc. in the same way as before (see Figure 5 ■).
. At this time, the range of variation in each feature quantity is set in consideration of the margin of the part shape. Incidentally, the above operation is carried out until all parts are registered (see FIG. 5).

In the following, parts recognition, position/angle calculation, etc. are performed in the same manner as in conventional robot vision devices (see Figure 5 ■), and the results are transmitted to the four-bot control device 3 ( (See Figure 5■).

FIG. 6 is a configuration diagram for explaining another embodiment of the present invention.

The difference from the embodiment shown in FIG. 1 is that an automatic drawing reading device consisting of a reading section 51, a scanner 52, a display, etc. is connected to the robot visual device 4 instead of the CAD device. In other words, even though CAD/CAM has become popular, there are still many industrial parts that are drawn by hand.

In order to transmit the shape of the part according to these drawings to the visual device on-line, it is necessary to digitize the part shape, and as a means for that purpose, in this example, an automatic drawing reading device is used to digitize the design data. is stored in an auxiliary storage device (not shown). The subsequent processing is the same as in the previous embodiment.

Furthermore, as another embodiment, a system using a digitizer and a CAD device in place of the automatic drawing reading device described above may be considered. This is shown in FIG. Here,
53 is a digitizer.

In addition, in the above explanation, an example was shown in which a CA I) device, an automatic drawing reading device, a CAD, a digitizer, etc. are connected to a robot visual device by a cable,
In the present invention, on-line connection is not an essential condition. In other words, the exchange of data between the CAD device and the robot visual device is not done on-line, but is done off-line using media such as magnetic tape or a USB disk.
It may be an exchange of 1ine. Rather, if the location where the CAD equipment is installed and the location where the robot and visual equipment are installed are quite far apart, the off
- There are cases where it is more practical to communicate via 1ine. This is shown in FIG. This is 70 tsubi one medium 54
This is an example using .

〔Effect of the invention〕

According to the present invention, when teaching component data to the visual device for four bots, instead of using the conventional show wing method, it is connected to a CAD device or a drawing reading device, so the following points are improved. Ru.

■ In the conventional method using show wings, the system could not be stabilized without repeating many show wings, taking into account all the positions, angles, and individual differences of parts, but with this method, Since it is linked with design data, part registration can be completed relatively easily.

■ In the conventional method, inherent variations such as quantization errors that accompany the digitalization of component shapes could only be dealt with by repeated show winging, but with this method, straight sections, circular arc sections, etc. can be dealt with from the design data. Since it is clearly understood, the range of variation can be estimated automatically to some extent.

■ Changes in ambient conditions (lighting conditions, etc.) or changes in the design of parts would have required the showwing process to be restarted from the beginning using the conventional method, but this method can handle this relatively easily.

[Brief explanation of drawings]

Figure 1 is a system configuration diagram for explaining the present invention in detail, Figure 2 is a partial configuration diagram thereof, Figure 3 is a configuration diagram showing the configuration of a reference data file, and Figure 4 is for explaining shape margins. , FIG. 5 is a freechard showing the processing procedure according to the present invention, FIG. 6 is a configuration diagram for explaining another embodiment of the present invention, and FIG. 7 is a still another embodiment of the present invention. ″
f: A configuration diagram for explaining, FIG. 8 is a configuration diagram for explaining another embodiment of the present invention, FIG. 9 is a configuration diagram showing a conventional example for a visual system for 0 bots, and FIG. 10 is a configuration diagram for explaining another embodiment of the present invention. Reference diagram for explaining the discrimination method using the SRi algorithm, No. 11
The figure is a reference diagram for explaining quantization errors. Code explanation 1... Solid-state camera, 2... Robot,
3...Robot control device, 4...Visual device, 5...CAD device, 51...Reading unit, 52...Scanner, 53...Digitizer, 54...70 Tsubi-medium, 6...
...Auxiliary storage device [,7,9...Display, 8...Keyboard, 10...Mouse, 11...Preprocessing circuit, 12・・・・・・2
Value conversion circuit, 13... Processing device, 131...
... Connectivity analysis section, 132 ... Feature quantity calculation section,
133...Special quantity discrimination unit, 134...
Standard pattern setting section, 14...External interface. Agent Patent Attorney Akio Namiki Agent Patent Attorney Kiyoshi Matsuzaki Fig. 2 10 Makzu Fig. 3 j I 5 Fig. 10 Fig. 12 ＾ Calculator NO'IES 12>7 χ3 total'X χ3θ meter WNO
YES NOYES13>1.80
1.34No YES Shiso・ri1 χ+Qt sum 0 y)'”
r, >5.20 Figure H (4)

Claims (1)

[Claims] 1) An imaging means for capturing images of various target parts, and a processing device that measures various geometric features from the imaging signals and identifies the target parts by comparing these with standard data. In a robot vision system that functions as the eyes of a robot, an external device that stores at least the design data of the target parts is connected to the system, and the shape data of various parts taken in from the external device is converted into straight lines and circles, respectively. In addition to dividing the shape into basic elements containing A learning method for a robot vision system, characterized in that learning is performed as standard data for each target part. 2) In the learning method described in claim 1,
A learning method for a visual system for a robot, wherein the external device is a CAD device, a drawing reading device, or a combination of a CAD device and a digitizer. 3) In the learning method according to claim 1 or 2, the robot is characterized in that variations in the overall shape of the parts can be corrected on a display using an input device including a mouse and a light pen. Learning methods in visual systems.