JPH05314258A - Image processing device and robot control device - Google Patents

Abstract

(57) [Abstract] [Object] The present invention relates to an improvement in an image processing device and a robot control device, and automatically distinguishes a color of an object even if black or white is not clear in advance. Then, the object is to accurately calculate the measurement information of the target object and to recognize the target object at high speed. The image processing apparatus includes a range setting unit 11 that sets a calculation target frame F in an image acquisition target 16, and an image acquisition target 1.
The counting means 12 for counting the number of white pixels Wi or the number of black pixels Bj on the target frame F set to 6 is compared with the number of white pixels Wi or the number of black pixels Bj on the target frame F. Comparison means 1
3 and a calculation means 14 for calculating the barycentric coordinate value of the target image in the target frame F, and the number of white pixels W on the target frame F
It is configured to include identifying the color tone of the image acquisition target 16 in the target frame F based on i and the number of black pixels Bj, and the robot control device includes the robot working unit 17 and the image acquisition unit 1.
8 and an image measuring unit 19 including the image processing apparatus of the present invention.

Description

Detailed Description of the Invention

[Table of Contents] Industrial Application Field of the Prior Art (FIG. 9) Problem to be Solved by the Invention (FIG. 10) Means for Solving the Problem (FIGS. 1 and 2) Action Embodiment (1) First Example 1 (FIGS. 3 to 5) (2) Second example (FIGS. 6 and 7) (3) Third example (FIG. 8)

[0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing device and a robot control device, and more specifically,
The present invention relates to improvements in an apparatus that calculates various measurement amounts from image acquisition information and an application apparatus thereof.

In recent years, along with automation of manufacturing processes of various electronic devices and precision machines, a robot control device for positioning a robot hand on a target object and performing various teaching work is used. In addition, a control method has been developed in which a television camera is mounted as the eyes of the robot and the teaching work of the target object is performed based on the position information measured from the image acquisition data of the target object.

By the way, according to the robot controller of the conventional example, in order to speed up the calculation process of the barycentric coordinate value of the target image in the image measuring unit, the color of the target object is black and the background color around the target object is preset. Is assumed to be white or the color of the object is white and the background color around it is black, and "1" is set to black and "0" is set to white as the weighting of the target image. The barycentric coordinate value is calculated based on

Therefore, in the case where the discrimination condition that the color of the object is black or white and the background color around the object is white is not assumed, that is, in the case of the object of an unknown color, it is required to recognize it. If so, the image processing becomes difficult.

Further, when an image of a target object and an object other than the target object are included in the calculation target frame, the calculation processing is limited even in the calculation of the barycentric coordinate value for which the internal screen is calculated. And the calculated barycentric coordinates may not match the barycentric position of the object.

Therefore, even if black or white is not clearly defined in advance with respect to the color of the object, it is automatically discriminated and the measurement information of the object is accurately calculated. There is a demand for a device capable of recognizing an object and its applied device.

[0008]

9 and 10 are explanatory views of a conventional example. Further, FIG. 9 is an explanatory diagram of an object recognition method of a robot controller according to a conventional example, FIG. 9 (a) is its configuration diagram, and FIG. 9 (b) is its image processing flowchart. ing.

For example, a robot control device for positioning the robot hand 1 on a target object 10 for various teaching work is shown in FIG. 9A, which is a robot hand 1, a television camera 2, an image measuring section 3 and a robot. It consists of the control unit 4.

The function of the apparatus is that when the robot hand 1 equipped with the television camera 2 is moved to the vicinity of the area of the object 10, the image of the area is captured by the television camera 2. Further, the image measuring unit 3 converts the image signal into binarized data relating to black and white gradation, and the data is arithmetically processed by the image measuring unit 3 to recognize the object 10. As a result, the robot hand 1 is positioned on the object 10 and teaching work such as gripping work is performed.

10 (a) and 10 (b) are acquired image diagrams of the object 10 for explaining the problems in the conventional example, and the image data of the area of the object 10 is binarized by a certain slice level. The figure shows the changed bright parts in white and the dark parts in black on the monitor TV.

For example, when the robot hand 1 is positioned on the object 10, the object 1 is measured by the image measuring unit 3.
Assuming that the color of 0 is black, the calculation target frame F is set in the image acquisition target 16, and the barycentric coordinates of the target image inside thereof are measured.

This is because the discrimination condition between the color of the object 10 and the background color around the object 10 is set in advance in the image measuring section 3 so as to speed up the calculation process of the barycentric coordinate value of the object image. ..

That is, in FIG. 9B, first, in step P1, it is assumed that the color of the object 10 is black in advance, and then,
In step P2, the camera input image is binarized by an appropriate level. Next, in step P3, the calculation target frame F is set at a position where the object 10 is most likely to exist.

For example, as shown in FIG.
In consideration of the separation distance between 0 and the object 9 other than the target object and the size of the target object 10, the size of the calculation target frame F related to the target image area in which the barycentric coordinates are calculated with respect to the image measurement unit 3 in advance. Is fixed. This is because the target object 10 and the object 9 outside the target object can be separated by setting the calculation target frame F in a certain range.

Thus, the region outside the calculation target frame F is not included in the barycentric coordinate calculation, and the calculation is executed only within that frame, whereby the calculation time can be shortened. next,
In step P4, for the inside of the calculation target frame F, black is weighted with "1" and white is weighted with "0" to obtain the barycentric coordinate values, and thereafter, in step P5, the barycentric coordinates previously obtained are determined. Positioning is performed as the center of gravity position of.

[0017]

By the way, according to the conventional example, in order to speed up the calculation process of the barycentric coordinate value of the target image in the image measuring unit 3, the color of the target object 10 is set to black in advance and its periphery is set in advance. It is assumed that the discrimination condition that the background color of is white. Further, the barycentric coordinate value is calculated based on the weighting of the target image such that black = “1” and white = “0”.

Therefore, when the discrimination condition that the color of the object 10 is black or white and the surrounding background color is white or black is not assumed, that is, the object 10 of an unknown color is recognized, it is recognized. If there is a request to do so, the image processing becomes difficult.

For example, in the image acquisition state (subject) in which the color of the object 10 is black and the surrounding background color is white, if the identification condition is not assumed in the image processing unit 3, In step P4 of FIG. 9B, the target 1
Since the image processing is performed with the color of 0 being white = “0” and the background color of the periphery being black = “1”, the correct barycentric coordinate calculation processing of the target image cannot be performed. For this reason,
In step P5, the barycentric position of the target object 10 may be determined by the incorrect barycentric coordinate value of the target image.

Further, when the image of the object 10 and the object 9 other than the object are included in the calculation target frame F as shown in FIG. Even in the calculation of the value, since the calculation process has a limit and its accuracy is lacking, the calculated barycentric coordinate is the object 10.
May not match the center of gravity position of.

This is because the difference in area is smaller than the error in the coordinate value of the center of gravity, which makes it difficult to reliably detect the object 10 from the difference in area. That is, in the method in which the calculation target frame F is fixed, even when the calculation target frame F includes only a part of the non-target object 9 in FIG. Since the distance from the barycentric coordinates of the object 10 to the object 9 is a long component, the error of the calculated value of the barycentric coordinates becomes large, which greatly deviates from the true barycentric coordinate position of the object 10. ..

As a result, the calculated area value of the object 10 in the calculation object frame differs from the actual area value of the object 11 that is scheduled, so that measurement is impossible or the acquisition area of the television camera 2 is erroneous, that is, It is processed as if the displacement of the robot hand 1 to which is attached is too large.

In such a case, the center of the calculation target frame F may happen to coincide with the calculated barycentric coordinate value of the internal screen. However, in the calculated barycentric coordinates including the pixels (noise image) of the object 9 other than the target object, the true calculation target frame F
The center of does not always match the image centroid coordinates of the object 10.

In this state, when the control system mistakenly recognizes the position of the center of gravity of the object 10 and shifts to the teaching work thereafter, the robot hand 1 collides with the object 10 or falls. Control error occurs.

As a result, the color of the object 10, that is,
If black or white is not clear in advance, the image measuring unit 3 may calculate an incorrect barycentric coordinate value.
This leads to a decrease in reliability of image processing. Further, when the gravity center coordinate value cannot be measured, the television camera 2 needs to be moved again, that is, the robot hand 1 needs to be moved again, which hinders the speeding up of the positioning process.

The present invention was created in view of the problems of the conventional example, and automatically distinguishes the color of the object even if black or white is not clear in advance. Thus, it is an object of the present invention to provide an image processing device and a robot control device that can accurately calculate measurement information of an object and recognize the object at high speed.

[0027]

1 and 2 show principle diagrams (Nos. 1 and 2) of an image processing apparatus and a robot control apparatus according to the present invention, respectively.

The first image processing apparatus of the present invention is shown in FIG.
As shown in FIG. 2A, the range setting means 11 for setting the calculation target frame F in the image acquisition target 16 and the number of white pixels Wi [i on the calculation target frame set in the image acquisition target 16 Wi [i
= 1, 2, 3 ...] or the number of black pixels Bj [j = 1, 2, 3
, And a comparing means 1 for comparing the number of white pixels Wi and the number of black pixels Bj on the calculation target frame F.
3 and a calculation unit 14 for calculating the barycentric coordinate value of the target image in the calculation target frame F, and the calculation is performed based on the number of white pixels Wi and the number of black pixels Bj on the calculation target frame F. The color tone of the image acquisition target 16 in the target frame F is identified.

In the first image processing apparatus of the present invention, the comparison means 13 determines the number of white pixels Wi and the number of black pixels Bj on the calculation target frame F as shown in FIG. 2B. It is characterized by comparing.

Further, the second image processing apparatus of the present invention is
In the first image processing apparatus, as shown in FIG. 1A, the comparing unit 13 sets the determination reference value DR of the preset white pixel number Wi or the black pixel number Bj and the calculation target frame. It is characterized in that the number of white pixels Wi on F and the number of black pixels Bj are compared.

The third image processing apparatus of the present invention is the same as the first and second image processing apparatus, wherein the preset reference value DR of the number of white pixels Wi or the number of black pixels Bj and the calculation are set. Warning means 15 for generating a measurement abnormality based on a comparison process with the number of white pixels Wi and the number of black pixels Bj on the target frame F
Is provided.

In the third image processing apparatus according to the present invention, the warning means 15 sets the preset number of white pixels W.
First comparison processing between the judgment reference value DR of i and the number of white pixels Wi on the calculation target frame F, and the judgment reference value DR of the number of black pixels Bj and the number of black pixels on the calculation target frame F It is characterized in that a measurement abnormality is generated based on the second comparison process with Bj.

Further, the robot controller of the present invention, as shown in FIG. 1 (b), has a robot working unit 17 for performing various teaching operations related to the object 10 and an image acquisition for acquiring an image of the object 10. And a control means 20 for controlling the input and output of the robot working unit 17, the image acquisition means 18, and the image measurement means 19, and the image measurement means 19 for measuring the position information of the object 10. The image measuring means 19 comprises the first to third image processing devices of the present invention, and achieves the above object.

[0034]

[Operation] According to the first image processing apparatus of the present invention, as shown in FIG.
As shown in (a), range setting means 11 and counting means 1
2. The comparison means 13 and the calculation means 14 are provided, and the comparison means 13 compares the number Wi of white pixels and the number Bj of black pixels on the calculation target frame F as shown in FIG. 2B.

For example, when the calculation target frame F is set to the image acquisition target 16 expanded in the memory area by the range setting means 11, the number of white pixels on the calculation target frame set in the image acquisition target 16 is set. Wi [i = 1, 2, 3 ...] Or the number of black pixels Bj [j = 1, 2, 3 ...] is counted by the counting means 12. Further, the comparison unit 13 compares the number of white pixels Wi and the number of black pixels Bj on the calculation target frame.

At this time, the comparison means 13 is used to
As shown in (b), the number of white pixels Wi on the calculation target frame F
And the number of black pixels Bj are compared. For example, when the number of white pixels Wi is greater than the number of black pixels Bj, the target object 10
Is black, and the image acquisition target 16 in the calculation target frame F is
It is possible to automatically identify the color tone of.

On the contrary, when the number of black pixels Bi is larger than the number of white pixels Wi, the color of the object 10 is white,
It is possible to automatically identify the color tone of the image acquisition target 16 in the calculation target frame F. Accordingly, the barycentric coordinate value of the target image in the calculation target frame F is calculated by the calculation means 14 based on the automatic identification processing.

Therefore, in the case where the discrimination condition that the color of the object 10 is black or white and the surrounding background color is white or black is not set (assumed), that is, the object 10 is an unknown color, Even if there is a request to recognize it, it is possible to easily execute the color tone identification processing of the image acquisition target 16.

As a result, it becomes possible to perform the correct arithmetic processing of the barycentric coordinate value of the target image, and it becomes possible to determine the barycentric position of the object 10 by the accurate barycentric coordinate value of the target image.

Further, according to the second image processing apparatus of the present invention, in the first image processing apparatus, the comparing means 13 shown in FIG. 1A has the preset number of white pixels Wi or black. The determination reference value DR of the number of pixels Bj is compared with the number of white pixels Wi or the number of black pixels Bj on the calculation target frame F.

Therefore, with respect to the unknown pixel on the calculation target frame F, the number of pixels of the color opposite to the pixel of the color of the target image in the calculation target frame F is counted, and the number of pixels is the judgment reference value. DR, for example, zero or less than a certain allowable number, is “good”, otherwise “bad”, that is, a part of the object 9 other than the target is within the calculation target frame F. It is possible to make an abnormality determination as if the object 10 overlaps the target image or the object 10 itself is out of the calculation target frame F.

As a result, even if the color of the object 10 is known, if the abnormality is determined, the process ends without performing arithmetic processing, and in other cases, the calculation target frame F The barycentric coordinate value of the target image within is calculated by the calculating means 14.

As a result, like the first image processing apparatus, it is possible to carry out the arithmetic processing of the correct barycentric coordinate value of the target image, and the barycentric position of the object 10 is determined by the accurate barycentric coordinate value of the target image. It becomes possible.

Further, according to the third image processing apparatus of the present invention, in the first and second image processing apparatuses, the judgment reference value DR of the preset white pixel number Wi or the black pixel number Bj is set.
And an alarm means 15 for generating a measurement abnormality based on a comparison process between the number of white pixels Wi and the number of black pixels Bj on the calculation target frame F.
Is provided.

Therefore, with respect to the unknown pixel on the calculation target frame F, the counting result of the number of white pixels Wi and the number of black pixels Bj on the calculation target frame F is set to the preset number of white pixels Wi or black. When the determination reference value DR of the number of pixels Bj is not satisfied,
The alarm means 15 issues an alarm or the like indicating that the measurement of the barycentric coordinate value has failed.

At this time, for example, the first comparison processing between the preset reference value DR of the number of white pixels Wi and the number of white pixels Wi on the calculation target frame F and the determination of the number of black pixels Bj are performed. A measurement abnormality is generated by the alarm means 15 based on the second comparison process of the reference value DR and the number of black pixels Bj on the calculation target frame F.

As a result, it becomes possible to notify the upper control system of the image processing apparatus and prevent a control error or the like in advance. According to the robot controller of the present invention,
As shown in FIG. 1B, a robot working unit 17, an image acquisition unit 18, an image measuring unit 19, and a control unit 20 are provided, and the image measuring unit 19 is composed of first to third image processing devices. There is.

For example, when performing various teaching operations related to the object 10, the robot working unit 1 is controlled via the control means 20.
7 is moved onto the area of the object 10, and the image of the object 10 is acquired by the image acquisition means 18. Further, the position information of the object 10 is measured by the image measuring means 19. At this time, the color tone of the image acquisition target 16 in the calculation target frame F is automatically identified by the image measuring unit 19 including the first to third image processing devices of the present invention.

Therefore, even if black or white is not clearly defined in advance for the color of the object 10, unknown pixels on the frame F to be calculated are subject to image acquisition in the frame F to be calculated. By adopting an image processing apparatus having a function of automatically identifying 16 color tones, it is possible to accurately calculate the barycentric coordinate value of the object 10.

As a result, unlike the conventional example, it is possible to accurately determine the measurement information of the object by automatically discriminating the color of the object 10 without assuming black or white in advance. It is possible to recognize the object 10 at high speed.

[0051]

Embodiments Next, embodiments of the present invention will be described with reference to the drawings. 3 to 8 are diagrams illustrating an image processing device and a robot control device according to an embodiment of the present invention.

(1) Description of First Embodiment FIG. 3 is a block diagram of a robot controller to which an image processing apparatus according to each embodiment of the present invention is applied, and FIG. 4 is an image for recognizing an object thereof. Each of the processing flowcharts is shown.

For example, in FIG. 3, a robot control device in which the respective functions of the first to third image processing devices of the present invention are combined and applied are shown in FIG. 3 as a robot hand 27, a television camera 28, an image measuring system 29 and a CPU. (Central processing unit) 30 and drive control unit 31 and the like.

That is, the robot hand 27 is one embodiment of the robot working unit 17, and is for carrying out various teaching work in relation to the object 10. For example, the object 10 is gripped.

The television camera 28 is an embodiment of the image acquisition means 18, and acquires an image of the object 10 and outputs its analog video signal S to the image processing system 29. For example, the TV camera 28 is the robot hand 27.
It is fixed to the side of the work area or installed on the ceiling surface of the work area.

The image measuring system 29 is an image measuring means 19
This is an example of measuring the position information of the object 10. The image measuring system 29 is the first aspect of the present invention.
~ It comprises a third image processing device.

For example, the image measuring system 29 has the calculation frame setting editor 21 and the pixel counting editor 2 connected to the data bus 34 as shown in the portion surrounded by the one-dot chain line in FIG.
2, a data comparison unit 23, a calculation unit 24, an alarm unit 25,
The signal processing unit 32 and the memory unit 33 are included.

That is, the calculation frame setting editor 21 is an embodiment of the range setting means 11 and sets the calculation target frame F to the image acquisition target 16. Pixel counting editor 2
Reference numeral 2 is an embodiment of the counting means 12, and an image acquisition target 16
Number of white pixels on the calculation target frame F set to
1, 2, 3 ...] or the number of black pixels Bj [j = 1, 2, 3
…] Are counted.

The data comparison unit 23 is an embodiment of the comparison means 13 and compares the number of white pixels Wi and the number of black pixels Bj on the calculation target frame F. For example, the data comparison unit 2
As shown in FIG. 2B, 3 directly compares the number Wi of white pixels and the number Bj of black pixels on the calculation target frame F. The function constitutes the first image processing apparatus of the present invention. In addition, the data comparison unit 23 determines that the preset number of white pixels W
i or the judgment reference value DR of the number of black pixels Bj and the calculation target frame F
The number of white pixels Wi and the number of black pixels Bj are compared. The second image processing device of the present invention is configured by the function.

The calculation section 24 is an embodiment of the calculation means 14 and calculates the barycentric coordinate value of the target image in the calculation target frame F. The alarm unit 25 is one embodiment of the alarm means 15, and includes a preset reference value DR of the number of white pixels Wi or the number of black pixels Bj set in advance and the number of white pixels Wi or black of the calculation target frame F. The measurement abnormality is generated based on the comparison process with the number of pixels Bj. For example, the alarm unit 25 performs the first comparison process between the preset reference value DR of the number of white pixels Wi and the number of white pixels Wi on the calculation target frame F and the determination reference value of the number of black pixels Bj. A measurement abnormality is generated based on the second comparison processing between DR and the number of black pixels Bj on the calculation target frame F. As a result, when the object 10 cannot be recognized or the recognition process cannot be performed, the alarm data Da is generated.
Based on the above, the image measurement system 29 can directly issue an alarm to the outside, and the third image processing apparatus of the present invention is configured.

Further, the signal processing unit 32 is the television camera 28.
The analog video signal S output from the binarization circuit is binarized, for example, it is composed of an A / D conversion circuit and its threshold circuit, and as a result, image data Din relating to black and white gradation is obtained.

The memory unit 33 temporarily stores the image data Din, sets the calculation target frame F, counts pixels, and various control data relating to an alarm regarding unmeasurable and a determination reference value DR. ..

As a result, in the image measuring system 29, the number of white pixels Wi and the number of black pixels B on the calculation target frame F are calculated.
The color tone of the image acquisition target 16 in the calculation target frame can be automatically identified based on j.

The CPU 30 is an embodiment of the control means 20 and controls the input / output of the robot hand 27, the television camera 28 and the image measuring system 29.
For example, the drive control data Dm is output to the drive control unit 31, and the alarm data Da is output to the alarm unit 35. The drive control section 31 controls the drive of the robot hand 27 in the X, Y, and Z directions based on the drive control data Dm.

In this way, according to the robot control device to which the image processing device according to each embodiment of the present invention is applied,
As shown in FIG. 3, a robot frame 27, a television camera 28, an image processing system 29, a CPU 30, and a drive control unit 31 are provided, and the image processing system 29 is connected to a data bus 34. It comprises a counting editor 22, a data comparing section 23, a computing section 24, a signal processing section 32, a memory section 33 and the like.

For example, when performing various teaching operations related to the object 10, the robot hand 27 is executed via the CPU 30.
Is moved onto the area of the object 10, and an image of the object 10 is acquired by the television camera 28. Further, the position information of the object 10 is measured by the image processing system 29.

At this time, the calculation target frame F is calculated by the image processing system 29 including the first to third image processing devices of the present invention.
The color tone of the image acquisition target 16 in the calculation target frame is automatically identified based on the number of white pixels Wi and the number of black pixels Bj. For example, the calculation target frame F is set by the calculation frame setting editor 21 in the image acquisition target 16 expanded on the area of the memory unit 32.

Further, the number of white pixels Wi [i = 1, 2, 3 ...] or the number of black pixels Bj [j = 1, 2, 3, ...] On the calculation target frame set as the image acquisition target 16. Are counted by the pixel counting editor 22, and the number of white pixels W on the calculation target frame F is calculated.
The data comparison unit 23 compares i and the number of black pixels Bj.

Therefore, even if black or white is not clear in advance for the color of the object 10, unknown pixels on the frame F to be calculated are subject to image acquisition in the frame F to be calculated. By adopting the image processing device for automatically identifying the 16 color tones, the accurate barycentric coordinate value of the object 10 can be calculated at high speed.

This makes it possible to speed up the positioning process of the robot working unit 17. In addition, erroneous recognition of the position of the center of gravity of the object 10 in the control system becomes almost zero, and a control error such as colliding the robot working unit 11 with the object 10 or causing it to tip over is avoided during subsequent teaching work. It becomes possible to prevent it as much as possible.

Next, a robot control apparatus for recognizing an object and gripping the object according to each embodiment of the present invention will be described while supplementing the operation of the image measuring system 29 of the apparatus.

FIG. 4 is a flow chart of image processing for recognizing an object according to the first embodiment of the present invention, and FIG. 5 is a supplementary explanatory view of the flow chart. For example, recognizing the object 10 as shown in FIG.
When gripping it, first in step P in FIG.
At 1, the robot hand 27 is moved onto the area of the object 10. At this time, when the drive control data Dm is output from the CPU 30 to the drive control unit 31, the drive control unit 31 causes the drive unit 31 to move the X, X, and X based on the control data Dm.
Drive control is performed in the Y and Z directions. As a result, the television camera 28 fixed to the side surface of the robot hand 27 is moved and moved onto the area of the object 10.

Next, in step P2, an image acquisition process of the object 10 under the attachment area of the television camera 28 is performed. At this time, the image of the object 10 is acquired by the television camera 28, and the analog video signal S is acquired by the image processing system 29.
Is output to.

Then, in step P3, the camera input image is binarized by an appropriate level. At this time, the signal processing unit 32 binarizes the analog video signal S output from the television camera 28, and outputs image data Din related to the black and white gradation. This image data Din is stored in the memory unit 3
Temporarily stored in 3.

For example, in FIG. 5A, when a monitor display is performed based on the binarized image data Din,
An image acquisition target 16 including an image of the object 10 and an image of the object 9 other than the object is obtained.

Further, in step P4, the calculation target frame F is set at the position where the object 10 is most likely to exist. At this time, the calculation frame setting editor 21 of the image processing system 29 sets the calculation target frame F in the image acquisition target 16 as shown in FIG. 5B via the CPU 30.

Next, in step P5, the number of white pixels Wi and the number of black pixels Bj of the pixels in the calculation target frame F are counted. At this time, the number of white pixels Wi on the calculation target frame F set as the image acquisition target 16 by the pixel counting editor 22
[I = 1, 2, 3 ...] or the number of black pixels Bj [j = 1,
2, 3, ...] are counted.

After that, in step P6, the number of white pixels Wi and the number of black pixels Wj of the pixels in the calculation target frame F are compared. At this time, if the number of white pixels Wi is larger than the number of black pixels Bj, or if the number of white pixels Wi and the number of black pixels W
If j is equal (YES), the process proceeds to step P7.

If the number of white pixels Wi is less than the number of black pixels Bj, that is, if the number of black pixels Bj is more than the number of white pixels Wi (NO), the process proceeds to step P8. Regarding the pixels on the calculation target frame F, for example, the number of white pixels Wi and the number of black pixels Wj are directly compared by the data comparison unit 23 based on the image data Din developed in the memory unit 33. (See FIG. 2B).

Here, for the pixels on the calculation target frame F,
When the number of white pixels Wi is greater than the number of black pixels Bj (YE
In step S7, black pixels are weighted by "1" and white pixels are weighted by "0" for the inside of the calculation target frame F in step P7 to obtain the barycentric coordinates Xg, Yg of the target image. At this time, black, which is seen as the color of the object 10, is weighted with “1” and white is weighted with “0” based on the image data Din, and (1) and (2) are assigned to the X-axis direction and the Y-axis direction, respectively. ) Formula is used to calculate the average.

[0081]

[Equation 1]

As a result, the object 1
The barycentric coordinates Xg and Yg of 0 can be obtained. Here, fi is “1” when the i-th pixel is black, “0” when it is white, xi, yi are the coordinate values of the X and Y axes of the pixel, and n is the total number of pixels in the screen. Is represented. The value of the denominator represents the number of pixels of the object, that is, the area.

When the number of white pixels Wi of the pixels in the calculation target frame F is smaller than the number of black pixels Bj in step P6 (NO), the process proceeds to step P8, and the calculation target frame F is calculated. With respect to the inside, the black pixel is weighted with "0", the white pixel is weighted with "1", and the barycentric coordinate X of the target image is weighted.
Find g and Yg. At this time, the barycentric coordinates Xg, Yg of the object 10 are obtained via the calculation unit 24 as in step P7.

Then, in step P9, the barycentric coordinates Xg, Yg previously obtained are recognized as the barycentric positions XG, YG of the object, and the various operations thereafter are performed. At this time, various teaching operations are performed by the robot hand 27 in relation to the object 10. For example, the robot hand 27 is operated by the television camera 28 via the drive control unit 31 based on the drive control data Dm.
Is moved in the X and Y directions by the mounting offset of, and thereafter, it is driven and controlled in the Z direction, and the object 10 is gripped.

As described above, according to the image measuring method of the robot controller for recognizing an object according to the first embodiment of the present invention, as shown in the processing flowchart of FIG. The number of white pixels Wi on F and the number of black pixels Bj are compared.

Therefore, as shown in FIG. 2B, the number of white pixels Wi on the calculation target frame F is directly compared with the number of black pixels Bj, so that the number of white pixels Wi is black. Number of pixels B
When the number is larger than j, the color of the object 10 is black, and the color tone of the image acquisition object 16 in the calculation object frame F can be automatically identified.

On the contrary, when the number of black pixels Bi is greater than the number of white pixels Wi, the color of the object 10 is white.
It is possible to automatically identify the color tone of the image acquisition target 16 in the calculation target frame F. As a result, steps P7 and P
At 8, the calculation unit 24 calculates the barycentric coordinate value of the target image in the calculation target frame F based on the automatic identification processing.

As a result, the color of the object 10 is black or white,
Even when the discrimination condition that the background color around the object is white or black is not set (assumed), that is, when the object 10 has an unknown color, the image acquisition target 16 It is possible to easily execute the color tone identification processing of.

For example, in an image acquisition state (subject) in which the actual color of the object 10 is black and the background color around the object 10 is white, the identification condition is not assumed in the image processing apparatus. Even if there is, since the number of white pixels Wi on the calculation target frame F and the number of black pixels Bj are compared by the data comparison unit 23, the color of the target object 10 as in the conventional example is determined in steps P7 and P8. White = "0", surrounding background color is black =
Erroneous image processing of "1" can be eliminated.

As a result, in step P9, the correct center-of-gravity coordinate values of the target image can be calculated, and the target object 10 can be accurately calculated by the center-of-gravity coordinates Xg and Yg.
It is possible to determine the barycentric positions XG, YG of. As a result, it is possible to improve the reliability of the image measurement processing related to the calculation of various measurement information.

(2) Description of Second Embodiment FIG. 6 is a flowchart of image processing for recognizing an object according to the second embodiment of the present invention, and FIG. 7 is a supplement of the flowchart of other embodiments. Explanatory drawing is shown respectively.

In FIG. 6, the difference from the first embodiment is that in the second embodiment, for the number of white pixels Wi on the calculation target frame F, alarm processing is performed according to the judgment reference value DR and the comparison result. Is.

That is, as in the first embodiment, the robot hand 27 is moved onto the area of the object 10 in step P1, and the image acquisition processing of the object 10 is performed in step P2. In step P2, the camera input image is binarized by an appropriate level, and in step P3, the first calculation target frame F is set at a position where the object is most likely to exist (see FIG. 7A). So far, this is the same as the first embodiment.

Next, in step P6, the number of white pixels Wi on the calculation target frame F is counted. At this time, the image acquisition target 1
The number of white pixels on the calculation target frame set to 6 Wi [i =
1, 2, 3 ...] are counted by the pixel counting editor 22.

Further, in step P7, the number Wi of white pixels on the calculation target frame F is compared with the determination reference value DR. At this time, the number of white pixels Wi on the target frame F is smaller than the determination reference value DR, or the number of white pixels Wi is less than the determination reference value D.
If it is equal to R (YES), the process proceeds to step P8. Here, the data comparison unit 23 compares the preset determination reference value DR of the number of white pixels Wi with the number of white pixels Wi in the calculation target frame F.

If the number of white pixels Wi on the target frame F is larger than the determination reference value DR (NO), the process ends abnormally. Note that the alarm unit 25 causes a measurement abnormality based on the comparison result.

Therefore, the image acquisition target 1 shown in FIG.
In the case of 6, if the number of white pixels Wi on the target frame F is smaller than the determination reference value DR in step P7 (Y
Since it corresponds to ES), the calculation target frame F is calculated in step P8.
The inside of the image is weighted with “1” for black and “0” for white, and the barycentric coordinate values Xg and Yg of the target image are obtained. At this time, as in the first embodiment, the barycentric coordinates Xg, Yg of the object 10 are obtained via the calculation unit 24.

As a result, the barycentric coordinates Xg, Yg previously obtained in step P9 are recognized as the barycentric positions XG, YG of the object 10, and various teaching operations thereafter are performed. Since the other processing is the same as that of the first embodiment, the description thereof will be omitted.

As described above, according to the image measuring method of the robot control device for recognizing the object according to the second embodiment of the present invention, as shown in the processing flowchart of FIG. 6, it is preset in step P7. The data comparison unit 23 compares the determination reference value DR of the number of white pixels Wi and the number of white pixels Wi on the calculation target frame F.

Therefore, with respect to the unknown pixel on the calculation target frame F, the number of pixels of the color opposite to the color pixel of the target image in the calculation target frame F, that is, the white pixel in the embodiment of the present invention. The number of pixels Wi is counted, and the number of pixels Wi is the judgment reference value D.
R, for example, zero or less than a certain allowable number, is “good”, otherwise “bad”, that is, a part of the object 9 other than the object is in the calculation target frame F. It is possible to end the abnormality determination as a state in which the object 10 overlaps the target image or the target 10 itself is out of the calculation target frame F.

As a result, even if the color of the object 10 is known, the abnormality determination ends without performing the calculation process. In other cases, the calculation target frame F is calculated in step P8.
The calculation of the barycentric coordinate value of the target image within the calculation unit 24 enables the calculation to be continued in the same manner as in the first embodiment.

As a result, as in the first embodiment, it is possible to carry out the arithmetic processing of the correct barycentric coordinate value of the target image, and in step P9, the accurate barycentric coordinate value Xg of the target image,
The center of gravity position XG, YG of the object 10 can be determined by Yg.

When the barycentric coordinate value cannot be measured, an alarm or the like is issued by the alarm unit 25 to inform the upper control system that an abnormality has occurred in the image measuring system, and the conventional example is used. It is possible to prevent such control mistakes in advance.

(3) Description of Third Embodiment FIG. 8 shows a flowchart of image processing for recognizing an object according to the third embodiment of the present invention.

In FIG. 8, the difference from the first and second embodiments is that in the third embodiment, the preset number of white pixels W
i and the judgment reference value DR of the number of black pixels Bj and the calculation target frame F
Based on the comparison processing between the number of white pixels Wi and the number of black pixels Bj above, the alarm unit 25 terminates the measurement abnormally.

That is, in FIG. 8, in step P1, the robot hand 27 is moved to the area of the object 10 in the same manner as in the first and second embodiments, and in step P2, the image acquisition process for the object 10 is performed. do. Further, in step P3, the camera input image is binarized by an appropriate level, and in step P4, a calculation target frame F is set at a position where an object is most likely. Here, the calculation target frame F is set as the image acquisition target 16 as shown in FIG.

Then, in step P5, as in the first embodiment, the number of white pixels Wi and the number of black pixels Bj of the pixels on the calculation target frame F are counted. At this time, the number of white pixels Wi [i = 1, 2, 3 ...] or the number of black pixels Bj [j = 1, 2] on the calculation target frame F set as the image acquisition target 16 by the pixel counting editor 22. , 3 ...] are counted.

Thereafter, in step P6, the number of white pixels Wi and the number of black pixels Wj of the pixels in the calculation target frame F are compared. At this time, if the number of white pixels Wi is larger than the number of black pixels Bj, or if the number of white pixels Wi and the number of black pixels W
If j is equal (YES), the process proceeds to step P7.

If the number of white pixels Wi is smaller than the number of black pixels Bj, that is, if the number of black pixels Bj is larger than the number of white pixels Wi (NO), the process proceeds to step P8. Regarding the pixels on the calculation target frame F, for example, the number of white pixels Wi and the number of black pixels Wj are directly compared by the data comparison unit 23 based on the image data Din developed in the memory unit 33. (See FIG. 2B).

Here, for the pixels on the calculation target frame F,
When the number of white pixels Wi is greater than the number of black pixels Bj (YE
In step S7, the number of white pixels Wi on the calculation target frame F is compared with the determination reference value DR in step P7. At this time, the number of white pixels Wi on the target frame F is equal to the determination reference value D.
If it is smaller than R or is equal to the judgment reference value DR (YES), the process proceeds to step P8. here,
The number of white pixels W preset by the data comparison unit 23
The determination reference value DR of i and the number of white pixels Wi on the calculation target frame F
And are compared (first comparison processing).

If the number of white pixels Wi on the target frame F is larger than the determination reference value DR (NO), the process ends abnormally. Note that the alarm unit 25 causes a measurement abnormality based on the comparison result.

Therefore, the image acquisition target 1 shown in FIG.
In the case of 6, the case corresponds to the case where the number of black pixels Bj on the target frame F is smaller than the determination reference value DR (YES) at step P7, and therefore the inside of the calculation target frame F at step P8. Are weighted with "1" for black and "0" for white, and the barycentric coordinate values Xg, Yg of the target image are obtained. At this time, as in the first embodiment, the barycentric coordinates Xg, Yg of the object 10 are obtained via the calculation unit 24.

If the number of black pixels Bj of the pixels in the calculation target frame F is larger than the number of white pixels Wi in step P6 (NO), the black pixels in the calculation target frame F are calculated in step P9. The number of pixels Bj is compared with the determination reference value DR. At this time, if the number of black pixels Bj on the target frame F is smaller than the determination reference value DR or if it is equal to the determination reference value DR (YES), the process proceeds to step P10. Here, the data comparison unit 23 compares the preset reference value DR of the number of black pixels Bj with the number of black pixels Bj on the calculation target frame F (second comparison processing).

If the number of black pixels Bj on the target frame F is larger than the determination reference value DR (NO), the abnormal termination is performed (see FIG. 7B). Note that the alarm unit 25 causes a measurement abnormality based on the comparison result.

Therefore, the image acquisition target 1 shown in FIG.
In the case of 6, the case corresponds to the case where the number of white pixels Wi on the target frame F is smaller than the determination reference value DR (YES) in step P10. Therefore, in step P10, the inside of the calculation target frame F is determined. Are weighted with "1" for white and "0" for black to obtain the barycentric coordinate values Xg, Yg of the target image. At this time, as in the first embodiment, the barycentric coordinates Xg, Yg of the object 10 are obtained via the calculation unit 24.

As a result, the barycentric coordinate values Xg, Yg previously obtained in step P11 are set to the barycentric position XG, YG of the object 10.
Recognize as, and perform various work. Since the other structures are similar to those of the first and second embodiments, the description thereof will be omitted.

As described above, according to the image measuring method of the robot controller for recognizing an object according to the third embodiment of the present invention, as shown in the processing flowchart of FIG. Based on the comparison processing of the number of white pixels Wi on F and the number of black pixels Bj, steps P7 and P9 are performed.
In the comparison processing of the judgment reference value DR of the number of white pixels Wi or the number of black pixels Bj set in advance and the number of white pixels Wi or the number of black pixels Bj on the calculation target frame F (first, second The measurement is abnormally terminated based on the comparison process).

Therefore, with respect to the unknown pixel on the calculation target frame F, the count result of the number of white pixels Wi and the number of black pixels Bj on the calculation target frame F is set to the preset number of white pixels Wi or black. When the determination reference value DR of the number of pixels Bj is not satisfied,
When the measurement ends abnormally, the alarm unit 25 issues a warning or the like indicating that the gravity center coordinate value cannot be measured.

At this time, the preset number of white pixels Wi
Of the white reference pixel number Wi on the calculation target frame F and the reference value DR of the black pixel number Bj
The measurement error is generated by the alarm unit 25 on the basis of the second comparison processing between the number of black pixels Bj on the calculation target frame F and Bj.

As a result, it is possible to notify the upper control system of the image processing apparatus and prevent a control error or the like in advance. As a result, like the first and second embodiments,
It is possible to accurately calculate the measurement information of the object 10.

Although the embodiment in which the image processing apparatus of the present invention is applied to the eyes of the robot control apparatus has been described, the same effect can be obtained when applied to pattern recognition processing and other image processing. Further, although the calculation target frame F has been described as having a quadrangular shape, it can be similarly applied to an arbitrary shape such as a circle or an ellipse.

Further, in each of the embodiments, the barycentric coordinates and the area are described as the measured values. Involved calculation processing may be performed.

In each of the embodiments, the case of the measurement processing relating to the X-axis and Y-axis two-dimensional images has been described, but it is also applicable to the case of the measurement processing relating to a stereoscopic image such as a CT scanner.

[0124]

As described above, according to the image processing apparatus of the present invention, the range setting means, the counting means, the comparing means and the calculating means are provided, and the comparing means includes the number of white pixels on the calculation target frame. And the number of black pixels are compared.

Therefore, the comparison means compares the number of white pixels and the number of black pixels on the calculation target frame, and when the number of white pixels is larger than the number of black pixels, the color of the object is black. As
On the contrary, when the number of black pixels is larger than the number of white pixels, the color of the object is white, and it is possible to automatically identify the color tone of the image acquisition target in the target frame. With this,
It is possible to easily recognize the case where the color of the object is black or white and the background color around the object is an unknown color for which the identification condition is not set and which is not set.

Further, according to the image processing apparatus of the present invention, the preset reference value of the number of white pixels or the number of black pixels is compared with the number of white pixels or the number of black pixels on the calculation target frame. Compared by means.

Therefore, for an unknown pixel on the calculation target frame, a part of the object other than the target object overlaps the target image in the calculation target frame, or the target object itself protrudes from the calculation target frame. It is possible to make an abnormality determination as the state of being present.

Further, according to the image processing apparatus of the present invention, the comparison processing of the preset reference value of the number of white pixels or the number of black pixels and the number of white pixels or the number of black pixels on the calculation target frame is performed. An alarm means is provided for generating a measurement abnormality based on the above.

Therefore, with respect to the unknown pixel on the calculation target frame, the counting result of the number of white pixels and the number of black pixels on the calculation target frame is a preset judgment criterion of the number of white pixels or the number of black pixels. If the value is not satisfied, the alarm means issues an alarm indicating that the barycentric coordinate value cannot be measured. This informs the upper control system of the image processing device,
It is possible to prevent control mistakes in advance.

As a result, it becomes possible to perform the arithmetic processing of the correct barycentric coordinate value of the target image, and it becomes possible to determine the barycentric position of the target object by the accurate barycentric coordinate value of the target image.

Further, according to the robot control device of the present invention, the robot working unit, the image acquisition means, the image measurement means and the control means are provided, and the image measurement means comprises the image processing device of the present invention.

Therefore, even if black or white is not clearly defined in advance for the color of the object, the color tone of the image acquisition object in the object frame is calculated for the unknown pixel on the object frame. Since it can be automatically identified, the accurate barycentric coordinate value of the object can be calculated. From this, unlike the conventional example, it is possible to accurately calculate the measurement information and to recognize at high speed without previously assuming black or white depending on the color of the object. Become.

This greatly contributes to the provision of a highly reliable image measuring device and a robot controller having a high-performance object recognition function to which the image measuring device is applied.

[Brief description of drawings]

FIG. 1 is a principle diagram (1) of an image processing apparatus and a robot control apparatus according to the present invention.

FIG. 2 is a principle diagram (No. 2) of the image processing apparatus and the robot control apparatus according to the present invention.

FIG. 3 is a configuration diagram of a robot control device to which an image processing device according to an embodiment of the present invention is applied.

FIG. 4 is a flowchart of image processing for recognizing an object according to the first embodiment of the present invention.

FIG. 5 is a supplementary explanatory diagram of the flowchart according to the first embodiment of the present invention.

FIG. 6 is a flowchart of image processing for recognizing an object according to the second embodiment of the present invention.

FIG. 7 is a supplementary explanatory diagram of a flowchart according to another embodiment of the present invention.

FIG. 8 is a flowchart of image processing for recognizing an object according to the third embodiment of the present invention.

FIG. 9 is an explanatory diagram of an object recognition method of a robot controller according to a conventional example.

FIG. 10 is an acquired image view of a target object for explaining a problem in the conventional example.

[Explanation of symbols]

 11 ... Range setting means, 12 ... Counting means, 13 ... Comparison means, 14 ... Calculation means, 15 ... Alarm means, 17 ... Robot working unit, 18 ... Image acquisition means, 19 ... Image measurement means, 20 ... Control means, F ... Calculation target frame, DR ... Judgment reference value, Wi [i = 1, 2, 3 ...] ... White pixel, Bj [j = 1, 2, 3 ...] ... Black pixel.

Claims (6)

[Claims] 1. A range setting means (11) for setting a calculation target frame (F) to an image acquisition target (16), and a number of white pixels on the calculation target frame set to the image acquisition target (16) ( Wi [i = 1, 2, 3 ...]) or the number of black pixels (Bj
[J = 1, 2, 3 ...]) counting means (12)
And a comparison means (13) for comparing the number of white pixels (Wi) and the number of black pixels (Bj) on the calculation target frame (F), and the barycentric coordinates of the target image in the calculation target frame (F). A calculation means (14) for calculating a value, and based on the number of white pixels (Wi) and the number of black pixels (Bj) on the calculation target frame (F), An image processing device characterized by identifying a color tone of an image acquisition target (16). 2. The image processing apparatus according to claim 1,
The image processing apparatus, wherein the comparing means (13) compares the number of white pixels (Wi) and the number of black pixels (Bj) on the calculation target frame (F). 3. The image processing apparatus according to claim 1,
The comparison means (13) determines a preset reference value (DR) of the number of white pixels (Wi) or the number of black pixels (Bj).
And an image processing apparatus for comparing the number of white pixels (Wi) and the number of black pixels (Bj) on the calculation target frame (F). 4. The image processing apparatus according to claim 3,
The number of white pixels (Wi) or the number of black pixels (B
An alarm means (15) for generating a measurement abnormality based on a comparison process between the judgment reference value (DR) of j) and the number of white pixels (Wi) or the number of black pixels (Bj) on the calculation target frame (F). ) Is provided. 5. The image processing apparatus according to claim 4,
The alarm means (15) sets a judgment reference value (DR) of a preset number of white pixels (Wi) and the calculation target frame (F).
First comparison processing with the number of white pixels (Wi) above, and a determination reference value (DR) of the number of black pixels (Bj) and the number of black pixels (Bj) on the calculation target frame (F). An image processing apparatus, wherein a measurement abnormality is generated based on the second comparison processing of 1. 6. A robot working unit (17) for performing various teaching operations relating to an object (10), an image acquisition means (18) for acquiring an image of the object (10), and the object (10). Image measuring means (19) for measuring position information of
And the robot working unit (17), image acquisition means (1
8) and a control means (20) for controlling the input / output of the image measuring means (19), wherein the image measuring means (19) comprises the image processing device according to any one of claims 1 to 5. Robot controller.