JPH01134573A - Image processing method - Google Patents

Abstract

PURPOSE:To shorten image processing time without deteriorating detecting resolution by processing the total of a picture with rough resolution at first and then processing an objective partial area with fine resolution. CONSTITUTION:An industrial TV camera 14 photographs bolts 11 in a box 10 from above and sends these video signals to a processor 15. The processor 15 applies A/D conversion to these video signals and stores the image data of fine resolution in a 1st memory 16 and the picture data of rough resolution in a 2nd memory 17 respectively. Then the processor 15 detects the contours of the bolts 11 in the box 10 based on the image data stored in the memory 17 and sets a partial area including the bolt 11 set at the highest position. Then the processor 15 reads the image corresponding to said partial area out of the memory 16 to obtain the attitude and the centroid position of the bolt 11 and sends a control signal to a robot controller 20.

Description

[Detailed Description of the Invention] Technical Field The present invention detects bolts in real time when, for example, an industrial robot takes out the bolts from a bin.
The present invention relates to an image processing method used to identify bolts to be removed.

BACKGROUND ART Typical prior art involves imaging bolts stacked in a bin using an industrial television camera, processing the image data to identify the bolt located at the top, and then identifying the bolt located at the top. There is one that determines the posture and center of gravity of a bolt and uses an industrial robot to extract the bolt.

In this prior art, in order to identify the bolt located at the top, image processing is performed on all pixels of the imaged screen at a fine resolution, and therefore a lot of time is required for the calculation process.

Therefore, the area that should actually be processed in order to identify the bolt is a part of the screen, and a region of interest that includes the bolt is set in advance in the screen, and the pixels within this region of interest are processed. There is a method to shorten the calculation processing time by doing the following.

As shown in FIG. 11, the field of view 33 of an industrial television camera is
A fixed region of interest 35 is set for this region of interest 3.
According to this prior art technique, in which image processing is performed only on pixels within 5 pixels, the image processing time can be shortened without lowering the detection resolution, but there are problems described below.

Problems to be Solved by the Invention In the prior art shown in FIG. 11, since the positional relationship between the industrial television camera and the object to be imaged is constant, the object is detected only when the entire port 36 is imaged within the region of interest 35. Image processing can be performed on bolts. Therefore, the first
As shown in FIG. 2, a portion of the bolt 37 is located in the region of interest 35.
If the area is included in the area, image processing is performed on the included part, but image processing is not performed on the part that is not included. , there is a problem in that no image processing is performed on the bolt 38 at all.

An object of the present invention is to provide an image processing method that solves the above-mentioned technical problems and reduces image processing time.

Means for Solving the Problems The present invention includes a first step of processing the entire image to be processed at a coarse resolution to detect a partial area within the screen in which an object is located; This image processing method is characterized in that it includes a second step in which processing is performed at a finer resolution only when the image is imaged.

According to the present invention, image processing is first performed on the entire image at a coarse resolution to identify a target, and then image processing is performed on a partial region including the target at a fine resolution to shorten the image processing time. Plan.

Embodiment FIG. 1 is a block diagram of an embodiment of the present invention. Inside the bin 10, a large number of bolts 11, which are objects to be imaged, are randomly stored. Working end 1 of industrial robot 12
3, the bolts 11 in the bin 10 are grasped and taken out one by one, and the taken out bolts 11 are used for assembly work at another location.
In order to be able to reliably grasp and lift the bolt 11, an industrial television camera 14 images the bolt 11 from above the bin 10 to obtain image data. From this image data, the posture and center of gravity of the bolt 11 at the top of the bin 10 are calculated, and the working end 13
1 can be held.

A video signal obtained by imaging with an industrial television camera 14 is given to a processing device 15, and is sent to a first memory 16 and a second memory 16.
It is stored in the memory 17 of. The first memory 16 stores image data consisting of pixels with a fine resolution of, for example, 256 pixels vertically by 256 pixels horizontally, and the second memory 17
For example, image data consisting of pixels with a coarse resolution of 64 pixels vertically by 64 pixels horizontally is stored. Fine-resolution image data and coarse-resolution image data are obtained from a video signal captured by the camera 14 by a method described later.

The output of the processing device 15 is connected to a display device 18 realized by a cathode ray tube or the like.

The display device 18 includes the first memory 16 and the second memory 17.
It is possible to display the stored contents of the memory or the calculation results of the processing device 15.

The other output of the processing device 15 is sent to the robot controller 20 via the interface 19.

The robot control device 20 controls the arm 12 and the working end 13.
For example, when the position is controlled by oil pressure, a hydraulic control signal is generated, and as another example, when the position is controlled by a motor, a motor drive signal is generated. This results in
The robot controller 20 controls the arm 12 and the working end 13 such that the working end 13 of the arm 12 can grip a single bolt 11 in a predetermined manner.

FIG. 2 is a flowchart for explaining the operation of the embodiment shown in FIG. Moving from step n1 to step n2, the industrial television camera 14 images the bolt 11 in the bin 10 from above, and the video signal is transmitted to the processing device 1.
Sent to 5. The processing device 15 converts the video signal from analog to digital in step n3, and as described above,
The image data is transferred to the first memory 16 and stored as image data with a fine resolution of 256 pixels vertically by 256 pixels horizontally.

In step n4, the video signal is converted from analog to digital, and as described above, is transferred to and stored in the second memory 17 as image data with a coarse resolution of 64 pixels vertically by 64 pixels horizontally.

FIG. 3 is a diagram for explaining the correspondence of coordinates between fine resolution image data and coarse resolution image data. FIG. 3(1) shows image data with a fine resolution of 1256 pixels M x 256 pixels horizontally, and the area surrounded by squares corresponds to ' pixels. In addition, FIG. 3 (2) shows image data with a coarse resolution of 64 pixels vertically by 64 pixels horizontally. Since it is necessary to detect the partial area and process the fine resolution image data corresponding to the partial area, it is necessary to find the coordinates of the fine resolution pixels from the coordinates of the coarse resolution pixels. The hatched pixel 21 at coordinates (i, j) shown in FIG. 3(2) corresponds to the hatched pixel 22 at coordinates (1, J) shown in FIG. 3(1). At this time, the conversion formula from coordinates (i, j) to coordinates (I, J) is: ! =4X (i-1>+1=4i-3...(1) J=
4x (j-1)+1=4j-3...(2).

In step n5, the bolts 11 in the bin 10 are determined from the rough resolution image data stored in the second memory 17.
Detect the contour line. FIG. 4 shows a method of detecting the contour line. FIG. 4(1) shows one of the bolts 11 photographed from above the bin 10 by the industrial television camera 14. ``The control device 15 takes in the image data from the second memory 17, performs a binarization process, and obtains the binarized image shown in FIG. 4(2). This binarization process is performed by comparing the value of each pixel of the image data with a predetermined threshold value. In FIG. ” pixel,
A region 25 shown with diagonal lines is a logic "0", and thereby the contour line 26 of the bolt 11 can be detected. That is, the contour line 26 is detected by finding the change point of the logical value in FIG. 4(2). FIG. 4(3) shows the contour line 26 obtained by the above-described method, which is, for example, a pixel of logic "1", and other areas 27 are logic "0".

In step n6, the bolt 11 located at the top within the bin 10 is determined from among the contours obtained in step n5,
A rectangular partial area including the bolt 11 is set. Fifth
The figure is a diagram illustrating a method for determining the bolt 11 located at the top within the bottle 10 from the contour line determined in step n5. FIG. 5 shows one contour; other contours have been omitted. The processing device 15 scans the search line 30 to the right from the search start point 29 at the upper left of the entire processing area 28 shown in FIG. 5, and searches for pixels having logic "l". Logic "1"
”, the processing device 15 sets the pixel P1 as the tracking start point. And as shown in Figure 7,
Pixel q1. around the pixel P1. q2. . . , q8 is searched for a pixel having logic "1" in the direction shown in FIG. logic r1. When a pixel is found, the pixel is designated P2 as shown in FIG. Then, using the same method as described above, pixels around the pixel P2 are searched for a pixel having logic "1". When such a search operation is repeated and the pixel P1, which is the tracing start point, is detected, contour tracing is completed and the pixels of the contour are extracted as shown in FIG.

Next, the pixels Pi, P2, . . . obtained in the above-mentioned tracking process are
・Compare the X-direction and X-direction addresses of PI3. Find the maximum and minimum values in the X direction, the maximum and minimum values in the X direction 0: From these values, coordinates (xt + y
y), (XJI ym), (xJ, y+), and (xt, 3'+) as the four vertices is determined as the partial area 31.

In step n7, the bolt 11 obtained in step n6 is
Find the posture and center of gravity position. The processing device 15 reads fine-resolution image data corresponding to the partial area 31 determined by the Stella 7n6 from the first memory 16, determines the direction of the bolt 11 with respect to the bottle 10 as the attitude of the bolt 11, and further determines the direction of the bolt 11 as the center of gravity position. Find the centroid of the area surrounded by the contour line.

In step n8, the processing device 15 sends a control signal for the working end 13 of the arm 12 to grip the bolt 11 to the robot control device 2 via the interface 19 based on the posture and center of gravity position of the bolt 11 determined in step n7.
Send to 0.

In step n9, the robot control device 20 controls the positions of the arm 12 and the working end 13 based on the control signal from the processing device 115, grips the bolt 11, and carries it out to a predetermined location. The series of controls ends at step nlO.

9 and 101m are diagrams illustrating the resolution in the X direction and the X direction of image data captured by the industrial television camera 14. FIG. FIG. 9 shows the display device 1
8, the reference numeral 32 indicates the scanning line on the screen and its direction, the scanning line starts scanning from the upper left corner of the screen to the right, and when it reaches the right edge, returns to the left edge and starts the first scanning. Scan just below the line from left to right. Repeat this step by step,
Finally, one screen is constructed by scanning to the lower right corner of the screen.

FIG. 10 shows a video signal waveform output from the industrial television camera 14. The horizontal synchronization signal 33 is a signal for synchronizing the scanning in the X direction in FIG. 9 between the transmitting side and the receiving side. The effective width 34 corresponds to one scanning line in the X direction of the screen shown in FIG. The resolution in the X direction is
5 is determined by the sampling frequency when converting the video signal from analog to digital. Therefore, in order to set the resolution of the screen in the X direction to 256 pixels, the number of samplings in the effective width 34 should be set to 256 times.

The resolution in the X direction is determined by the number of scanning lines per screen. General NTS C (NationalTe
levision System Co import m1ttee
) type of TV camera has 52 scanning lines per screen.
There are 5 pieces. However, since the number of effective lines as a screen is 493, it can have a maximum resolution of 493.

Therefore, in order to set the resolution of the screen in the X direction to 256, it can be achieved by employing 256 of the 493 scanning lines and thinning out the other scanning lines.

In the above example, first, the screen with coarse resolution is 64 pixels vertically x 64 pixels horizontally, and the screen with fine resolution is 112 pixels.
Although 56 pixels x 256 pixels is selected, other resolutions may be selected.Also, in the binarization process for contour detection in step n5 in FIG. The value of each pixel is compared with the value, but if the bolt 11 cannot be detected, the threshold value may be changed for processing.
Even if the surrounding brightness changes, the bolt 11 can be reliably detected.

Although the above-mentioned embodiments have been described in relation to bolts, other embodiments of the present invention can be similarly implemented with respect to objects other than bolts.

Effects According to the present invention, image processing can be performed on a processing target without lowering detection resolution, and image processing time can be significantly shortened. Furthermore, since the processing device automatically determines the partial area no matter where the image processing target is captured within the screen, image processing can be performed without being constrained by the position of the target within the entire image. .

[Brief explanation of the drawing]

FIG. 1 is a block diagram of an embodiment of the present invention, FIG. 2 is a flowchart for explaining the operation of the processing device 15, and FIG.
The figure is a diagram to explain the coordinate correspondence between a screen with fine resolution and a screen with coarse resolution, Figure 4 is a diagram to explain the method of detecting contour lines, and Figure 5 is a diagram to explain the method of detecting a partial area. Figures 6 to 8 are diagrams to explain the contour tracing method, Figure 9 is a diagram showing the scanning direction of the scanning line, and Figure 10 is the video signal waveform. 11 and 12 are diagrams for explaining the prior art. DESCRIPTION OF SYMBOLS 10... Bin, 11... Bolt, 12... Arm, 13... Working end, 14... Industrial TV camera,
15... Processing device, 16... First memory, 17.
. . . second memory, 18 . . . display device, 19 . . . interface, 20 . . . robot control device, 21.
22...Pixel, 26...Contour line, 29...Search start point, 30...Search line, 31...Partial area agent Patent attorney Kei Saikyo Figure 1, ts2, Figure 3 Figure 4 Figure 5 M6 Figure 9 Figure 10

Claims (1)

[Claims] A first step of processing the entire image to be processed at a coarse resolution to detect a partial area within the screen where the object is located; and then processing only the partial area at a fine resolution. An image processing method characterized by comprising a second step of performing.