JPH011076A - Image processing device for real-time center of gravity position detection - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed Description of the Invention] <Industrial Application Field> The present invention relates to an image processing device for detecting the center of gravity position in real time, and is used, for example, to detect the center of gravity position of a workpiece by pattern recognition when gripping the workpiece in an assembly robot. be.

<Prior Art> There are two types of conventional methods for detecting the center of gravity of an object using pattern recognition.

i) Image information from a television camera is temporarily stored in an image memory, and then connected areas are determined by software, and the X and Y coordinates of significant pixels are accumulated for each connected area, and this cumulative value is used as the corresponding Divide by the area of the connected region.

ii) Based on the image information from the television camera, a dedicated circuit calculates the cumulative values of the X and Y coordinates of significant pixels and the area of the pattern (number of significant pixels), and calculates the cumulative coordinate values within the vertical synchronization period of the video signal. The center of gravity position is determined at video rate using software processing that divides by area.

<Problems to be Solved by the Invention> In the method i) described above, as the pattern area increases, the amount of data to be processed increases dramatically, which reduces the execution speed and requires a long time for calculation processing. It becomes necessary. This is fatal, for example, when gripping workpieces flowing on a production line in assembly robot sorting.

In addition, in method ii), when two or more independent patterns exist within the field of view, all the patterns are combined to find one comprehensive center of gravity position, so the relative positions of multiple workpieces are It is impossible to seek a relationship.

<Means for solving the problem> The image processing device for real-time barycenter position detection according to the present invention calculates the X coordinate of a significant pixel for each partial region in which an image plane is divided into a predetermined number of pixels a first calculation means that cumulatively adds the Y coordinate and the number of significant pixels; and a first calculation means that cumulatively adds the Y coordinate and the number of significant pixels; and a label that distinguishes between connected partial areas and unconnected partial areas in partial areas in which the area of significant pixels is equal to or larger than a predetermined value, to the partial areas; and a second calculation means for calculating the center of gravity position from the cumulative addition value of the first calculation means for each set of connected partial regions based on the label.

<Operation> The image processing device for real-time barycenter position detection according to the present invention cumulatively adds the X coordinates, Y coordinates, and number of significant pixels of significant pixels for each partial region, and also performs cumulative addition of the A labeling process is applied to each partial region to distinguish whether a certain partial region is continuous or not, and the center of gravity position is determined from the cumulative addition value for each collection of partial regions where the area of significant pixels is greater than a predetermined value based on each label. calculate.

<Example> The image processing device for detecting the center of gravity position of this example captures an input image within a plane of 256 x 256 pixels as shown in FIG. Area B
At (M, N), the cumulative value of each of the X and Y coordinates of significant pixels and the number of significant pixels are determined, and the measurement results are stored in a storage device. Within the partial area, during the period when the scanning point P (x, y) for the image passes through the partial area B (M, N), the process of accumulating the X and Y coordinates of significant pixels and counting the significant pixels is executed. do. Furthermore, 1024
Without waiting for the completion of processing of one frame consisting of (= 32 x 32) partial areas, partial area information is sequentially processed from the processed partial areas, and the center of gravity positions of each of a plurality of patterns are determined at a video rate.

FIG. 1 shows the configuration of this embodiment. In the figure, 1 is a television camera, 2.3 is an object whose center of gravity is to be determined, 4 is a synchronization separation circuit, 5 is a binarization circuit, and 7 is an S counter. , 8 is Y
Counter, 9 is the X counter.

12.16.20 is a full adder, 24 is a synchronous control circuit, 2
5 is a storage device, 26 is a CPU, and 27 is a ROM.

28 is a RAM, and 29 is a host system.

A television camera 1 captures two stationary or moving objects 2.
, 3 within its field of view, and sends a video signal mixed with a synchronization signal to a synchronization separation circuit 4. Fig. 1 shows the case of two objects as an example, but depending on the size and complexity/simplicity of the shape of the objects, even if many more objects exist within the same field of view, In this embodiment, the center of gravity position of each object can be measured in real time (video rate).

The synchronization separation circuit 4 extracts a horizontal synchronization signal HD, a vertical synchronization signal VD, and a clock signal φ from the input video signal. The video signal is sent to the binarization circuit 5. The binarization circuit 5 generates in real time a binary information series VDAT that is binarized into black and white using an appropriate darkness value such as a fixed darkness value or a floating darkness value set in advance for this video signal.

The S counter 7 counts the number of significant pixels by counting the clock VCK obtained by masking the clock for identifying pixels with the binary information series VDAT using the AND gate 6. The X counter 8 identifies the Y coordinate y of the point P by counting the horizontal synchronization signal HD. X counter 9
is the clock signal φ. By counting the
Identify the coordinate X. The full adder 12 adds the output of the X counter 8 to the cumulative value of the Y coordinate stored in the flip-flop 13 up to that point. This added value is stored in the flip-flop 13. The full adder 16 adds the output of the X counter 9 to the accumulated value of the X coordinate stored in the flip-flop 17 up to that point.

This added value is stored in flip-flop 17.

The full adder 20 adds the number of significant pixels, the cumulative value of the Y coordinate, and the cumulative value of the The cumulative value of the Y coordinate and the cumulative value of the X coordinate stored in the flip-flop 18 are added. These added values are each stored in the storage device 25.

The synchronization control circuit 24 operates in synchronization with the vertical synchronization signal VD and the horizontal synchronization signal HD, and generates an address signal for the storage device 25 and control signals necessary for each process. The storage device 25 is composed of a two-bottom RAM, and stores the number of significant pixels, the cumulative addition value of the X coordinate, and the cumulative addition value of the Y coordinate of the significant pixels.

The CPU 26 executes control of the entire system including labeling processing, calculation of the center of gravity position, and data transmission/reception with the host system 29, which will be described later.

The processing procedure will be explained below.

i) By inputting the clock VCK to the S counter 7 for counting the number of significant pixels,
The number of significant pixels, that is, the area of significant pixels in the scanning area 8M≦X≦8M+7 is counted. This counting result is stored in the flip-flop 10 by the clock CKI. After that, the S counter 7 is set to 0 by the clear pulse CL1.
It is initialized to prepare for counting the number of significant pixels in the next partial region B (M+1.N).

The data stored in the flip-flop 10 is added to the area data of the significant pixels of the partial area B (M, N) held in the storage device 25 during a period in which the scanning area is 8M+8≦X≦8M+15. The result of this addition is stored in the storage device 25.
are stored in the same area. That is, the clock CKIO
The path buffer 22 is disabled by this, and the address of the area where the area data of the partial area B (M, N) is stored is applied from the synchronization control circuit 24 to the storage device 25, and is further output to the storage device 25. An enable signal is applied. The area data output by the storage device 25 is thereby stored in the flip-flop 21 by the clock CK9. In parallel with this process, the clock CK
4 through the path buffer 11 to the flip-flop 1
The area data held by 0 is input to the full adder 20.

However, at this time, the path buffers 15 and 19 are in a disabled state. Next, the synchronization control circuit 24
Storage device 25 is brought into a data input state and path buffer 22 is enabled.

Then, the addition result of the full adder 20 is stored in the area of the storage device 25 that holds the area data of the partial area B (M, N).

1i) The Y-coordinate cumulative addition Y counter 8 is initialized to 0 by the vertical synchronizing signal VD, and then the point P is counted by counting the horizontal synchronizing signal HD.
Identify the Y coordinate y of . The flip-flop 13 is initialized to O by a clear pulse CL2 prior to measurement in the partial region B (M, N). Scan area is 8M≦X≦
During a period of 8M+7, the output Y of the Y counter 8 is sent to the full adder 12.
, the frequency of significant pixels is cumulatively added, and the addition result is stored in the flip-flop 13 by the clock CK2. The contents of this flip-flop 13 are stored in the flip-flop 14 by the clock CK5. Thereafter, the flip-flop 13 is initialized to 0 by the clear pulse CL2, and the next partial area B (M+1.N
) in preparation for cumulative addition of Y coordinates of significant pixels.

The data stored in the flip-flop 14 is added to the cumulative addition data of the Y coordinate of the partial area B (M, N) held by the storage device 25 while the scanning area is 8M+8≦X≦8M+15, and the addition result is are stored in the same area of the storage device 25. That is, the pass buffer 22 is disabled by the clock CKIO, and the synchronization control circuit 24 transfers the partial area B (M,
The address of the area in which the accumulated data of the Y coordinate of N) is stored, and an output enable signal is also applied to the storage device 25. The Y-coordinate cumulative data output by the storage device 25 is thereby stored in the flip-flop 21 by the clock CK9. In parallel with this process, the accumulated data of the Y coordinate held by the flip-flop 14 is sent to the full adder 2 via the path buffer 15 by the clock CK7.
It is input to 0. However, in this case, the path buffer 11
．． 19 is in a disabled state. Next, the synchronization control circuit 24 places the storage device 25 in a data input state and enables the path buffer 22. Then, the addition result of the full adder 20 is stored in the area of the storage device 25 that holds the accumulated data of the Y coordinate of the partial area B (M, N).

1ii) X coordinate cumulative addition By counting, the Xyi mark X of the point P is identified. The flip-flop 17 is
Prior to measurement in partial region B (M, N), it is initialized to O by clear pulse CL3. Scanning area is 8M≦
During the period of X≦8M+7, the output X of the X counter 9 is at the point P (
x, y) are significant pixels, they are cumulatively added in the full adder 16, and the result of this addition is stored in the flip-flop 17 by the clock CK3. The contents of this flip-flop 17 are stored in the flip-flop 18 by the clock CK6. Thereafter, the flip-flop 17 is initialized to 0 by the clear pulse CL3, and prepares for cumulative addition of the X coordinates of significant pixels in the next partial area B (M+1.N).

The data stored in the flip-flop 18 is added to the cumulative addition data of the X coordinate of the partial area B (M, N) held by the storage device 25 during the period when the scanning area is 8M+8≦X≦8M+15, and the addition result is are stored in the same area of the storage device 25. That is, the pass buffer 22 is disabled by the clock CKIO, and the synchronization control circuit 24 transfers the partial area B (M,
The address of the area where the cumulative addition data of the X coordinate of N) is stored, and an output enable signal is also applied to the storage device 25. The X-coordinate cumulative data output by the storage device 25 is thereby stored in the flip-flop 21 by the clock CK9. In parallel with this process, the X coordinate accumulated data held by the flip-flop 18 is sent to the full adder 2 via the path buffer 19 by the clock CK8.
It is input to 0. However, in this case, the path buffer 11
-15 is disabled. Next, the synchronization control circuit 24 places the storage device 25 in a data input state and enables the path buffer 22. The addition result of the full adder 20 is stored in the area of the storage device 25 that holds the X-coordinate cumulative data of the partial area B (M, N).

The above processing i)+iiLiii) is executed over the entire one frame, and data of the following formula is sequentially stored in the cumulative addition data storage area of the partial area in the storage device 25.

X (M, N) = Σ Σ X 'C(X* y
) ;yxHXIIM X coordinate cumulative addition value Y (M, N) = Σ Σ y−C(x, y); y+
N XIIM Y coordinate cumulative addition value S (M, N) = Σ Σ C (x, y)
; Area ymW x mH where C(x, y) represents binary image data at point P (x, y);

The synchronization control circuit 24 generates the address of the storage device 25 and various control pulses as follows. That is, the address of the storage device 25 is generated by combining the top five digits of the output of the Y counter 8 and the top seven digits of the output of the X counter 9 in the format shown in FIG. In this case, ADRII and ADRIO are used to identify the partial region internal area storage section, partial region internal X coordinate cumulative addition value storage section, and partial region Y coordinate cumulative addition value storage section, respectively, and ADRO-ADR4 is used to identify the partial region B (M).

N) X coordinates 0≦M≦31 are identified, and ADR5 to ADR
9, Y coordinate of partial area B (M, N) 0≦N≦3
1 is identified, and furthermore, a clock pulse and a clear pulse are generated at the timing shown in FIG.

In order to eliminate the influence of noise that occurs when inputting an image, the CPU 26 executes the process described below using a threshold Th for the area information, and calculates the center of gravity position. FIG. 6 shows this processing procedure, and will be explained below according to this flowchart.

Step # of sending a start signal to the synchronous control circuit 24
1), then a label relationship correspondence table and a label area data table, which will be described later, are initialized (#2).

The path buffer 23 is enabled by the clock CKII, the upper five bits of the count value of the Y counter 8 are read, and the Y coordinate N of the partial area B(M, N) currently being processed is read (#3).

Assuming that the Y coordinate of the partial area for which the CPO 26 has completed processing is CN, when N-1≦CN, the process returns to step #3 to avoid address conflict within the storage device 25 and continues monitoring the value of N. . When N-1>CN, CN=N-1
and the X coordinate of the partial area where the CPU 26 starts processing is 0.
Set M to -l (#5).

Perform the operation CM=CM+1 #6) and read out the area S (CM, CN) of significant pixels in the partial area B (CM, CN) from the storage device 25 #7), S (CM, CN)
If CN)<Th, a stepwise jump is performed to proceed to processing of a new partial area row.

In the case of S (CM, CN)≧Th, partial region B(M
, N) is the area S(CM-1°CN-
1), S (CM-1, CN), S (CM.

CN-1), S (0M+1.CN-1) (7) are all smaller than the threshold Th, that is, when there is no 8-neighbor connection, the process moves to step #13. However, CM-1
<O or CN-1<0(7), then S(CM-1,C
N-1)=0 or S (CM-1°CN)=0 or S(CM, CN-1)=0 or S(CM+l, CN-
1)=O.

S (CM-1, CN-1), S (CM-1, CN)
.

S (CM, CN-1), S (0M+1.CN-1)
If at least one of them is not smaller than the threshold Th, there is already a labeled partial area in the 8-neighbor connected partial areas of the partial area B (CM, CN) of interest, so B is determined according to the following formula. (CM, CN) (D label) Determine LL (CM, CN) (#10). This label L (CM, CN) is an integer from 1 to 256, and is adopted in order from ■. In the initial state, all The label of the subregion is zero.

L (CM, CN) = MAX (L (CM-1, C
N).

L (CM-1, CN-1), L (CM, CN-1)
.

L (0M+1.CN-1)) That is, let the square partial region of interest be B (M, N),
The largest label among the labels attached to each partial area in the connected set of partial areas shown in FIG. 5 is taken as the label of the partial area of interest.

Next, in the connected aggregate of partial areas shown in FIG. 5, the connected state of the labels is investigated. That is, in this connected set of partial areas, if the labels of all partial areas are the same, the process moves to step #16 and subsequent steps. In this connected collection, L(CM.

If there is a label L (k, A) different from CN), create a label relationship correspondence table as shown in Figure 7 (#
12). After this, the process moves to step #16 and subsequent steps.

The label relationship correspondence table in FIG.
M, CN) and the label L
The connecting subregion identified by (k, l) is 8
This indicates that it is a neighborhood connection.

This label relationship correspondence table further includes labels L (k, l
) and the label L(i, j
) is an 8-neighbor connection, and the connected subregion identified by label L (p, q) and the connected subregion identified by label L(i, j) are labeled L ( It shows that there is 8-neighbor connection through the connecting partial region identified by k and G.

In FIG. 7, the lower label represents the label of a newly investigated partial area, and the upper label represents an existing label whose connection state has been newly confirmed as a result of the investigation.

If the partial region of interest is not connected to any of the partial regions for which processing has already been completed, a new label L (CM, CN) is added to the partial region B (CM, CN) according to the following formula.
CN) (#13). .

L (CM, CN) = MAX (L (p, q))
+ However, p=o~CM, q=1~CM.

Next, a new label is registered in the lower label column of the label relationship correspondence table (#14). The upper level at this time is O. Next, for cumulative addition processing of data in the connected partial area, a label L (CM, CN) is additionally registered at the end of column 1 of the label area data table shown in FIG. 8 (#15).

In step #16, the cumulative addition values of the X coordinate and Y coordinate 1 area S are further cumulatively added for each label, and each cumulative addition value for the label value l is set as LX(ff).

Let LY (f) and Ls (Il). Then,
A coordinate cumulative addition process, a Y coordinate cumulative addition process, and an area cumulative addition process are performed according to the following equations to create a label area data table.

LX (L (CM, CN)) = LX (L (CM, CN))
CN))+X (CM, CN) LY (L (CM, CN))=LY (L (CM,
CN))+Y (CM, CN) LS (L (CM, CN))=LS (L (CM,
CN))+S (CM, CN) (CM, CN) <(31, 31), the process returns to step #6 or #3 and processes a new partial area row. If (CM, CN) = (31, 31),
In order to calculate the center of gravity position of each connected label area, the process moves to step #19.

In step #19, the CPU 26 executes the label relationship correspondence table (
7) in reverse to obtain the youngest label value in.

Then, using this label value in, LX (in), L
Y (ln), LS (/n) in the label area data table (
Figure 8). Next, the upper label value 1m of the label value ln is obtained from the label relationship correspondence table. This label value 1
Each data LX(j!) in the label area indicated by m
m), L Y (1m).

Cumulatively add the label area name data indicated by the label value ln to LS (1m), and delete the label (lffJn) from the label relationship correspondence table. Repeat until no more labels can be found.

Next, in step #20, the center of gravity position is calculated for each connected label area. As a result of the processing in step #19, the labels remaining in the label relationship correspondence table are 1゜l z, -, l i
(i is the number of input patterns).

At this time, the gravity center positions of each of the plurality of input pattern patterns can be obtained by performing the following calculation process, assuming that the gravity center positions of each pattern are GXi and GYi.

In the above-described embodiment, the storage device 25 was used in a time-sharing manner, but a device having a dedicated storage device for each information of area, X coordinate, and Y coordinate may be similar in principle to the present invention. It is. In addition, in the example, a square with 8 pixels on one side was used as the size of the partial area, but C
In order to reduce the waiting time of the PU and improve the processing speed, a method of performing preprocessing on a partial area in which several pixels are connected to obtain the center of gravity position is an example of the present invention in which the shape of the partial area is changed.

<Effects of the Invention> As described in detail above, in the present invention, a plane that captures an input image is divided into partial regions each consisting of a predetermined number of pixels, and the X and Y coordinates of significant pixels and the number of pixels are determined for each partial region. At the same time, by labeling the partial regions, we distinguish between partial regions with consecutive significant pixels and partial regions with non-consecutive significant pixels, and calculate the centroid position from the cumulative addition value for each label. The center of gravity position of each input cover turn can be calculated in real time by simultaneous parallel processing.

Since the present invention can detect the center of gravity positions of multiple objects at a video rate, it can be applied to gripping objects by robots on production lines, product inspection, position measurement of high-speed moving objects, and the like. Furthermore, since labeling processing is performed, the number of objects within the field of view can be counted at the video rate. Furthermore, since area measurement is performed for each pattern, inspection for chips and scratches during product inspection can be performed simultaneously and in parallel at video rate. Each of these features is fast (
Since it can operate at video rates), it can become a core technology for visual feedback when automating adjustment work at the time of shipping of various products. Furthermore, by adding a device that switches between multiple cameras under the control of the CPU, one system of the present invention can handle multiple production lines by taking advantage of the high-speed processing of the present invention.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing the configuration of the embodiment of the present invention, FIG. 2 is a diagram showing the configuration of the image analysis plane of the embodiment of the present invention, FIG. 3 is a signal timing chart of the embodiment of the present invention, and FIG. 4 5 is a diagram showing the address generation format in the synchronous control circuit according to the embodiment of the present invention, FIG. 5 is a diagram illustrating a connected aggregate of partial areas according to the embodiment of the present invention, and FIG. 6 is a flowchart showing the processing procedure of the embodiment of the present invention. , FIG. 7 is a diagram showing a label relationship correspondence table according to the embodiment of the present invention, and FIG. 8 is a diagram showing a label area data table according to the embodiment of the present invention. 1...TV camera 2.3...Object 4...Synchronization separation circuit 5...Binarization circuit 7...S counter 8...Y counter 9...X counter 10.13, 14.17, 18.21...Frissopflovub 12.113.20...Full adder 24...Synchronization control circuit 25...Storage device 26...CPU 27...ROM 28...RAM Patent Applicant Sharp Corporation Agent Patent Attorney Nishi 1) New Figure 4 Father

Claims (1)

[Claims] a first calculating means for cumulatively adding the X and Y coordinates and the number of significant pixels of a significant pixel for each partial region in a partial region in which the image plane is divided into a predetermined number of pixels; a labeling means for attaching a label to the partial region to distinguish between partial regions that are connected to each other and partial regions that are not connected; An image processing device for detecting a center of gravity position in real time, comprising: second calculation means for calculating a center of gravity position from an added value.