JPH0452773A - Picture supervisory method - Google Patents

Abstract

PURPOSE:To allow dispersion in the size of a characteristic area and dislocation by providing a dead zone area near a binarized level and supervisoring and judging a picture based on a binary code binarized beyond the dead zone area. CONSTITUTION:An output voltage value in a characteristic area part (w) to which work is executed is changed compared to the other part. The part is binarized at the prescribed binarized level Vth and a cross point between the binarized level Vth and an output voltage is set to be the dead zone area so as to remove it from an inspection area. Since a binarized signal from the object of normal supervisory is I(Vth), the area is between the area of I(Vth-DELTA) and the area of I(Vth+DELTA) and the characteristic area of the object of normal supervisory in the inspection area becomes the same as the area of I(Vth+DELTA). Thus, the dead zone area is provided near the code change point at the time of binarization and the picture is supervised and judged beyond the dead zone area. Thus, permissible dislocation and the variance of the size in the characteristic area of the object of supervisory are permitted and supervisory following the target of supervisory can be executed.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention is useful for monitoring whether IC elements, etc. are correctly mounted on a flat substrate, or whether holes are correctly drilled in an object to be monitored, etc., in the manufacturing and assembly process, for example. The present invention relates to an image monitoring method for carrying out.

[Prior Art] To determine the quality of length, pattern, etc., for example, to determine whether the length of the object to be inspected (object to be monitored) is as specified or whether the shape of the object is as specified, Take an image with a camera, etc., specify an inspection area in a window that includes the feature area where the length and shape appear, and measure the physical quantity by counting the number of pixels in that area where the length and shape appear. The quality is determined based on whether the number of pixels is within a permissible value or not.

By the way, the accuracy of the position of the hole and the accuracy of the hole diameter are important when it comes to visual inspection after processing in the manufacturing and assembly process, for example, to monitor whether the hole is drilled correctly after the hole is drilled in the process. Instead, in many cases, it is sufficient that a hole with a predetermined diameter is opened near a predetermined position. Further, in the element mounting process, the mounted elements may vary in size, and the mounting position may often vary to some extent.

[Problem to be solved] Therefore, in inspections (hereinafter referred to as monitoring) based on vague and abstract criteria such as visual inspections, the above deviations,
Variations etc. must be allowed.

Therefore, the purpose of the invention is to
It is an object of the present invention to provide an image monitoring method that can tolerate positional deviations, etc., and can also tolerate variations in monitoring conditions.

[Means for Solving the Problems] In order to achieve the above object, the present invention provides an image monitoring method in which a monitoring image is binary coded at a binary level and a monitoring judgment is made using the binary code. A dead zone area is provided in the vicinity of the digitization level, and monitoring determination is performed based on the binary code binarized outside the dead zone area.

[Example] Hereinafter, an example of the present invention will be described in detail based on the drawings.

First, the overall configuration of the image monitoring device used in the present invention will be explained. In Fig. 4, monitoring target 1 is a manufacturing line, etc., where holes are made during the processing process, element mounting process, etc. (not shown)
Under illumination 2, an imaging means 3 takes an image of the object to be processed. Imaging is performed for both a preliminary image before processing and a subsequent image after processing. Input control signals for completion of machining preparation and completion of machining in the processing machine are input to the main body 7 of the apparatus through the input peripheral line 5. The operating conditions of the device main body 7 and the like are input using a keypad 8. Human control signal or line 5
Each time the device is manually operated, the device body 7 performs preliminary monitoring and post-monitoring 1, and outputs the results as an output control signal from the output line 6. The main body 7 of the apparatus is equipped with a CRT 7a capable of displaying the image taken by the imaging means 3, as well as the binarized image signal and the like.

Next, the internal configuration of the device main body 7 will be explained using FIG. 5.

A program for operating the CPU 22 is stored in the ROM 21 . In particular, the ROM 21 includes a program for extracting inspection areas for pre- and post-monitoring,
A program for taking a product image of an inspection image and an inspection area is stored (see FIG. 1). Note that the program for extracting the inspection area and the program for taking the product image of the inspection image and the inspection area can be provided in the apparatus main body 7 as separate hardware.

The RAM 23 stores data such as the inspection area in pre-monitoring and post-monitoring, the size and position of N inspection blocks, the reference value (number of pixel change points) and tolerance value for pass/fail judgment in each block. There is. Conditions such as the size and position of the inspection block, reference values and tolerance values for each block, etc. While looking at the screen ``a'', input from the keypad 8, etc., and store it in the RAM 23 via the I/O port 29.

Based on the commands in the ROM 21 and the data in the RAM 23, the CPU 22 sets inspection areas in pre-monitoring and post-monitoring, and determines the quality of each test block in pre-monitoring and post-monitoring.

The image signal captured by the imaging means 3 is binarized through the A/D circuit 30 and stored in the 71 norm memory 31, and
The contents of the frame memory 31 are transferred to the CRT 7 via the display circuit 32.
It is displayed on a. Note that the 71 norm memory 31 stores a prior image and a subsequent image, respectively.

Next, details of this embodiment will be explained using FIGS. 1.2.3 and 6.7.

The image monitoring process consists of two monitoring operations: pre-monitoring and post-monitoring. Preliminary monitoring involves inputting an image of the object to be monitored before processing and determining whether it is in a normal state.

If an abnormality is detected, processing is immediately stopped. Post-event monitoring determines whether the pressurization completion status is normal or not.

If an abnormality is detected, immediately retest or cancel the next test.

In FIG. 6, the monitoring conditions are set (602), a normal preliminary image of the monitoring object 1 and the normal monitoring object 1.
The post-images after normal processing are input, and each is binarized by the A/D circuit 3o (603). In each step (7), the feature amount in the inspection area is calculated and used as a reference value for pass/fail judgment, and the area is divided into each inspection block (604).

Below is the method for setting the inspection area in post-monitoring.

The calculation method with features etc. will be explained.

Next, we will explain how to set the inspection area.

FIG. 2(a) shows the output voltage in the normal post-image from the imaging means 3, and shows the processed characteristic region portion (hole) W.
The output voltage value changes with respect to other parts, and by binarizing at a predetermined binarization level vth, this part can be distinguished from other parts, and the feature amount of this part can be calculated. By using this as a reference value and comparing the feature amount obtained by inspection of each monitored object with this reference value, the quality of the monitored object can be determined. However, in monitoring, the variation is 1
Since it is desired to allow positional deviation, etc., the vicinity of the intersection between the binarization level Vth and the output voltage is excluded from the inspection area as a dead zone area (see FIG. 2(b)).

In order to do this, a predetermined amount of slight change in the binarization level Δ
is determined, binarized using Vth-Δ and Vth+Δ, the exclusive OR ☆ of these is taken, and the result is logically negated to obtain the inspection area B. In other words, the binarized image is I
(Vth), then B-A (I (Vth-4)
*I (Vth+li) This equation is realized in a block diagram as shown in the part J in Figure 1.The value of Δ is based on the allowable deviation of 1 position due to the variation in the size of the characteristic region of the object to be inspected and monitored. It is preferable to set the feature area so that when it is within the range, the outline of the feature area falls within the dead zone area.

Furthermore, how to obtain the inspection area will be explained using a specific example shown in FIG.

Assume that a characteristic region consisting of a round part and a hook part is generated in the monitoring screen S consisting of the entire photographing screen of the imaging means 3 or a part thereof surrounded by a window or the like. After taking a normal post-mortem image, we take the exclusive OR of I (Vth-Δ) in figure (a) and I (Vth+Δ) in figure (b). (b) A portion with a different sign appears as code 1, and a portion with the same sign appears as code O, so that the dead zone region shown in (C) of the same figure is obtained. Therefore, by logically negating this, the inspection area B shown in FIG. 3(d) can be obtained. Here, inspection area B has a binarization level of Vth+Δ
When the sign change position and the binarization level are Vth−Δ
The area between the sign change position and the sign change position is deleted from the inspection area as a dead zone area.

Next, when the inspection area is determined, the feature amount of the normal monitoring object in this inspection area is calculated (604).

Here, since the binary signal of the normal monitoring object is I (Vth), its area is the area of I (Vth - Δ) and I (Vt
Therefore, the characteristic region of the normal monitoring object in the inspection region is the same as the region I (Vth + Δ). Therefore, the number of pixels in the area (V th + Δ) is counted and used as a reference value for quality determination.

Next, the monitoring area S is set to the optimal size N for pass/fail judgment.
(not shown). Then, the number of the above reference values in each test block is calculated. Also, the allowable value for determining the quality of each inspection block is set.

The above explanations of setting the inspection area, calculating reference values for pass/fail judgment, dividing into test blocks, and setting tolerance values are for post-monitoring, but settings and calculations are performed in the same manner for pre-monitoring.

This completes the pre- and post-reference image teaching (6 0
4).

Next we move on to actual monitoring. When the processing preparation end signal is received via the line 5 (605), the machined surface of the monitored object 1 is imaged, the pre-inspection image is input and binarized, and a binary signal 1a is obtained (606). It is stored in the memory 31. Since this signal Ia is also used for post-monitoring, it is stored even after the pre-monitoring ends. Then, the characteristic amount in the inspection area is determined and compared with the reference value for each block, and if it is within the tolerance value, it is determined to be good, and if it is outside the tolerance value, it is determined to be defective (607), and the result is outputted as a control signal C. (6)
08).

When the pre-monitoring is judged to be good, the processing is performed, and when the processing end signal is received (609), the image of the monitored object 1 after processing is taken, and the post-inspection image is input and binarized to generate a binary code. The signal 1b is obtained (610) and stored in the frame memory 31.

Then, as shown in FIG. 7, a difference image Ic between the pre-binary code signal 1a and the post-binary code signal Ib is calculated using exclusive OR (704. See the part in FIG. 1). As a result, as shown in FIG. 3(f), the portion that has changed due to processing is extracted as an image with reference numeral 1. It is assumed that the portion extracted as the image 1 of the object to be inspected and monitored consists of a round portion U and a hook-shaped portion V. The round part U is not misaligned compared to the normal one, and its diameter is within the dead zone area, and the hook-shaped part V is smaller in size than the normal one. is within the dead zone region, but is shifted to the left by more than the allowable value.

Then, by taking the logical product of the difference image 1c and the inspection area B, the part within the inspection area is extracted as the inspection value A (7
05. (See Figure 1). As shown in Figure 3(g), the third
Of the image shown in FIG. 3(f), a portion within the inspection area is extracted as the inspection value A. Here, since the part of the dead zone region is not extracted as the test value A, only the part u1 that is the same as the reference feature region of the round part U is extracted, and the piece-shaped part ■ is the hook-shaped part within the reference feature region. The part Vl covered by (2) and the part v2 which protrudes outside the dead zone area due to the position shift to the left. v3 is extracted.

Then, the test value A is divided into each test block, and the number of pixels Pi that changes between before and after in each test block is
becomes (707). Then, it is determined whether Pi is within the allowable value or not, and a pass/fail judgment is made for each test block (708 to 712
), output the results as post-monitoring results (612
). In FIG. 3(g), a portion of the piece-shaped reference feature region that is not covered by the portion vl, and a portion that is not covered by the dead zone region v2. Due to the existence of v3, if the results of inspection of the test blocks in which these test blocks exist exceed the allowable values, those test blocks are determined to be defective. It should be noted that among the remaining pixels remaining in the inspection area B, those due to noise are small in number, so they fall within the allowable value during inspection of the inspection block, and are therefore ignored during determination.

If any inspection block is determined to be defective,
If it is possible to cancel the next process or re-inspect, re-inspect the inspection block determined to be abnormal.

All blocks are good, and the result of post-mortem monitoring (6
If 12) is good, the next processing is continued and the same monitoring as above is repeated.

In the above embodiment, post-monitoring is performed by calculating the logical product of the difference image between the pre-image and post-image and the inspection area.
It is possible to remove noise such as minute changes before and after the process.
Misjudgments caused by them can be prevented.

In the post-monitoring of the above embodiment, the test image is a difference image between the binarized pre-image and post-image, or the test image is binarized as a post-image, and this post-image and the test area B
The test value A may be obtained by performing a logical product with the following.

In addition, when the test image is a difference image between the binarized pre-image and post-image, in the j part of Fig. 1, the input normal image signal is divided into the pre-binarized pre-image and post-image A difference image between the two images may be used, and operations such as binarization, exclusive OR, and logical negation may be applied to this image.

[Effects] In the present invention, a dead zone is provided in the vicinity of the sign change point during binarization, and monitoring judgment is performed outside this dead zone. Monitoring can be performed in accordance with the purpose of monitoring by allowing for irregularities in size.

In addition, even if the output of the imaging means changes due to changes in the illuminance of the monitoring surface, if the changes are within a predetermined range, the dead band will be absorbed by this and correct monitoring can be performed. .

[Brief explanation of the drawing]

Fig. 1 is a block diagram showing the configuration of a circuit that obtains inspection values from the output of the imaging means in an area excluding the dead band area, and Fig. 2 (a) is a block diagram showing the configuration of a circuit that obtains inspection values from the output of the imaging means in an area excluding the dead band area. The output of the means is Vth+Δ and Vth−Δ
An explanatory diagram for explaining the operation of binarizing with
is an explanatory diagram showing the inspection area, and Figures 3(a) to (c) are B
- An explanatory diagram for explaining the detection operation of the dead zone region using the calculation formula of Afl (Vth - Δ) ☆ I (Vth + Δ)),
Fig. 3 (cl) is an explanatory diagram showing the inspection area obtained as a result, Fig. 3 (e) is an explanatory diagram showing the characteristic area of the normal monitoring target, and Fig. 3 (f) and (g) are the explanatory diagram showing the inspection area obtained as a result. An explanatory diagram for explaining the binary encoding operation for an object, Fig. 4 is an external perspective view showing how the image monitoring device monitors the object to be monitored, and Fig. 5 shows the internal configuration of the image monitoring device. The block diagram shown in FIG. 6 is a flowchart for explaining the image monitoring operation, and FIG. 7 is a flowchart for specifically explaining the post-monitoring process in the flowchart of FIG. 6. Vth...Binarization level. that's all

Claims (1)

[Claims] In an image monitoring method in which a monitoring image is binary coded at a binarization level and a monitoring judgment is made based on the binary code, a dead band area is provided near the binarization level, and a dead band area is provided outside the dead band area. An image monitoring method characterized in that the monitoring determination is made based on a binary code that has been binarized.