JPH10274626A - Image processing device - Google Patents

Abstract

(57) [Summary] [Problem] Image processing capable of accurately inspecting a defect of an inspection object without falsely reporting a defect even if there is variation in the shape of the inspection object or variation in the appearance of an image of the inspection object. Get the device. Kind Code: A1 A camera for individually imaging a teaching sample and an inspection object, and generating unique image data from a teaching image that is an image of the teaching sample TS captured by the camera, and capturing the inspection object. An approximation image is restored from the inspection image which is the image of the object and the generated unique image data, and a difference image between the inspection image and the approximation image is generated.
A CPU 5 is provided to determine the quality of the inspection object by performing particle analysis on the binarized image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus for performing positioning control and inspection of a workpiece based on image data of the workpiece handled by an industrial robot or an industrial machine.

[0002]

2. Description of the Related Art FIG. 33 is a block diagram of a defect inspection apparatus for a wafer to be inspected using a pattern recognition method disclosed in Japanese Patent Publication No. 6-325181. In the figure,
W1 is a non-defective wafer for creating a basic pattern vector, and W2 is an inspected wafer to be inspected. A test pattern vector is created from these wafers W1 and W2.

M is a moving mechanism for moving the wafers W1 and W2, OP is a microscope for enlarging and viewing a pattern on the wafer surface, 1 is a CCD camera for capturing an image of the wafer surface enlarged by the microscope OP, and 3 is a CCD. A display for monitoring an image captured by the camera 1, 2 is a CC
An image input device for A / D converting and digitizing an image captured by the D camera 1, a frame memory 4 for storing digitized image data, and a pattern memory 5 described later using the image data of the frame memory 4. A central processing unit (hereinafter referred to as CP) that calculates the similarity between the pattern of the non-defective wafer W1 and the pattern of the wafer to be inspected W2 by the procedure of the recognition method.
U) and 6 are memories for storing calculation programs and intermediate results, and 8 is a display device for displaying inspection results.

Next, the operation and operation of the conventional defect inspection apparatus will be described with reference to the flowchart of FIG.
The non-defective wafer W1 is placed on the moving mechanism N and the front surface of the wafer is CC
The image data is digitized by the image input device 2 and stored in the frame memory 4 (43).
-1). Based on the image data, a set of basic pattern vectors is obtained by a method described later (34-2), and a linear subspace L is obtained.

[0005] The number of basic pattern vectors obtained here is assumed to be m. An n-dimensional (n = X ×) in which these two-dimensional basic pattern vectors (X × Y [pixels]) are arranged in one line
A set of basic pattern vector the basic pattern vector of _{Y) Xi = (X i1,} X i2, ..., and X _in) ^T. Where T
Represents transposition. Let L be a linear subspace spanned by these m pattern vectors (34-3). This linear subspace is represented by equation (1).

L = {X _i ∈Rn | i = 1, 2,..., M} (1)

Next, while placing the wafer to be inspected on the moving mechanism T and scanning the entire surface of the wafer pattern with the CCD camera 1, the basic pattern vector is represented in n dimensions similarly to the inspection wafer.
A test pattern X is obtained (34-3). Here, X is projected onto L, and a vector Xc belonging to the complement space Lc of L is obtained. If X is similar to the basic pattern vector, X is X
_i is approximately equal to any of the patterns, and is substantially included in the linear subspace L.

However, if X has an element different from the basic pattern vector, X is not included in the linear subspace L, and a complementary space Lc of L exists for the different element. Therefore, a value obtained by projecting X onto L and adding a minus sign to the norm | Xc | of Xc represents the similarity between the basic pattern vector and the test pattern vector (34−
5).

Any norm can be used as long as it satisfies the mathematical definition. L2 norm {(Xc) ^T Xc} ^0.5
Is a typical example. It is determined whether the norm exceeds the threshold (34-6). If the norm is smaller than the threshold, it is determined that there is a defect (34-7). If the norm is larger than the threshold, it is determined that there is no defect (34-6). -8). It is checked whether the entire wafer has been inspected (34-9). If the inspection is not completed, the process returns to step 34-3 to capture the image of the remaining non-inspection wafer. If the whole wafer is inspected, the result is displayed on the display device 8.

When orthogonal projection is used as the projection, a vector Xc belonging to the orthogonal complement space Lc of X is obtained by the following equation.

1. From the set of m basic pattern vectors X _i ( _i = 1, 2,..., M), a linear subspace L
, The orthogonal vector H _i ( _i = 1, 2,..., M).

H ₁ = X ₁ (2)

[0013]

(Equation 1)

However, (X _i , H _k ) represents the inner product of X _i and H _{k. If} | H _i | = 0 during the calculation of equation (3), X _i is X ₁ Is discarded since X _i is represented as a linear combination of X _i−1 to X _{i−1, and} the calculation is continued for X _{i + 1} .

2. The test pattern vector X is orthogonally projected onto L, and Xc is obtained by equation (4).

[0016]

(Equation 2)

M 'represents the number of orthogonal vector columns Hi.
As described above, if | Hi | = 0 during the calculation of equation (3), m ′ becomes smaller than m, and otherwise m ′ = m.

[0018]

The defect detection method using the conventional apparatus can perform pattern recognition without being affected by the positional deviation of the pattern. However, a pattern obtained by imaging one good product and storing it in the frame memory is stored in the computer. Since a set of teaching patterns is generated by moving with, only the displacement of the position can be dealt with.

However, in practice, the inspection target at the time of inspection includes not only the positional displacement of the pattern but also the shape of the pattern. That is, the appearance of the pattern often changes depending on the irradiation angle with respect to the inspection object. In addition, for example, when inspecting a machined part, the pattern may vary depending on the shape of the curved surface of the contour of the object to be inspected or the manner in which the irregularities change due to misalignment during machining, wear of the tool, or the like.

When a pattern inspection of a semiconductor wafer is performed, not only does the entire pattern shift, but also the relative positions of the patterns change through several etching steps.
Also, the pattern changes due to variations in the time taken for the chemical process. Furthermore, even if the same product is manufactured by a different part manufacturing line or part supplier, the surface gloss, the direction of processing scratches, the roughness, the shape of the chamfer, and the like vary. Then, they appear as variations in the pattern.

Since such a variation cannot be generated by the basic pattern vector generation method according to the conventional method, there is a possibility that even if a good product is inspected, it is determined to be defective due to the variation. Further, in the conventional defect inspection method, since the pass / fail judgment is performed by threshold processing the absolute value of the norm value of the complementary space vector (= difference image), the entire inspection image is compared with the basic pattern. Therefore, when the inspection image is entirely different from the basic pattern, it is determined to be defective. Further, even if a local area in the inspection image has a defect, when the area is small, the area is ignored and may be determined as a non-defective product.

Further, when the operator adjusts the threshold value of the norm value for performing the quality judgment by experience, know-how is required for the adjustment. Therefore, it is difficult for a user who has little knowledge of image processing to set an appropriate threshold.

When performing a defect inspection on a production line,
In many cases, a user wants to judge pass / fail based on the size of a defect, but it is difficult to compare a threshold with a physical quantity such as the size of a defect to be extracted by a conventional method.

The present invention has been made in order to solve the above-described problems. Even if there is a variation in the shape of the inspection object or a variation in the appearance of the image of the inspection object, the defect of the inspection object can be obtained. It is an object of the present invention to obtain an image processing apparatus capable of inspecting a defect with high accuracy without falsely reporting the defect.

[0025]

According to a first aspect of the present invention, there is provided an image processing apparatus comprising: an imaging stage for individually imaging a teaching sample and an inspection target; and an image of the teaching sample captured by the imaging unit. A unique image data generating unit that generates unique image data from a certain teaching image, an approximate image restoring unit that restores an approximate image from the inspection image that is the image of the captured inspection object and the generated unique image data, Difference image generation means for generating a difference image between the inspection image and the approximate image, binarization means for binarizing the generated difference image, and pass / fail of the inspection object by particle analysis of the binary image Determining means for determining.

An image processing apparatus according to a second aspect of the present invention provides an image processing apparatus which has a relation between the number of unique image data used for restoring an approximate image and an inspection processing time, or an inspection represented by the number of inherent image data and a contribution rate. Calculation means for calculating the relationship with the reliability, and display means having a function of simulating each of the relations calculated by the calculation means to the user at the time of inputting an image of the teaching sample.

An image processing apparatus according to a third aspect of the present invention provides a moving table for moving a teaching sample, illumination for irradiating the teaching sample at a predetermined angle, a controller for controlling movement of the moving table, A controller that controls the intensity of the illumination, and a teaching image variation generating unit that issues a command to the controller and changes an illumination position and an illumination intensity with respect to the teaching sample to generate a teaching image having variation in the image. .

According to a fourth aspect of the present invention, there is provided an image processing apparatus, comprising: a matching data generating means for determining an inspection area in an inspection image which is an image of an inspection object, and generating and storing template matching data based on the inspection area; ,
A unique image data generating unit that cuts out each inspection area from a teaching image that is a captured image of a plurality of teaching samples based on the template matching data and generates unique image data from the image data of the cut out inspection area; Approximation image restoring means for restoring an inspection area image of the inspection object image cut out by the extraction means and an approximate image from the generated unique image data; difference between the inspection area image of the inspection object image and the approximate image; The image processing apparatus includes a difference image generation unit that generates an image, a binarization unit that binarizes the generated difference image, and a determination unit that determines the quality of the inspection object by performing a particle analysis on the binary image.

An image processing apparatus according to a fifth aspect of the present invention includes a normalizing means for normalizing the density of each of the captured teaching images.

According to a sixth aspect of the present invention, there is provided an image processing apparatus comprising: a class calculating circuit for obtaining a feature amount of a class in an eigenspace using a taught image used for teaching as a non-defective class; It is provided with quality judgment means for judging the quality of the inspection object based on the middle coordinates and the feature amount of the class.

An image processing apparatus according to a seventh aspect of the present invention includes a Mahalanobis distance calculating means for calculating a distance between a class on an eigenspace and a certain point by using a Mahalanobis distance.

According to an eighth aspect of the present invention, there is provided an image processing apparatus which divides a taught image vector into a non-defective class and a defective class in an eigenspace, and calculates a distance between each class and an inspection image in the eigenspace. And a pass / fail judgment unit for judging pass / fail of the inspection object based on the calculated distance.

According to a ninth aspect of the present invention, there is provided an image processing apparatus, comprising: a contribution rate recording means for recording a contribution rate of each unique image data at the time of generating an approximate image; A rate state determination unit and a re-teach warning unit that issues a warning to newly teach an object when it is determined that the contribution rate is not appropriate are provided.

According to a tenth aspect of the present invention, there is provided an image processing apparatus comprising:
Correlation value recording means for recording a correlation value of template matching when cutting out an inspection area image from an image to be inspected by template matching, and correlation state determination means for judging the validity of the teaching image from a change rate of the correlation value And a re-teach warning unit that issues a warning to teach a new teaching sample when it is determined that the correlation value has deteriorated.

An image processing apparatus according to an eleventh aspect of the present invention
The teaching image is acquired from the teaching sample at regular intervals during the inspection operation or in conjunction with replacement of the component supply lot to be inspected.

[0036]

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiment 1 FIG. Hereinafter, an image processing apparatus having a defect inspection function according to the present invention will be described with reference to the drawings. FIG. 1 is a configuration diagram of an image processing apparatus according to the present embodiment. In the figure, reference numeral 1 denotes a camera as an imaging unit for imaging an inspection object for teaching / inspection, 2 denotes an image input device which receives an image signal input from the camera 1 and converts the image signal into a digital signal, and 3 denotes a camera. A display for displaying an image signal input from 1 as a predetermined image, 4 is a frame memory for storing an image signal of a digital signal converted by the image input device 2, and 5 is an image signal stored in the frame memory 4. A memory for storing calculation programs and intermediate results; a unique image data storage device for storing unique image data as teaching data;
Is a display device for displaying the inspection result, and 9 is an input device such as a mouse or a keyboard for operating the teaching process and the start and end of the inspection process. Note that the CPU 5 constitutes a unique image data generation unit, an approximate image restoration unit, a difference image generation unit, and a determination unit.

FIG. 2 is a flowchart showing the flow of the processing of the defect inspection apparatus. First, the teaching operation is performed as follows. Non-defective sample to be inspected (teaching sample)
Collect several. Install a good sample under the same environment as the inspection. The non-defective sample is taken by the camera 1, and the image is converted into a digital signal by the image input device 2 and stored in the frame memory 4 (2-1). This operation is repeated to obtain a plurality of non-defective sample images. Next, a set of unique images is obtained from a plurality of non-defective sample images (teaching images) by a method described later, and the unique images are arranged in descending order of the unique values and stored as teaching data (2-2).

Next, 2-3 to 2-10 of the flowchart will be described.
The method of the inspection process will be described in accordance with the respective steps.
An inspection object is set on the moving mechanism M, imaged by the camera 1, and an image of the inspection object is captured (2-3). This inspection image is projected on a unique image. The projection method is performed in the same manner as the conventional method. Then, an approximate image is generated by projection (2
-4). The difference between the approximate image created by the projection and the image to be inspected captured in 2-3 is taken to create a difference image (2-
5).

Next, binarization processing of the difference image is performed (2-
6) A region having a large difference is cut out and labeling is performed on the binary image. The area of each labeled region in the image is determined by particle analysis (2-7).
Then, it is determined whether or not the area exceeds a preset threshold value (2-8). If the area of the region does not exceed the threshold, it is extracted as defective and displayed (2-9). If the area of the region exceeds the threshold value, it is determined that there is no defect, that is, a non-defective product (2-10).

FIG. 3 illustrates the inspection of a part having a hole at a specific position of a T-shaped plate as a specific non-defective inspection sample. In order to perform teaching, first, several test objects (T-shaped plates) of the same type are collected as teaching samples for teaching. Even for the same type of inspection object, variations occur during the manufacturing process. For example, when a hole is drilled in an inspection object using a punching machine, the position, size, shape, and the like of the hole vary.

In the case where the inspection object is made by casting, variations occur due to a deviation of a machine tool for polishing a casting surface and a skill of an operator. The variation in this case is a variation within an acceptable range as a product and does not mean a defect. In order to inspect a defect, first, each inspection object is imaged by the camera 1 as a teaching sample. Each captured image is input to the image input device 2 as a teaching image, is converted into a digital signal, is captured by the CPU 5, and is subjected to image processing.

That is, the CPU 5 generates a unique image for extracting a common component from each input teaching image. The generated unique images are arranged in order from a unique image having a large unique value, and all the teaching images can be reproduced by a sum obtained by applying an appropriate weight to the unique image. The unique image information is stored as unique image data storage device 7 as good product teaching information (unique image data).
Is stored in

Next, when the inspection is performed, the inspection object is imaged by the camera 1 and an inspection image is input. The inspection image is reproduced using the unique image. The inspection object also has a variation, but if the variation has the same variation as that in the image already taught, the variation can be expressed using the unique image. However, variations that are not in the teaching image, for example, defects such as scratches and chips, cannot be represented by the unique image,
The closest approximation image is an image having no defect of the object.

Therefore, if a difference image between the approximate image and the input inspection image is created, the normal part disappears and the defective part is extracted.
Next, noise is removed by binarizing the difference image, and a portion of the remaining region whose area is larger than the threshold value is determined as a defect.

How to Obtain Unique Image: X _i =
{X _i1 , ..., x _in }. Where i is 1 <= i <= m
And represents the number of the teaching image. m is the number of images taught. Also, n represents the size of the image. For example, n = 4096 for an image of 64 × 64 pixels.

2. An average image c ^(p) of these teaching images is obtained.

[0047]

(Equation 3)

3. A difference image x ′ _i = x _i− c ^(p) obtained by subtracting the average image from each image, and 1 <= i <= m is created.

4. A set X of difference images is represented by X = ｛x ′ ₁ , ..., x ′
Define _m ｝ ^T. T represents transposition. 5. Let Q be the covariance matrix XX ^T of X. Q is an m × m-dimensional matrix. The eigen equation λ _iei = Q _ei is solved using an eigen value solution such as the Jacobi method to obtain an eigen image e _i .

Embodiment 2 Next, an image processing apparatus according to Embodiment 2 will be described with reference to the drawings. FIG. 4 is a configuration diagram of the image processing apparatus according to the present embodiment. In FIG. 1, FIG.
The same reference numerals indicate the same or corresponding parts. This image processing apparatus has a configuration in which a unique image contribution ratio calculation circuit 10 as a calculation unit is added to the image processing apparatus according to the first embodiment.

FIG. 5 is a flowchart showing the flow of the processing of the image processing apparatus. The teaching work is performed as follows.
A plurality of good samples of the inspection object are collected as teaching samples. Install a good sample in the same environment as the inspection. A non-defective sample is taken by the camera 1 (5-1), and the image is stored in the frame memory 4 as a teaching image. This is repeated to obtain a plurality of images of the object to generate a unique image. This set of unique images is determined and arranged as teaching data in ascending order of the unique values (5-2). The processing so far is the same as the teaching mode in the first embodiment.

Next, a trial mode is entered (5-3). In the trial mode, the parameters are adjusted while checking whether the inspection can be performed correctly with the data taught so far. Embodiment 1
Generates an approximate image using the unique image. However, when the variation of the product is small or when the required specification of the defect inspection is gradual, the approximate image can be generated with a small number of the unique images.

A reduction in the processing time can be expected by reducing the number of unique images used to restore the approximate image. Accordingly, the relationship between the processing time and the reliability is simulated and shown to the user in advance, and the contribution ratio of the unique image is set as shown in FIGS. 6A and 6B so that the user can select an appropriate number of unique images. The information is displayed on the display means 8 as the display means.

In the trial mode (5-3), first, an object to be inspected is imaged by the camera 1, and an inspection image is input to the image input device 2 (5-4). Next, in the teaching mode, an approximate image is generated using the unique image of the non-defective sample image stored in the frame memory 4 (5-5), and the contribution ratio of each unique image at that time is shown in FIG. It is displayed on the display means 8 as shown in (b) (5-6). In the example of FIG.
A line is drawn at a position of 70% with 0% as a target and shown to the user.

In FIG. 6A, since the variation of the product is small and the contribution ratio exceeds 70% up to the third unique image, the user selects 3 unique images. Some products require up to the ninth unique image to achieve a contribution rate of 70% as shown in FIG. 6B. In this case, the contribution rate can be increased by increasing the number of unique images, and the reliability can be improved.

If the time required for the inspection process is displayed together with the contribution rate, the tact time on the production line can be estimated. After the user selects the number of unique images corresponding to the contribution ratio, the user inputs the number of images m '(5-7), and creates an approximate image using the generated m' unique images to perform defect detection processing. (5-8). This defect detection processing is performed in the same manner as in the first embodiment, and the result of the defect detection is displayed on the display 8.
(5-9).

The user confirms whether or not the defect display result matches the user's intention, that is, whether a defect of the same degree as the user's intention has been extracted, and outputs data indicating a match or mismatch from the input device 9 to the CPU 5. (5-1)
0). If it does not match the user's intention, the user enters the unique image number input menu again to increase the number of unique images and perform a test (5-5 to 5-10).

When a result satisfactory to the user is obtained, a trial is performed for another sample (object) (5-12).
Further, it is determined whether to continue the trial mode (5-13),
When the user determines that the parameter has been sufficiently adjusted through the trial mode, the trial mode is terminated (5-1).
4). However, if the trial mode is to be continued,
Return to and input the inspection image.

Embodiment 3 Next, an image processing apparatus according to Embodiment 3 will be described with reference to the drawings. FIG. 7 is a configuration diagram of the image processing apparatus. In the figure, the same reference numerals as those in FIG. 1 indicate the same or corresponding parts. The image processing apparatus includes an illumination 11 for irradiating an object, an illumination controller 13 for adjusting the intensity of the illumination 11, a table T on which the object is placed and moved in the XYZ and rotational directions, and a movement of the table T. The table controller 14 to be adjusted and the condition setting data storage device 12 for managing the states of the controllers 13 and 14 and issuing a movement instruction are added. The teaching image variation generating means is constituted by the CPU 5.

FIG. 8 is a flowchart showing the flow of the processing of the image processing apparatus. The teaching work is performed as follows.
One or more teaching samples (non-defective samples) TS of the inspection object are prepared and set on the table T (8-1). next,
The lighting condition and the table moving condition are determined based on the lighting 11 and the table condition setting data stored in the condition setting data storage device 12 (8-2, 8-3). The CPU 5 sends an instruction for generating a taught image variation to each of the controllers 13 and 14 in accordance with the determined conditions, and controls the movement of the illumination 11 and the table T (8-4, 8-).
5).

Next, the teaching sample TS is imaged by the camera 1 and the image is stored in the frame memory 5 (8-
6). Similarly, a process of moving an illumination condition or a table position to capture an image is performed. It is determined from the display image on the display 3 whether or not the teaching operation is to be repeated based on the stored image (8-
7) When the teaching operation is repeated, the procedure returns to 8-2 again. When the teaching operation is not repeated, it is determined whether or not to teach with another sample (8-8). When teaching with another sample, the process returns to 8-1 and another teaching sample TS to be taught is placed on the table T.

When teaching is not performed with another sample, a unique image is obtained based on the captured image in the same manner as in the first embodiment, and the unique image is stored in the unique image data storage device 7 as teaching data. (8-9). The illumination conditions and table movement conditions may be prepared in advance by using variation data, or may be input by the user from the input device 9 as needed. When the teaching operation is repeated, the teaching operation is repeated to acquire a plurality of images of the teaching sample TS.

FIGS. 9A and 9B show examples of variations. In the example shown in the figure, a cylindrical part is used as the teaching sample TS. In FIG. 9A, a1 is an image when the teaching sample TS is directly below the camera 1. In the figure, a2 is an image when the camera 1 is located below the teaching sample TS. In the figure, a3 is an image when the camera 1 is located above the teaching sample TS. By intentionally creating these images by moving the table T, even if a slight positioning error occurs, a false report as a defect is not issued at the time of inspection.

In FIG. 9B, b1 indicates a case where strong teaching is applied to the teaching sample TS, and the outline is only blurred when the teaching sample TS is directly below the camera 1. In the figure, b2 and b3 are images when the teaching sample TS is illuminated obliquely. It is possible to cope with variations in the inclination at the time of setting the teaching sample TS depending on the irradiation direction of the illumination.

Embodiment 4 Next, an image processing apparatus according to a fourth embodiment will be described with reference to the drawings. FIG. 10 is a configuration diagram of the image processing apparatus according to the present embodiment. This image processing apparatus is similar to the image processing apparatus according to the first embodiment, except that a positioning template data storage device 15 and an area-of-interest extraction circuit 16 that extracts and transfers only an image of an inspection area of interest.
Is added. Note that the CPU 5 constitutes a matching data generating means.

FIG. 11 is a flowchart showing a flow of processing of the image processing apparatus according to the present embodiment. The teaching work is performed as follows. A plurality of non-defective samples for determining the position of the inspection area are collected, and one of them is selected and installed in the same environment as the inspection. The non-defective sample is taken by the camera 1 (11-1), and the image is stored in the frame memory 4 and displayed on the display 3. The user determines the position of the inspection area from the image while looking at the image displayed on the display 3 to determine the position of the inspection area by using a cursor or numerical input (11-2).

There are various positioning methods. Here, a positioning method based on the template matching method will be described. A template image area of the inspection area is cut out from the image specified by the user, template matching data is generated, and stored in the positioning template data storage device 15 and, at the same time, stored in the memory 6 as a non-defective image (11-3). Next, a plurality of defect inspection samples are prepared, sequentially placed on the table T, captured by the camera 1, and an image is input by the image input device 2.
(11-4, 11-5).

An inspection area is positioned from each of the defect inspection sample images using template matching data, and the inspection area is automatically cut out by the template matching method and stored in the memory 6 as a non-defective image (11-6). ). This is repeated to obtain a plurality of images of the inspection area (11-7). It is determined whether or not the teaching is to be terminated (11-7). If the teaching is determined to be completed, a unique image is obtained by using the images of the plurality of inspection areas and stored as inspection unique image data (11-7). -8). If the teaching is continued, the process returns to step 11-4, and the defect sample is imaged.

Next, an inspection mode is entered using the unique image data for inspection. In the inspection mode, the inspection object is set, the image is captured by the camera 1, and an image is input (11-9). Next, the position of the inspection area (area of interest) is cut out from the image of the inspection object using the positioning data of the inspection area stored at the time of teaching by the template matching method and stored as an inspection image in the memory (11-10, 11-). 11).

An approximate image is generated using the inspection specific image, a difference image from the input image (inspection image) is generated, defects are extracted, and pass / fail judgment is performed (11-12, 11-13). If the inspection is completed, the process is terminated. If the inspection is not completed, the process returns to step 11-9 to image the next inspection object (1).
1-14).

FIG. 12 is a specific inspection sample illustrating a defect inspection of a component having a hole formed at a predetermined position on a T-shaped plate. In order to perform teaching, a plurality of inspection objects are first collected for teaching. One of them is imaged as a positioning sample of the inspection area, and the positioning area is determined and cut out. In FIG. 12, a square area centered on the hole is specified. This area is set as an inspection area.

Here, the area used for positioning and the inspection area are the same area, but they may be different areas. In that case, the relative coordinates of the positioning area and the inspection area are stored, and after the positioning, the inspection area is cut out according to the relative coordinates and the following processing is performed. Eigenvector generation processing for extracting a common component from the extracted image of the inspection area is performed.

The following processes are performed in the same manner as in the first embodiment.
Inspection eigenvector data is generated and stored. When performing an inspection, an image of an inspection object is captured, and an inspection area is cut out of the image using the positioning data. The following inspection processing is performed in the same manner as in the first embodiment.

Embodiment 5 Hereinafter, an image processing apparatus according to the fifth embodiment will be described with reference to the drawings. FIG. 13 is a configuration diagram of the image processing apparatus according to the present embodiment. This image processing apparatus has a configuration in which an image normalization circuit 17 as a normalizing means for normalizing an image signal sent from the image input apparatus 2 to the frame memory 4 is added to the image processing apparatus according to the first embodiment. .

FIG. 14 is a flowchart showing the flow of the processing of the image processing apparatus. The teaching work is performed as follows. Collect multiple non-defective samples of the inspection object. Place each sample in the same environment as the test. After imaging the non-defective sample with the camera 1, the images of these non-defective samples are
A normalization process is performed by the image capturing image normalization circuit 17 (1).
4-1 and 14-2). Here, the normalization means the normalization of the image density. The standard deviation and the average density are aligned with the first image. Normalize image to obtain unique image Frame memory 4
(14-3). Hereinafter, the eigenvectors are obtained and stored as inspection data as in the first embodiment.

Next, the inspection mode is entered. In the inspection mode, an inspection object is set and an image is input (14-4). After normalizing the input image in the same manner as in teaching, the image is stored in the frame memory 4 (14-5). Hereinafter, similar to the first embodiment, an approximate image is generated from the unique image (14-6), and a difference image from the inspection image is generated (14-7).

After the binary image of the difference image is created, the area of the inspection object part is obtained (14-8, 14-9). It is determined whether or not the obtained area exceeds a threshold value, and defect extraction and pass / fail determination are performed (14-10). If the threshold value is exceeded, it is determined that there is no defect (14-12), and if it does not exceed the threshold value, it is determined that there is a defect (14-11).

FIG. 1 is a specific sample illustrating a defect inspection of a part having a hole formed at a predetermined position on a T-shaped plate.
5 As shown in the figure, the density distribution of the input teaching image or inspection image changes due to a change in environment (weather, time, etc.) at the time of imaging.

By normalizing this, an image having the same density distribution as the first image is generated. By using the normalized image, a highly reliable defect inspection that is not affected by changes in the environment at the time of imaging can be expected.

Embodiment 6 FIG. Hereinafter, an image processing apparatus according to Embodiment 6 will be described with reference to the drawings. FIG. 16 is a configuration diagram of the present image processing apparatus. In this image processing apparatus, a parameter space (class) feature quantity calculating circuit 18 for obtaining the center of gravity and range of non-defective products in the eigenspace and a non-defective feature quantity storage device 19 for storing the calculation result are added to the image processing apparatus according to Embodiment 1. Configuration.

FIG. 17 is a flowchart showing the flow of processing of the image processing apparatus according to the present embodiment. First, the teaching operation is performed as follows. Until a unique image of a non-defective sample of the inspection object is obtained, processes 17-1 and 17-2 perform the same processes as in the first embodiment. Next, as shown in FIG. 18, the center of gravity and range of the coordinates of all the images are obtained and stored in the non-defective feature amount storage device 19 as non-defective data (17-3, 17-).
4).

Next, the inspection processing method will be described. Until an approximate image of the inspection image is generated, the same processing as in the first embodiment is performed along 17-5 and 17-6. Thereafter, the coordinates of the inspection image in the eigenspace are obtained (17-7). Next, comparison with the coordinates of the non-defective data is performed (17-8), and if the coordinates in the unique space of the inspection image are within the range of the coordinates of the non-defective data, it is determined to be non-defective without defects (17-10). If it is out of the range, it is determined that there is a defect and is defective, and the result is shown to the user (17-9).

Embodiment 7 FIG. Next, an image processing apparatus according to Embodiment 7 will be described with reference to the drawings. FIG. 19 is a configuration diagram of the image processing apparatus according to the present embodiment. This image processing apparatus has a configuration in which a Mahalanobis distance calculation circuit 21 is added to the image processing apparatus according to Embodiment 6 as Mahalanobis distance calculation means for normalizing a distance in an eigenspace.

FIG. 20 is a flowchart showing the flow of the processing of this image processing apparatus. First, the teaching operation is performed as follows. Processes 20-1 to 20-3 from capturing a teaching image to obtaining a unique image from this image and obtaining the barycenter of the coordinates of all the images of the teaching sample of the inspection object are the same as those in the sixth embodiment. . After this processing, the center of gravity and the covariance matrix of the non-defective class (teaching image class) are obtained and stored in the conscience data storage unit 19 as non-defective data (20-
3).

Next, the inspection processing method will be described. Processing 20-3 to 20- until an approximate image of the inspection image is generated
5 performs the same processing as in the sixth embodiment. Then, the distance between the good class (teaching image class) and the inspection image class in the eigenspace is calculated using the Mahalanobis distance (20).
-6), if the magnitude of the distance is within the range as shown in FIG. 21, it is determined to be good, and if it is out of the range, it is determined to be defective and shown to the user.

Mahalanobis distance Assuming that each class follows a multidimensional normal distribution, the Mahalanobis distance gives the most natural distance from some data to the class. That is, the Mahalanobis distance from the image xi to the class μ is expressed as di in Equation 5 as a distance in a space in which the influence of the inherent variance of the class is normalized.
Is determined by

[0087]

(Equation 4)

Ω _jk is the (j, k) element of the inverse matrix W = V ⁻¹ of the variance-covariance matrix V of the class. n is the number of spatial dimensions. Takagi, Shimoda
Supervision: Image Analysis Handbook, University of Tokyo Press, 19
There are 91.

Embodiment 8 FIG. Next, an image processing apparatus according to Embodiment 8 will be described with reference to the drawings. FIG. 22 is a configuration diagram of the image processing apparatus according to the present embodiment. This image processing apparatus has a configuration in which a failure data storage device 21 for storing failure data is added to the sixth embodiment. Note that the CPU 5 constitutes a class calculation means.

FIG. 23 is a flowchart showing the flow of a teaching process of the image processing apparatus according to the present embodiment. First,
The teaching work is performed as follows. Until the non-defective sample (teaching image) of the inspection object is taken in and the unique image is obtained, 23
The processing of -1, 23-2 is the same as that of the first embodiment. Further, after capturing the image of the inspection object (23-
2) The inspection mode (23-4) for determining whether each image is a non-defective product or a defective product is the same as that described in each embodiment.

The user selects a non-defective or non-defective sample from the results of the non-defective / defective judgment and inputs the sample from the input device (23-5). Further, the center of gravity and range of the coordinates of all the images are obtained, and it is determined whether the data is good data or defective data (23-6). If the image is non-defective, the image is added to the non-defective class and stored in the non-defective data storage device 19 (23-7). If it is defective, the image is stored in the defective data storage device 21 in addition to the class of the defective product (23-8).

Next, a method of the defect inspection processing will be described.
Until an approximate image of the inspection image is generated, the same processing as in the sixth embodiment is performed. After that, the coordinates in the unique space are obtained, compared with the above-mentioned non-defective data, judged as non-defective if within the range of non-defective products, and compared with defective data if out of range. Is determined and shown to the user (23-9, 23-10).

If neither is included, the user selects non-defective / defective products in the same manner as the teaching operation, and asks the input device to input the selection result to the CPU (23-5). If the selection result is a non-defective product, the image to be inspected is added to the non-defective class (23-7). If the selection result is defective, the inspection image is added to the defective class (23-
7).

Embodiment 9 FIG. Next, an image processing apparatus according to Embodiment 9 will be described with reference to the drawings. FIG. 25 is a configuration diagram of the image processing apparatus according to the present embodiment. This image processing apparatus has a configuration in which a unique image contribution rate state check function 22 as contribution rate state determination means and a contribution rate recording apparatus 23 as contribution rate recording means are added to the first embodiment. In addition,
The CPU 5 constitutes a re-teach warning unit.

FIG. 26 is a flowchart showing the flow of the teaching process of the image processing apparatus according to the present embodiment. The teaching operation (26-1) is performed in the same manner as in the first embodiment. Inspection (2
6-2) is performed as follows. An approximation image is generated by imaging the object to be inspected and using a unique image taught in advance. The contribution ratio indicating the ratio of the contribution of the unique image used to generate the approximate image to the generation of the approximate image is stored in the contribution ratio recording device (26-3).

Next, the contribution data of the past t1 times (for example, t1 = 16) is loaded by the state check function of the contribution recording (26-4). FIG. 27 shows an example of the contribution ratio.
In the example shown in the figure, the contribution rate is initially 70% or more and is stable, but the contribution rate suddenly drops in the middle and remains low. In such a case, it can be estimated that the object has changed, so that re-teaching can provide a more accurate inspection (26-
5).

Therefore, a warning is issued when the contribution rate is low continuously, for example, when the contribution rate falls below 70% five times or more consecutively (26-6). When a warning is issued, the user once ends the inspection process and starts teaching again (26-
7) Update the unique image (26-8). Then, it is determined whether or not to continue the inspection (26-9). If the inspection is to be continued, the process returns to 26-1 to perform the teaching operation. However, if it does not continue, the process ends.

Embodiment 10 FIG. Next, an image processing apparatus according to Embodiment 10 will be described with reference to the drawings. FIG. 28 is a configuration diagram of the image processing apparatus according to the present embodiment. This image processing apparatus has a configuration in which a correlation value state check circuit 24 as correlation value state determination means and a positioning correlation value recording device 25 as correlation value recording means are added to the first embodiment.

FIG. 29 is a flowchart showing the flow of a teaching process of the image processing apparatus according to the present embodiment. The teaching operation (29-1) is performed in the same manner as in the first embodiment. The inspection is performed as follows. The object to be inspected is imaged, and the inspection area is positioned using template data taught in advance (29-2). The correlation value calculated at the time of template matching is stored in the correlation value recording device 25 (29-3).

Next, the correlation value data of the past t1 times (for example, t1 = 16) is loaded by the correlation value recording state check function (29-4). Fig. 3 shows an example of time history of correlation values
0 is shown. In the example shown in the figure, the correlation value is initially 0.7 or more and is stable, but the correlation value suddenly drops in the middle and remains low. In such a case, it can be estimated that the target object has changed (29-5), so that the correct positioning can be achieved by re-teaching.

Therefore, a warning is issued when the correlation value is continuously low, for example, when the correlation value is less than 0.7 for five consecutive times (29-6). The teaching process of the positioning template is performed (29-7), and the positioning template is updated (29-8). Then, it is determined whether or not to continue the inspection (26-9).
Then, the teaching operation is performed (29-1). However, if it does not continue, the process ends.

Embodiment 11 FIG. Next, an image processing apparatus according to Embodiment 11 will be described with reference to the drawings. FIG. 31 is a configuration diagram of the image processing apparatus according to the present embodiment. This image processing apparatus is different from the first embodiment in that a teaching timing data storage device 28 in which teaching timing data is stored in advance, a timer 26 that counts time with the operation of the image processing device, and lot replacement timing information Device controller 27 that outputs
A communication management device 29 for inputting information output from the CPU 5 to the CPU 5 is provided.

The CPU 5 checks the teaching timing data stored in the teaching timing data storage device 28 with the lot replacement timing information output from the device controller 27, and performs a teaching process if the timings match. Alternatively, the CPU 5 inputs time data from a timer and performs a teaching process when a preset time comes.

FIG. 31 is a flowchart showing the flow of the teaching process of the image processing apparatus according to the present embodiment. The teaching operation (32-1) is performed in the same manner as in the first embodiment. After performing the inspection process (32-2) as in the first embodiment, the time of the timer 26 is checked to perform the teaching timing check (32-3), and the lot exchange timing information from the device controller 27 is fetched. The equipment lot status is checked (32-4). Then, it is determined whether it is the teaching timing by comparing the teaching timing data or the time by the timer (32-5). If it is not the teaching timing, the teaching processing is performed without newly taking in the inspection object, and the unique image is updated (32-6, 32-7). After the teaching data is updated, the process returns to 32-1 to continue the inspection process.

As a result, by communicating with the device controller 27, the teaching image is acquired at the timing when the variation of the sample is likely to occur, such as when the device is started up, restarted, or when the lot is changed. As a result, it is possible to obtain an image including the variation, thereby reducing false reports of defect occurrence and improving the accuracy of extracting a defective portion.

[0106]

According to the first aspect of the present invention, since a plurality of images are taught, not only the movement of the object to be inspected but also the variation of the shape and the appearance of the object can be accommodated. The presence / absence of a defect can be determined, and the binarization and particle analysis processing of the image to be inspected are performed, so that a local abnormality (defect) can be detected.

According to the second aspect of the present invention, since the relationship between time and reliability is shown to the user in advance at the time of teaching, there is an effect that the optimum number of unique images can be determined.

According to the third aspect of the present invention, by placing a sample on the teaching table and controlling the camera, the illumination and the table, it is possible to generate and teach in advance the variations expected during the inspection, so that the object can be taught. There is an effect that only defects can be correctly extracted even if there is variation in the manufacturing process.

According to the invention of claim 4, there is an effect that a defect can be accurately extracted by performing positioning in advance. Further, there is an effect that false information can be reduced by processing only a necessary area.

According to the fifth aspect of the present invention, since the inspection is performed after normalizing the density of the input image, there is an effect that erroneous recognition due to the density fluctuation of the input image can be reduced.

According to the sixth aspect of the present invention, the pass / fail judgment is made at the stage of projecting onto the eigenspace, so that there is an effect that the calculation time can be reduced.

According to the seventh aspect of the present invention, since the eigenspace of an image is represented by the Mahalanobis distance, there is an effect that the criterion can be set uniformly and can be easily handled.

According to the eighth aspect of the present invention, since the pass / fail judgment is made based on the distance in the space where the image is mapped to the eigenspace and divided into clusters, there is an effect that the setting of the threshold value becomes unnecessary.

According to the ninth aspect of the present invention, it is possible to correct the eigenvector to an appropriate one by judging the validity of the eigenvector from the change in the contribution rate and warning of re-teaching, so that the defective product This has the effect of reducing false reports.

According to the tenth aspect of the present invention, the validity of the template image used for positioning is determined from the change in the correlation value, and the positioning template image is corrected to an appropriate one by warning of re-teaching. Therefore, there is an effect that false recognition of a defect can be reduced.

According to the eleventh aspect of the present invention, since the teaching image is automatically acquired during the inspection processing, the first teaching is one.
There is an effect that the teaching work is simple with a sample. In addition, if an image is taken in at the time of lot exchange, an image including the variation as a product of the inspection object can be obtained, so that there is an effect that false information is reduced and the accuracy of extracting a defective portion is improved.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an image processing apparatus according to a first embodiment.

FIG. 2 is a flowchart illustrating the operation of the present embodiment.

FIG. 3 is a diagram illustrating a defect detection method according to the present embodiment.

FIG. 4 is a configuration diagram of an image processing apparatus according to a second embodiment.

FIG. 5 is a flowchart illustrating the operation of the present embodiment.

FIG. 6 is a diagram when a contribution ratio of a unique image according to the present embodiment is displayed on a screen.

FIG. 7 is a configuration diagram of an image processing apparatus according to a third embodiment.

FIG. 8 is a flowchart illustrating the operation of the present embodiment.

FIG. 9 is a diagram illustrating a variation in a display screen according to the present embodiment.

FIG. 10 is a configuration diagram of an image processing apparatus according to a fourth embodiment.

FIG. 11 is a flowchart illustrating the operation of the present embodiment.

FIG. 12 is a diagram illustrating a defect detection method according to the present embodiment.

FIG. 13 is a configuration diagram of an image processing apparatus according to a fifth embodiment.

FIG. 14 is a flowchart illustrating the operation of the present embodiment.

FIG. 15 is a diagram illustrating a defect detection method according to the present embodiment.

FIG. 16 is a configuration diagram of an image processing apparatus according to a sixth embodiment.

FIG. 17 is a flowchart illustrating the operation of the sixth embodiment.

FIG. 18 is a diagram illustrating a method for obtaining the center of gravity and range of an image according to the present embodiment.

FIG. 19 is a configuration diagram of an image processing apparatus according to a seventh embodiment.

FIG. 20 is a flowchart illustrating the operation of the present embodiment.

FIG. 21 is a diagram illustrating a method for obtaining a Mahalanobis distance according to the present embodiment.

FIG. 22 is a configuration diagram of an image processing apparatus according to an eighth embodiment.

FIG. 23 is a flowchart illustrating the operation of the present embodiment.

FIG. 24 relates to the embodiment;

FIG. 25 is a configuration diagram of an image processing apparatus according to Embodiment 9;

FIG. 26 is a flowchart illustrating the operation of the present embodiment.

FIG. 27 is a graph showing the degree of change in the contribution ratio with respect to the transition of the teaching time according to the present embodiment.

FIG. 28 is a configuration diagram of an image processing apparatus according to a tenth embodiment.

FIG. 29 is a flowchart illustrating the operation of the present embodiment.

FIG. 30 is a graph showing the degree of change in the rate of change of the correlation value of template matching with respect to the transition of the teaching time according to the present embodiment.

FIG. 31 is a configuration diagram of an image processing apparatus according to an eleventh embodiment.

FIG. 32 is a flowchart illustrating the operation of the present embodiment.

FIG. 33 is a configuration diagram of a conventional image processing apparatus.

FIG. 34 is a flowchart illustrating an operation of a conventional image processing apparatus.

[Explanation of symbols]

1 camera, 2 input devices, 4 frame memory, 5
CPU, 7 unique image data storage device, 10 unique image contribution ratio calculation circuit, 11 illumination, 12 condition setting data storage device, 13 illumination controller, 14 table controller, 15 template data storage device, 16 target extraction circuit, 17 image normal Conversion circuit, 18 class feature quantity calculation circuit, 19 good feature quantity storage device, 20 Mahalanobis distance calculation circuit, 21 defective data storage device,
22 contribution ratio recording / status check circuit, 23 contribution ratio recording device, 24 correlation value recording / status check circuit, 25 correlation value recording device, 27 device controller, 28 teaching timing data storage device.

Claims (11)

[Claims] An imaging unit configured to individually capture a teaching sample and an inspection target; a unique image data generating unit configured to generate unique image data from a teaching image that is an image of the teaching sample captured by the imaging unit; Approximation image restoration means for restoring an approximation image from an inspection image which is an image of the imaged inspection object and the generated unique image data; and a difference image which generates a difference image between the inspection image and the approximation image. An image processing apparatus comprising: a generating unit; a binarizing unit configured to binarize the generated difference image; and a determining unit configured to perform a particle analysis on the binary image and determine whether the inspection object is good or not. . 2. A calculation for calculating a relationship between the number of unique image data used for restoring an approximate image and an inspection processing time, or a relationship between the number of unique image data and an inspection reliability represented by a contribution ratio. 2. The image processing apparatus according to claim 1, further comprising: a display unit having a function of simulating a user at the time of teaching to take in an image of the teaching sample, the relationship calculated by the calculating unit. . 3. A moving table for moving the teaching sample, illumination for irradiating the teaching sample from a predetermined angle, a controller for controlling the movement of the moving table, and a controller for controlling the irradiation intensity of the illumination. A teaching image variation generating means for issuing a command to these controllers to change an illumination position and an illumination intensity with respect to the teaching sample to generate a teaching image having a variation between the images. An image processing apparatus according to claim 1. 4. A matching data generating means for determining an inspection area in an inspection image which is an image of an inspection object, generating and storing template matching data based on the inspection area, and a captured image of a plurality of teaching samples. A unique image data generating unit that cuts out each inspection region from the teaching image based on the template matching data, and generates unique image data from the image data of the cut out inspection region; and an inspection target that is cut out by the inspection region cutting unit. Approximation image restoration means for restoring an approximate image from an inspection area image of an image and the generated unique image data; difference image generation means for generating a difference image between the inspection area image of the image to be inspected and the approximate image; A binarizing means for binarizing the generated difference image; and a particle analysis of the binary image and the inspection object The image processing apparatus characterized by comprising a quality determining means. 5. The image processing apparatus according to claim 1, further comprising a normalizing means for normalizing the density of each of the captured teaching images. 6. A class calculating means for obtaining a feature amount of a class in an eigenspace using a taught image used for teaching as a non-defective class, and using a coordinate in the eigenspace and a feature amount of the class when an inspection image is given. The image processing apparatus according to claim 1, further comprising: a quality determination unit configured to determine quality of the inspection target. 7. A Mahalanobis distance calculation means for calculating a distance between a class on an eigenspace and a certain point using a Mahalanobis distance.
An image processing apparatus according to claim 1. 8. Dividing means for dividing a teaching image vector into a good class and a defective class in an eigenspace, distance calculating means for calculating a distance to each class in the eigenspace, and a distance calculated by the calculated distance The image processing apparatus according to claim 6, further comprising a quality determination unit configured to determine quality of the inspection target. 9. A contribution rate recording means for recording a contribution rate of each unique image data at the time of generating an approximate image, a contribution rate state determination means for judging the validity of a teaching image from a change rate of the contribution rate, and a contribution rate validity. The image processing apparatus according to claim 2, further comprising: a re-teach warning unit that issues a warning to newly teach a target object when it is determined that the object is bad. 10. A correlation value recording means for recording a correlation value of template matching when cutting out an inspection area image from an image to be inspected by template matching, and judging the validity of the teaching image from a change rate of the correlation value. 5. The method according to claim 4, further comprising: a correlation state determination unit that performs a correlation state determination, and a re-teach warning unit that issues a warning so as to newly teach a teaching sample when the correlation value is determined to be degraded. Image processing device. 11. The image processing according to claim 1, wherein a teaching image is acquired from the inspection target at regular intervals during the inspection operation or in conjunction with replacement of the component supply lot to be inspected. apparatus.