WO2012139088A2

WO2012139088A2 - System and method for generating multiple, interlaced images using a single scanning camera with multiple, alternating light sources

Info

Publication number: WO2012139088A2
Application number: PCT/US2012/032682
Authority: WO
Inventors: Wei-Song TAN; David KOSEN; Thomas R. NEWTON; Steven R. WACHTEL; Agustin DAZA
Original assignee: ACCUSENTRY Inc
Current assignee: ACCUSENTRY Inc
Priority date: 2011-04-08
Filing date: 2012-04-09
Publication date: 2012-10-11
Anticipated expiration: 2013-10-08
Also published as: WO2012139088A3

Abstract

A system and method for generating multiple, interlaced images using one scanning camera with multiple, alternately strobed light sources. The system is comprised of a CCD line scanning camera, two or more strobed light sources, an image processing computer, and electronic circuitry facilitating the synchronization of the CCD line scanning camera and the light sources. As the object of interest is imaged by the line scanning camera, the light sources are individually and sequentially strobed as each line is exposed. The resulting electronic image will contain interlaced lines, with each line illuminated by only one of the light sources. The resulting image will then be de-interlaced by the image processing computer to produce multiple, separate images concordant with the number of light sources, with each image illuminated by a single light source.

Description

SYSTEM AND METHOD FOR GENERATING MULTIPLE, INTERLACED IMAGES USING A SINGLE SCANNING CAMERA WITH MULTIPLE, ALTERNATING LIGHT

SOURCES

[0001 ] CROSS REFERENCE TO PRIOR CO-PENDING PATENT APPLICATIONS

[0002] This application claims the benefit of prior co-pending US Provisional Patent Application Serial Number 61 /51 6,821 filed on April 8, 201 1 .

[0003] FIELD OF THE INVENTION

[0004] The invention relates generally to the field of manufacturing and particularly to machine vision, i.e. the systems and methods or imaging products for the purposes of product inspection.

[0005] BACKGROUND

[0006] Product inspection during the manufacturing process is a critical component to ensure a consistent, quality product. For high-speed production, it is often necessary to employ an automated machine vision system. Such a system will allow not only the detection of a bad product, but identify process problems more quickly. This allows corrections and adjustments to the manufacturing process which will serve to increase the machine production capability and reduce waste. There are many machine vision systems which are capable of high-speed product inspections.

[0007] As the demand for product quality and consistency increases, it is often desirable to employ multiple cameras to inspect the product not only at multiple stages of manufacture, but also under different lighting conditions. As the number of cameras increases, it not only increases capital expenditure, but it also adds complexity. Each camera must be mounted and maintained. Each camera requires a lens, power supply, communication cable and controller card in the image processing PC. Each camera must interface with the product line's programmable logic controller (PLC) for the purposes of triggering the camera and sending reject signals. Whether the machine vision systems are installed at the original equipment manufacturer (OEM) or as an aftermarket upgrade, physical space is generally a constraint.

l [0008] US Patent 6,493,079 discloses a machine vision system, which instead of viewing a single face of a cube, for example, the patent suggests orientation of the camera such that three faces are visible by positioning a corner of the cube nearest the lens. The image is subsequently divided up to allow each face to be processed separately.

[0009] US Patent 7,030,400 discloses an approach which is nearly the opposite of the instant invention. Instead of using two lights with a single camera, the authors of this patent propose to use a single light with two cameras. One camera observes the transmitted light through the web, while the other observes the reflected light off of the web. This method is simpler than the invention presented herein, but is limited to acquiring only front and backlit images from the same spectrum of light.

[0010] SUMMARY OF THE INVENTION

[001 1 ] In the exemplary embodiment of the instant apparatus, many of the above mentioned costs and complexities can be avoided. For example, it is commonly known that some physical qualities of the product, such as thickness and density, are best imaged by transmitted light, while other characteristics, such as color and surface imperfections, are best imaged by reflected light. Prior solutions require the production line to have physical space to mount two light sources and two cameras, with all the required accoutrements. The instant apparatus eliminates one of these cameras and its associated components. If three lights are to be employed, the benefit of the instant apparatus is correspondingly greater.

[0012] Beyond any obvious cost benefit to the instant apparatus, additional information can be obtained during processing of the acquired images. In machine vision, products are often inspected or processed by, for example, detecting the edges of a particular component and then measuring the distance between the edges. Specifically, in absorbent hygiene product inspections, typically a product is inspected during manufacture as a continuous web, prior to being cut into discrete items. When the location, or phase, or components are measured relative to the "cut line", a certain degree of error is inevitably associated with the placement of this outline in the image processing software. Using the instant apparatus to image an absorbent hygiene product under multiple lighting conditions, the error in placement of the outline between images from each camera is eliminated. Additionally, it will be possible to measure the distance from a component visible only under one type of lighting to another component visible under another type of lighting. These benefits are uniquely present when utilizing the instant apparatus because in each image of the product under different types of lighting, the product is in the same exact position, orientation, and time.

[0013] The preferred embodiment of the present invention can comprise a system and method for generating multiple, interlaced images using one line scan camera with multiple, alternating light sources. The system can include a CCD line scan camera, two or more strobed light sources, an image processing computer, and electronic circuitry facilitating the synchronization of the CCD line scan camera and the light sources. This apparatus is used for machine vision product inspections where different physical qualities of a product are revealed under different spectrums, orientations, angles, focuses of light, or any combination thereof. The apparatus allows the product to be inspected under multiple light sources using only a single camera. This reduces cost, complexity and required space on a manufacturing production line.

[0014] The product to be inspected can be either continuous or discrete.

[0015] This system can be extended to a system wherein the image processing computer processes images gathered from multiple cameras.

[0016] This method can be employed where the light spectrums are visible, ultra-violet, or infrared.

[0017] This method can be extended to application in which one or more lights sources are not strobed, but constantly powered.

[0018] Furthermore, the invention is not limited to use with line scan cameras. For example, an area scan camera in which multiple lines or rows are imaged at the same time can be employed. This type of camera can be employed to sequentially image a two dimensional sections or blocks of a product under inspection, and these sequential images can be combined into a single interlaced images in which the representative data or dimensions of the multiple sections retain the same relationship by virtue of use of the same camera, in the same manner as discussed with reference to the exemplary embodiment employing a line scan camera.

[0019] An area scan camera can also be employed in one additional manner. Instead of sequentially imaging sections of a product, under different lighting conditions, an area scan camera can be employed with multiple light sources illuminating different sections at the same time. Thus when the area scan camera is activated, the camera itself or other control components of the system can activate multiple light sources. The resultant, multi-section image can then be processed by essentially the same image processing computer or software as was employed in processing the sequentially illuminated and interlaced images described with reference to the exemplary embodiment depicted herein.

[0020] Instead of employing an area scan camera, a line scan camera of the type employing multiple lines sensors can be employed in the same manner as would an area scan camera.

[0021 ] BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a block diagram of interlaced lines can camera and light arrangement.

[0023] Figure 2 is a description of components of a representative product to be imaged.

[0024] Figure 3 is an illustration of the imaging of a product using the instant system and method.

[0025] Figures 4A and 4B are illustrations of the resulting de-interlaced images using white light Figure 4A and UV light Figure 4B.

[0026] Figure 5 shows an interlaced image acquired using two lights (one front light and one back light) and the instant system and method for capturing interlaced line scan camera images. [0027] Figure 6 shows an enlarged portion of interlaced image acquired using two lights (one front light and one back light) and the instant system and method for capturing interlaced line scan camera images.

[0028] Figures 7A and 7B are de-interlaced images from Figure 6 exhibiting front lighting to the left and back lighting to the right.

[0029] Figure 8 is a layout of the machine vision system architecture, referred to herein as SVC, in which at least one line scan camera employs the method and apparatus of the present invention.

[0030] Figure 9 is a view of an alternate embodiment of this invention in which an area scan camera is used instead of a line scan camera as shown in Figure 3.

[0031 ] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] As shown in Figure 1 , the preferred embodiment of the system 1 consists of an image processing computer 2, line scan camera 3, two or more lights 4 and 5, light power supply and controller 6, camera power supply 7 and encoder 8. When the product is in position to be imaged, the production line PLC 9 will send a trigger signal to the image processing computer 2. The image processing computer 2 will in-turn instruct the line scan camera 3 to begin acquiring an image. At this point, the line scan camera 3 will wait for each line trigger signal to be sent from the encoder 8 by way of the camera power supply and the light power supply and controller 6. The light power supply controller 6 will alternately strobe either the Light 4 or the Light 5. In general, Light 4 will be strobed as the line scan camera 3 exposes the first line of the image. Light 5 will be strobed as the line scan camera 3 exposes the second line of the image. Light 4 will be strobed as the line scan camera 3 exposes the third line of the image, and Light 5 will be strobed as the line scan camera 3 exposes the fourth line of the image, and the lines will continue to be exposed in an alternating manner. This sequence will continue until the predefined number of lines composing the full image is acquired. The resulting image is sent back to the image processing computer 2 for inspection. [0033] When the product show in Figure 2 is being imaged by a single line scan camera 3 with two lights, one white light 4 and one UV light 5. This arrangement is displayed in Figure 3. In this particular example, the white light and UV light are both front lights. Using the instant system and method, the inter-laced image is generated by alternately strobing the white light and UV as the product moves across the camera's field of view. This raw image is sent to the image processing computer, where it is de-interlaced into two separate images. As displayed in Figure 4A, one image appears as if it had been captured in the conventional method using white light alone. The second image appears as if it had been captured in the conventional method using UV light alone, as shown in Figure 4B.

[0034] For example, consider the product 10 shown in Figure 2. This product 10 is a feminine hygiene pad. It is composed of four main components: the non- woven 1 1 , elastic 12, core 1 3 and release tape 14. The elastic 12 is secured in position by hot-melt adhesive, which cannot be visually inspected except under ultraviolet light which causes an additive in the adhesive to fluoresce visible light. When inspecting this product 10, under strobed white light alone, the de-interlaced image shows all components except the adhesive 15 as represented in Figure 4A. When inspecting the product 10 strobed by an ultra-violet (UV) light alone, the de-interlaced image shows the adhesive 1 5, which fluoresces under UV light, as well as fainter lines showing other components, but it is not possible to visualize the elastic 12 in Figure 4B. However, it is important to know the relative position of the adhesive 15 relative to the elastic 12 to confirm that the elastic 15 can be properly secured as part of the feminine hygiene pad. Figures 4A and 4B thus represent two different sets of product inspection data or two different product inspection images. Using traditional methods of imaging, it would be possible to capture and process images of each product under each type of light by using two separate cameras, but it would not be possible to measure the relative positions of the adhesive and the elastic within an individual product. Note further that in the UV image, little other identifiable features would be evident and it could not be known if the adhesive was out of position or if the camera trigger signal was out of phase with the product. However, since the relative position of the product relative to the lights and single camera, as well as the timing of the inspection is known and will remain relatively constant, it is now possible to measure the position of the elastic 14 relative to the adhesive 15, especially using image processing software.

[0035] Traditionally, it is very challenging to reliably match up images of a single product taken with different cameras at different stages of a high speed production line. However, using the instant method and apparatus, this problem is no longer existent. For example, referencing the position of the adhesive seen in the UV lit image with the elastic seen in the white lit image becomes a trivial matter since both images are processed in the same computer at the same time.

[0036] An image acquired using a prototype of system and method according to this invention is shown in Figure 5. A small portion of the image has been enlarged in Figure 6 where it is evident that the interlaced image was generated by using two lights. One light provided front illumination, and one light was behind the product to provide back illumination. Figures 7A and 7B show the two resulting de- interlaced images clearly demonstrating the positions of each light.

[0037] With this invention, a line scan camera 3, which will integrate with the machine vision architecture, referred to herein as SVC architecture which includes an image processing computer, and will require no additional controller cards or hardware except for an encoder to correlate the action of the machine vision system with a production line on which it will be employed. For this apparatus to function properly, the timing and synchronization of light strobes 4, 5 and the camera line exposures must be tightly controlled. This is accomplished by using the line scan camera 3 itself to control the timing. The layout for the SVC architecture incorporating this system for generating interlaced images using one line scan camera and multiple, alternating light sources is shown in Figure 8. It will be possible to daisy-chain any combination of area scan cameras, line scan cameras, and line scan cameras with the camera 3, which is used with alternating lights 4 and 5.. When this method is employed, the camera 3 will control the timing of the light switching and line exposures thought the SVC Linescan LED Dual Light Controller 6. The camera head will generate a control signal which will allow the Controller 6 and the SVC Power Supply to ultimately switch the lights.

[0038] The invention is not limited to use with line scan cameras. As shown in Figure 9 an area scan camera 3A in which multiple lines or rows are imaged at the same time can be employed. This type of camera can be employed to sequentially image a two dimensional sections or blocks of a product under inspection, and these sequential images can be combined into a single interlaced images in which the representative data or dimensions of the multiple sections retain the same relationship by virtue of use of the same camera, in the same manner as discussed with reference to the exemplary embodiment employing a line scan camera. In the embodiment of Figure 9, lights 4A and 5A would be activated to illuminate the area to be scanned by area scan camera 3A.

[0039] An area scan camera can also be employed in one additional manner. Instead of sequentially imaging sections of a product, under different lighting conditions, an area scan camera can be employed with multiple light sources illuminating different sections at the same time. Thus when the area scan camera is activated, the camera itself or other control components of the system can activate multiple light sources. The resultant, multi-section image can then be processed by essentially the same image processing computer or software as was employed in processing the sequentially illuminated and interlaced images described with reference to the exemplary embodiment depicted herein.

[0040] Instead of employing an area scan camera, a line scan cameras of the type employing multiple lines sensors can be employed in the same manner as would an area scan camera.

[0041 ] This invention is not limited to the specific embodiments depicted herein and it would be apparent to one of ordinary skill in the art that this system and method could be expanded or improved in the following ways:

[0042] Two or more lights could be used.

[0043] A single multi-spectrum light could be used alone by strobing only certain spectra of LEDs, or in conjunction with single spectrum lights.

[0044] One or more lights could be constantly powered on, as opposed to strobed.

[0045] The de-interlaced images could be captured at different times (frame triggers), camera gains, image dimensions, exposure times or resolutions in the direction of motion from one another. The de-interlaced images can also be of different binings. Binnings means that the charge on two or more pixels on the CCD sensor are added together, so that the image resolution is reduced but the image will be brighter, depending upon how many pixels are added together.

[0046] The image processing computer could communicate with one or more line scan cameras using the instant interlacing method, line scan cameras using traditional lighting, or area scan cameras or any combination thereof.

[0047] The strobing pattern for the light could be selectively adjusted.

[0048] The images could be inspected with or with-out de-interlacing.

[0049] The product to be inspected could be continuous or discrete.

[0050] The system could be used to strobe one or more lights for the purpose of extending the usable lifetime of said lights.

[0051 ] The lights employed in this invention can also be activated before the camera to allow the lights and the illumination to stabilize before the scan or image is generated or to insure that the pulse width activating the lights is of sufficient length, so that a higher quality image may be generated.

[0052] The image processing computer could be included inside the line scan camera, as in a "smart camera" configuration.

[0053] The system can be employed to inspect a continuous product in paper form, such as paper, film, adhesive or no-woven roll goods, and the term product is not limited to discrete product, such as shown in the representative examples.

Claims

WE CLAIM:

1 . A machine vision system for product inspection comprising:

a scanning camera for sequentially scanning a product as the product moves past the line scanning camera;

two individual light sources, each individual light source being activated upon receipt of a signal from the scanning camera, the individual light sources being activated during different light scans, wherein the scanning camera combines individual scans to form an interlaced image;

an image processing computer receiving the interlaced image and generating two de-interlaced images, wherein the two de-interlaced images are processed to yield separate product inspection images, with a relationship between the separate inspection images on separate de-interlaced images are related because the de- interlaced images were originally generated using the same scanning camera.

2. The machine vision system of Claim 1 wherein the two individual light sources are sequentially, alternately activated so that adjacent scans on the interlaced image are generated using different light sources.

3. The machine vision system of Claim 1 wherein one light source is visible light and the other light source is a non-visible light source.

4. The machine vision system of Claim 1 wherein one light source is positioned to shine through the product and be incident upon the scanning camera and the other light source is position so that light reflected from the product is incident upon the scanning camera.

5. The machine vision system of Claim 1 wherein both light sources are activated at the same time during selected scans forming the interlaced image.

6. The machine vision system of Claim 1 wherein the scanning camera comprises a line scan camera.

7. The machine vision system of Claim 1 wherein the scanning camera comprises an area scan camera.

8. A method of inspecting a product including multiple product components as the product moves along a production assembly line to determine if all components are present and to determine the dimensional relationship between separate components; comprising the steps of: illuminating portions of the product by at least two light sources with one of the light sources being only intermittently activated;

generating an interlaced image comprising multiple scans generated by a single camera, different portions of the image being generated under different lighting conditions; and

deinterlacing the interlaced image to form at least two separate de-interlaced images of the product with the dimensional relationship between components being the same on both de-interlaced images.

9. The method of Claim 8 wherein the deinterlaced images are inspected by an image processing computer.

10. The method of Claim 8 wherein the interlaced image is generated by a single line scan camera in which different scanned lines are generated under different lighting conditions.

1 1 . The method of Claim 8 wherein the interlaced image is generated by a single area scan camera in which different sections scanned by the single area scan camera are generated under different lighting conditions.

12. The method of claim 8 wherein at least part of the interlaced images and at least one of the de-interlaced images is formed by light fluorescing from a product component when illuminated by the intermittently activated light source.

13. A machine vision system for product inspection comprising:

two separate light sources, at least one of the light sources being

intermittently activated;

a single line scan camera sequentially activated to generate an interlaced electronic image comprising a series of line scans, selected line scans synchronized with intermittent activation of the one of the light sources with other line scans occurring at times when the at least one of the light sources is unactivated; and

an image processing computer receiving the interlaced electronic image and de-interlacing the interlaced electronic image to generate two separate de-interlaced electronic images so that the two de-interlaced electronic images can be processed by the image processing computer to yield product inspection data in which a constant relationship between date in the two de-interlaced electronic images is maintained because both de-interlaced images were formed by only the single camera.

14. The machine vision system of Claim 13 wherein both of the two separate light sources are intermittently activated, activation of separate light sources occurring at different, sequential times.

15. The machine vision system of Claim 14 wherein each of the two separate light sources is activated only when the other light sources is unactivated.

16. The machine vision system of Claim 14 wherein the two separate light sources are strobed at different times.

17. The machine vision system of Claim 14 wherein one of the light sources is a source of visible light and the other of the light sources is a source of ultra-violet light.

18. The machine vision system of Claim 14 wherein the two separate light sources are sequentially activated.

19. The machine vision system of Claim 13 wherein the single line scan camera is fixed relative to product moving on a moving production line.

20. The machine vision system of Claim 13 wherein one light source is positioned so that reflected light from of a product is incident upon the single line scan camera and the other light source is positioned so that light transmitted through the product is incident upon the single line scan camera.

21 . The machine vision system of Claim 13 wherein light incident upon the line scan camera from one light source is reflected and light incident upon the line scan camera due to illumination by the second light source comprises light fluoresced by a product component illuminated by light from the second light source.

22. The machine vision system of Claim 13 wherein dimensional relationships between product components on the separate de-interlaced images remain identical because the interlaced image, from which the two de-interlaced images are formed, is generated by the same camera.