阅读说明：本技术 兼具aoi及ai的瑕疵检测方法及其装置 (Flaw detection method and device with AOI and AI functions ) 是由 魏源钟 许智钦 于 2019-10-17 设计创作，主要内容包括：一种兼具AOI及AI的瑕疵检测方法及其装置,其主要是先对待测物加以取像后,再借由自动化光学检测(Automated Optical Inspection,简称AOI)与人工智能(Artificial Intelligence,缩写为AI)检测器两者个别的优势与弱点,透过其前后程序组合建立方法,来完成互补其缺点,使漏检率仍然可以维持最低的前提下,同时具有误检率也维持最低的优势。(A flaw detection method and device with AOI and AI functions includes picking up image of object to be detected, utilizing respective advantages and weaknesses of automatic Optical detection (AOI) and Artificial Intelligence (AI) detector, building up method by combining former and latter programs to complement said disadvantages, making detection leakage rate be kept at minimum and having advantage of keeping error detection rate at minimum.)

1. A defect detection method with AOI and AI functions is characterized by comprising the following steps:

firstly, taking images, namely taking images of an object to be measured by a camera unit, applying the camera unit or a movable mechanism such as a mechanical arm to combine a light source device and a light source photographing control module, applying a plurality of lighting modes for a plurality of times to switch on and off each group of light sources by different light source control methods, editing the intensity and the angle of the light sources and combining the repeated photographing function of a camera to achieve the function of quickly taking image data;

adjusting the detection sequence, and selecting a repeatable combination establishment method of the procedures before and after detection of the automatic optical detection AOI module and the artificial intelligent AI detector for detection according to the object to be detected and the individual advantages and weaknesses of the automatic optical detection AOI module and the artificial intelligent AI detector;

carrying out first detection according to the sequence of adjusting the detection sequence;

inputting the first detection data into a database, namely outputting the first detection data of the automatic optical detection AOI module or the artificial intelligent AI detector and storing the first detection data into the database for subsequent detection or storing;

extracting data from a database for continuous detection, namely extracting the data which needs to be detected from the database by the automatic optical detection AOI module or the artificial intelligence AI detector for continuous data detection;

and inputting the data after continuous detection into a database, namely outputting the data continuously detected by the automatic optical detection AOI module or the artificial intelligent AI detector and storing the data into the database for subsequent detection or storing.

2. The method of claim 1, wherein when the inspection object is mainly used for inspecting a relatively simple image background, such as a frame defect of a smart phone, the method comprises:

firstly, imaging an object to be detected;

then, an automatic optical inspection AOI module is used for inspecting the information of the object after image capture and carrying out an automatic optical inspection AOI algorithm so as to set and find out a circled inspecting range and output the range to a database;

the circled detection range is read from the database and input into an artificial intelligence AI detector for detection, and the artificial intelligence AI detector can screen various types which are not classified into defects, such as dust and the like, to obtain a classification result, and the classification result is output and stored into the database.

3. The method of claim 1, wherein the inspection object is mainly used for inspection of a relatively complex image background but requiring precise definition of a defect area, such as a wood grain surface, and comprises:

firstly, imaging an object to be detected;

then, an artificial intelligent AI detector is used for imaging the object and then an object detection algorithm is applied to find out a boundary frame of the defect, and the boundary frame is output and stored in a database after the defect is classified;

and reading the image in the boundary frame from the database to perform object segmentation by an automatic optical inspection AOI module, and outputting the segmented result to the database.

4. A flaw detection device having both AOI and AI functions, comprising:

loading a collection platform for placing an object to be tested;

the light source device is arranged on the periphery above the loading and collecting platform, is provided with a lamp source fixture with a coaxial light source, a linear light source, a backlight source, an annular light source, a spherical light source and the like, and is combined with a movable mechanism such as a mechanical arm and a light source photographing control module to apply a plurality of times of lighting modes so as to provide different light source control methods for switching on and off each group of lamp sources and editing the intensity and the angle of the lamp sources;

the camera unit and the light source device are arranged on the periphery above the loading and collecting platform in a staggered mode, and the camera unit comprises camera tools, line camera tools and the like, and is combined with a camera to achieve the function of rapidly collecting data by repeatedly shooting;

the movable mechanism can be a lifting mechanism conveying mechanism or a mechanical arm and the like;

the automatic optical detection AOI module is communicated with the camera unit and the database to receive the information of the object to be detected after image capture or after image connection and perform automatic optical detection AOI algorithm and the like;

an artificial intelligence AI detector which is communicated with the camera unit and the database to receive the information of the object to be detected after image capture or connection and applies an object detection algorithm and the like;

and the database is respectively communicated with the automatic optical inspection AOI module and the artificial intelligence AI detector so as to store the result of the image acquisition and detection of the object to be detected in the database unit and provide information for continuous inspection or other application terminals.

Technical Field

The invention relates to a flaw detection method and a device thereof with AOI and AI; the method and the device are applied to Optical detection, in particular to a method and a device for detecting through a repeatable combination establishment method of a front program and a back program of an Automatic Optical Inspection (AOI) detector and an Artificial Intelligence (AI) detector.

Background

At present, general flaw detection methods are of two types, one type is Automatic Optical Inspection (AOI), namely a high-speed high-precision Optical image detection system, and the defects that machine vision is matched with geometric/gray-scale value conditions set by engineers to serve as a detection standard technology and an Optical instrument is used manually for detection are improved; the other is detection using Artificial Intelligence (AI), which means that a machine performs deep learning to present image interpretation similar to human Intelligence to achieve the purpose of detection.

The calculation method of the former automatic Optical Inspection (AOI for short) adopts the following steps: binarization (binaryzation) is the simplest method for image segmentation, wherein the Binarization can convert a gray image into a binary image, the gray level of a pixel which is greater than a certain critical gray level value is set as a gray maximum value, and the gray level of a pixel which is less than the critical gray level value is set as a gray minimum value; secondly, Histogram (Histogram) statistics is a graphic representation of data distribution, and is a two-dimensional statistical chart, and two coordinates of the two-dimensional statistical chart are respectively a statistical sample and a certain attribute measurement corresponding to the sample; edge detection (Edge detection), which uses image processing and computer vision capabilities to identify points in the digital image where changes in brightness are significant, typically where significant changes in image attributes reflect attributes including (i) discontinuities in depth, (ii) discontinuities in surface orientation, (iii) changes in material attributes, and (iv) changes in scene illumination; and fourthly, dividing the object, namely dividing the object through automatic analysis of the content of the video. For example, the image with the mean value removed can retain the texture information of the image and reduce the influence of the light source change and the shadow on the background, or the object can be segmented by setting a gray level threshold or a differential threshold of the gray level, so as to achieve the purpose of rapidly and effectively segmenting the object.

Because the required tightness can be set by the automatic Optical Inspection (AOI for short), the method has the advantage of low omission factor, but at the same time, the over kill rate (or the defect of too high error rate) is also caused, so that the basic disadvantage that the detection in the automatic Optical Inspection (AOI for short) mode cannot be compatible exists in the prior art.

In addition, the detection method of Artificial Intelligence (AI) mainly uses deep learning (deep learning), and its calculation method adopts: firstly, Object Detection (Object Detection), namely classifying (Image Classification) pictures by using a Convolutional Neural Network (CNN), solving another problem by using a Neural Network (NN), classifying and positioning (Classification with localization), namely putting a target Object into a bounding box (bounding box), detecting (Detection) all objects and positioning (localization) the objects; and secondly, a classifier (classifier) for making a classification decision by clustering objects with similar features through linear combination of the features. The features of an object are usually described as feature values and in vectors as feature vectors. There are generative models, which use the principle of modeling conditional probability, and discriminant models (discriminant models), which attempt to maximize the output of a training set (training set).

Because of the deep learning and the imitation of human Intelligence, the above method has the advantages of better environmental adaptation and higher overall accuracy, but has the disadvantages that if the object detection method in the deep learning is used alone, the missing detection rate cannot be as low as AOI, which may cause the risk of defective products flowing into the market, and if the classifier in the deep learning is used alone, the position of the defect cannot be located, and the accuracy is not good due to the small proportion of the defect part in the whole image area, which is a difficult problem to overcome.

Therefore, in view of the deficiencies of the prior art, the inventors of the present invention have developed a method and an apparatus for detecting defects with AOI and AI over many years.

Disclosure of Invention

The main technical problem to be solved by the present invention is to overcome the above-mentioned defects in the prior art, and provide a defect detection method and device with AOI and AI, which apply the individual advantages and weaknesses of the traditional automatic Optical Inspection (AOI for short) and Artificial Intelligence (AI) detectors, and complete the complementation of the disadvantages through the combined procedure of the two, so that the missing Inspection rate can be maintained at the lowest, and the false Inspection rate is the lowest complete effect requirement; the method has wider applicability through various detection methods established by repeatedly combining procedures before and after matching of an Automatic Optical Inspection (AOI) and an Artificial Intelligence (AI) detector.

The technical scheme adopted by the invention for solving the technical problems is as follows:

a flaw detection method and device with both AOI and AI are suitable for Optical detection, in particular to a method and device for detecting through a repeatable combination establishment method of a front program and a back program of an Automatic Optical Inspection (AOI) detector and an Artificial Intelligence (AI) detector.

A method for detecting defects with both AOI and AI includes such steps as using camera fixture and movable mechanism to combine with light source to apply multiple lighting modes for different light source control methods, switching on or off the light sources, editing the intensity and angle of light sources, combining with camera to repeat shooting, taking image, and using the Automatic Optical Inspection (AOI) and Artificial Intelligence (AI) detectors to detect their advantages and weaknesses by repeating their combination.

When the detection object is mainly used for detecting defects of a relatively simple image background, such as a frame of a smart phone, the detection object firstly captures images of an object to be detected, then an automatic optical detection AOI module detects information of the object after image capture and carries out an automatic optical detection AOI algorithm to set and find out a circled detection Range (ROI) and output the ROI to be stored in a database, then the circled detection range is read from the database and input into an artificial intelligence detection AI classifier for detection, and the artificial intelligence AI detector can screen various types which are not classified into the defects, such as dust and the like, to obtain classification results and output the classification results to the database.

When the detection object is mainly used for detecting that the image background is relatively complex, but the defect area needs to be accurately defined, such as the wood grain surface, the object to be detected is firstly imaged, then an object detection (object detection) algorithm is applied to the object after the image of the object is imaged by using an artificial intelligent AI detector, a bounding box (bounding box) of the defect is found out, the defect is classified and then output and stored in a database, then the image in the bounding box is read from the database to automatically perform object segmentation by an optical inspection AOI module, and then the segmented result is output and stored in the database.

A flaw detection method and device with AOI and AI functions, the device mainly includes a loading and collecting platform for placing the object to be detected; the light source device is arranged on the periphery above the loading and collecting platform, is provided with a lamp source fixture with a coaxial light source, a linear light source, a backlight source, an annular light source, a spherical light source and the like, and is combined with a movable mechanism such as a mechanical arm and a light source photographing control module to apply a plurality of times of lighting modes so as to provide different light source control methods for switching on and off each group of lamp sources and editing the intensity and the angle of the lamp sources; the camera unit and the light source device are arranged on the periphery above the loading and collecting platform in a staggered mode, and the camera unit comprises a surface camera jig, a line camera jig and other applicable camera jigs and combines a camera to repeatedly shoot; the movable mechanism can be a lifting mechanism conveying mechanism or a mechanical arm and the like; the automatic optical detection AOI module is communicated with the camera unit and the database to receive the information of the object to be detected after image capture or after image connection and perform automatic optical detection AOI algorithm and the like; an artificial intelligence AI detector, which is communicated with the camera unit and the database to receive the information of the object to be detected after image capture or after image connection and apply object detection algorithm, etc.; and the database is respectively communicated with the automatic optical inspection AOI module and the artificial intelligence AI detector so as to store the result of the image acquisition and detection of the object to be detected into the database and provide information for continuous inspection or other application terminals.

Therefore, the invention provides a flaw detection method with AOI and AI and a device thereof, which mainly comprises the steps of firstly providing a plurality of lighting modes by combining a camera tool and a movable mechanism with a light source device to switch on and off various groups of light sources by different light source control methods, editing the intensity and the angle of the light sources, combining the repeated shooting function of a camera to achieve the function of quickly collecting data, then detecting by using a combined establishment method of Automatic Optical Inspection (AOI) and Artificial Intelligence (AI) detectors to complement the defects by using the respective advantages and weaknesses of the AOI and the AI detectors through a combined establishment method of front and back programs, so that the automatic Optical detection I algorithm performed by an automatic Optical detection I module is firstly performed when the detected object is mainly used for detecting the image background relative to the simple AOI such as the flaw of a smart phone frame, the detection flow of the whole product has the functions of low missing detection rate and low false detection rate, when the detection object is mainly used for detecting that the image background is relatively complex but the flaw area needs to be accurately defined, such as the wood grain surface, the detection flow of the artificial intelligent AI detector has higher background adaptability, the flaw boundary frame (bounding box) and the flaw are classified and output, which cannot be directly realized by an automatic optical detection AOI module algorithm, the artificial intelligent AI detector outputs a database after finding out the flaw boundary frame (bounding box) and then reads data by the automatic optical detection I module and then divides the object, because the automatic optical detection AOI module can accurately segment the range of the flaw by using the AOI algorithm and calculate the flaw area, the defect that the AI algorithm used by the artificial intelligent AI detector cannot accurately calculate the area is complemented, and the automatic optical detection AOI module becomes an effective creative idea of the invention.

The invention has the advantages that the individual advantages and weaknesses of the traditional Automatic Optical Inspection (AOI) and Artificial Intelligence (AI) detectors are applied, and the combined program of the AOI and AI detectors is used to complement the defects, so that the undetected rate can be kept to be the lowest, and meanwhile, the complete effect requirement that the false detection rate is the lowest is met; the method has wider applicability through various detection methods established by repeatedly combining procedures before and after matching of an Automatic Optical Inspection (AOI) and an Artificial Intelligence (AI) detector.

Drawings

The invention is further illustrated with reference to the following figures and examples.

FIG. 1 is a flow chart of a method of practicing the present invention.

Fig. 2 is a flow chart of the present invention when the detection object is mainly used for relatively simple detection of the image background.

Fig. 3 is a flow chart of the present invention for detecting objects, which is mainly used for detecting relatively complicated image backgrounds.

FIG. 4 is a side view of the apparatus of the present invention.

Fig. 5 is a block diagram of the present invention.

The reference numbers in the figures illustrate:

1 Loading and collecting platform

2 light source device

20 coaxial light source

21 line light source

3 Camera Unit

4 automatic optical inspection AOI module

5 Movable mechanism

6 artificial intelligence AI detector

7 database

Detailed Description

The following embodiments are provided to illustrate and describe the embodiments of the present invention, and those skilled in the art can easily understand other advantages and effects of the present invention from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention.

Referring to fig. 1, fig. 2, and fig. 3, with the remaining drawings showing, the present invention provides a defect detecting method and apparatus having both AOI and AI, the method mainly includes:

firstly, taking images, namely taking images of an object to be detected by using a camera unit 3, applying the camera unit 3 or a movable mechanism 5 such as a mechanical arm to combine a light source device 2 and a light source photographing control module 4, applying a plurality of lighting modes for a plurality of times to provide different light source control methods for switching on and off each group of light sources, editing the intensity and the angle of the light sources and combining a repeated shooting function of a camera to achieve a function of rapidly taking image data;

adjusting the detection sequence, and selecting a repeatable combination establishment method of the procedures before and after detection by the automatic optical detection AOI module 4 and the artificial intelligent AI detector 6 for detection according to the object to be detected and the respective advantages and weaknesses of the automatic optical detection AOI module 4 and the artificial intelligent AI detector 6;

and carrying out first detection according to the sequence of the adjustment detection sequence.

The first detection data is input into the database 7, which means that the data detected for the first time by the automatic optical detection AOI module 4 or the artificial intelligent AI detector 6 is output and stored in the database 7 for subsequent detection or storage;

the data is stored in the database 7 for continuous detection, which means that the automatic optical detection AOI module 4 or the artificial intelligence AI detector 6 extracts the data to be detected from the database 7 for continuous data detection.

The data after continuous detection is input into the database 7, which means that the data continuously detected by the automatic optical detection AOI module 4 or the artificial intelligence AI detector 6 is output and stored in the database 7 for subsequent detection or storage.

When the detection object is mainly used for detecting image backgrounds such as a frame defect of a smart phone, the detection object first captures an image of an object to be detected; then, the automatic optical inspection AOI module 4 is used for inspecting the information of the object after image capture and performing automatic optical inspection AOI algorithm to set and find out the detection Range (ROI) of the circle selection and output the detection range and store the detection range in the database 7; the circled detection range is read from the database 7 and input into the artificial intelligence AI detector 6 for detection, and the artificial intelligence AI detector 6 can screen various types which should not be classified into defects, such as dust, to obtain a classification result, and then the classification result is output and stored in the database 7.

When the inspection object is mainly used for inspection of a relatively complex image background but requires accurate definition of a flaw area, such as a wood grain surface (as shown in fig. 3), the object to be inspected is firstly imaged; then, an artificial intelligent AI detector 6 is used for imaging the object and then an object detection algorithm is applied to find out a bounding box (bounding box) of the flaw, and the flaw is classified and then output and stored in a database 7; the image in the bounding box is read from the database 7 to be segmented by the automatic optical inspection AOI module 4, and the segmented result is output to the database 7.

Please refer to fig. 4 and 5 with the remaining figures; the invention relates to a defect detection method with AOI and AI and a device thereof, which provides the following steps:

loading the collection platform 1 for placing the object to be tested;

a light source device 2, which is arranged on the upper periphery of the loading and collecting platform 1, the light source device 2 is provided with a coaxial light source 20, a linear light source 21, a backlight source, an annular light source, a spherical light source and other groups of light source fixtures, and is combined with a movable mechanism 5 such as a mechanical arm and a light source photographing control module 4 to apply a plurality of times of lighting modes so as to provide different light source control methods for switching on and off each group of light sources and editing the intensity and the angle of the light sources;

the camera unit 3, the camera unit 3 and the light source device 2 are arranged on the periphery of the upper part of the loading and collecting platform 1 in a staggered way, and comprise camera tools such as camera tools, line camera tools and the like, and the camera tools are used and are combined with the repeated shooting function of a camera to achieve the function of rapidly collecting data;

a movable mechanism 5, wherein the movable mechanism 5 can be a lifting mechanism conveying mechanism or a mechanical arm and the like;

an automatic optical inspection AOI module 4 communicating with the camera unit 3 and the database 7 to receive the information of the object to be inspected after image capture or continuous information and performing automatic optical inspection AOI algorithm and the like;

an artificial intelligence AI detector 6 communicating with the camera unit 3 and the database 7 to receive information of the object to be detected after image capture or after image capture and applying a detection (object detection) algorithm;

and the database 7 is respectively communicated with the automatic optical inspection AOI module 4 and the artificial intelligence AI detector 6 so as to store the result of the image acquisition and detection of the object to be detected in the database 7 and provide information for continuous inspection or other application terminals.

Therefore, the invention provides a flaw detection method with AOI and AI and a device thereof, which mainly comprises the steps of firstly using a camera tool 3 and a movable mechanism 5 to combine with a light source device 2 to apply a plurality of lighting modes for a plurality of times to provide different light source control methods to switch on and off each group of light sources, editing the intensity and the angle of the light sources and combining with the repeated shooting function of a camera to achieve the function of quickly collecting data, then using an automatic optical detection AOI module 4 and an artificial intelligent AI detector 6 to carry out detection through a front and back program combination establishment method to complement the defects, so that the missed detection rate can be kept to the lowest on the premise, and simultaneously has the advantage that the false detection rate is kept to the lowest, when the detection object is mainly used for detecting the image background which is relatively simple, such as the flaw of a frame of an intelligent mobile phone, the automatic optical detection AOI algorithm which is firstly carried out by the automatic optical detection AOI module 4, the false detection rate can be ensured to be low enough, the defective products are prevented from flowing into the market, the artificial intelligent AI detector 6 in the next step of the detection process can effectively and greatly reduce the false detection rate and avoid unnecessary loss, the detection process has the advantages of both, and the detection process of the whole product has the functions of low false detection rate and low false detection rate; when the inspection object is mainly used for inspection of a relatively complex image background but requiring accurate definition of a defect area, such as a wood grain surface, the inspection flow of the artificial intelligent AI detector 6 has high background adaptability, and a defective boundary box (bounding box) and a defect are classified and output, which cannot be achieved by the automatic optical inspection AOI module 4 directly using an automatic optical inspection AOI module algorithm, so that the artificial intelligent AI detector 6 finds out the defective boundary box (bounding box) and then outputs the database 7, and the automatic optical inspection AOI module 4 reads data for object segmentation, because the automatic optical inspection AOI module 4 can accurately segment the defect range by using the AOI algorithm and calculate the defect area, the defect area loss advantage that the AI algorithm used by the artificial intelligent AI detector 6 cannot accurately calculate the area is complemented, thus becoming an effective idea of the present invention.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention in any way, and all simple modifications, equivalent variations and modifications made to the above embodiment according to the technical spirit of the present invention still fall within the scope of the technical solution of the present invention.