阅读说明：本技术 基于机器视觉的智能手机玻璃盖板丝印区缺陷检测方法 (Method for detecting defects of silk-screen area of glass cover plate of smart phone based on machine vision ) 是由 张宪民 欧阳健燊 李常胜 汤传刚 郝强 于 2019-09-14 设计创作，主要内容包括：本发明公开了一种基于机器视觉的智能手机玻璃盖板丝印区缺陷检测方法,包括如下步骤：采集手机屏图像；读取相关参数信息；视窗检测；对手机盖板丝印区轮廓进行大缺陷提取；分割检测区域为丝印区、孔与字符区、光带区与干扰区；获取各区域的缺陷；修正手机盖板的轮廓,获取崩边的缺陷信息；利用神经网络分类器,把缺陷进行点状、线状和面状缺陷的分类；根据缺陷定义的标准进行对缺陷的筛选；利用深度学习对线缺陷、IR孔缺陷和字符缺陷进行深层缺陷分类,线缺陷分类包括毛丝和划痕、IR孔缺陷等；统计各类缺陷的形貌信息。本发明能实现多种型号的通用性应用,针对不同的检测标准进行在线调整,能快速准确提取麻点、毛丝、划痕和脏污等缺陷。(The invention discloses a method for detecting defects of a silk-screen area of a glass cover plate of a smart phone based on machine vision, which comprises the following steps: acquiring a mobile phone screen image; reading related parameter information; detecting a window; extracting large defects of the outline of the silk-screen area of the mobile phone cover plate; dividing the detection area into a silk screen area, a hole and character area, an optical band area and an interference area; acquiring defects of each area; the method comprises the steps of correcting the outline of a mobile phone cover plate, and acquiring defect information of edge breakage; classifying the point-shaped, linear and surface-shaped defects by using a neural network classifier; screening the defects according to the standard defined by the defects; deep defect classification is carried out on line defects, IR hole defects and character defects by utilizing deep learning, wherein the line defect classification comprises broken filaments, scratches, IR hole defects and the like; and counting the shape information of various defects. The invention can realize the universal application of various types, carries out online adjustment aiming at different detection standards, and can quickly and accurately extract the defects of pockmarks, broken filaments, scratches, dirt and the like.)

1. A method for detecting defects of a silk-screen area of a glass cover plate of a smart phone based on machine vision is characterized by comprising the following steps:

collecting an image of a glass cover plate of the mobile phone;

reading related parameter information, including template information of an outer contour, upper and lower limits of a global threshold, size information of a mobile phone cover plate, interference area information, the type of the cover plate, average gray value of a silk-screen area, pixel equivalent, hole and character and other template information and defect screening standard information of the silk-screen area;

window detection, detecting whether the mobile phone cover plate exceeds a window, if so, continuing to perform subsequent operation, otherwise, ending the detection and outputting error report information;

extracting large defects of the outline of the silk-screen area of the mobile phone cover plate;

dividing a detection area, including a silk screen area, a hole and character area, an optical band area and an interference area;

acquiring defects of each detection area;

the method comprises the steps of correcting the outline of a mobile phone cover plate, and acquiring defect information of edge breakage;

classifying the point-shaped, linear and surface-shaped defects by using a neural network classifier;

screening the defects according to the standard defined by the defects;

deep defect classification is carried out on line defects, IR hole defects and character defects by utilizing deep learning, wherein the line defect classification comprises broken filaments and scratches, and the IR hole defects and the character defects comprise broken edges and internal defects;

and (4) counting the shape information of various defects, including the area, the position, the length, the width, the minimum circumscribed circle, the maximum inscribed circle and the external ellipse.

2. The machine vision-based defect detection method for the silk-screen area of the glass cover plate of the smart phone as claimed in claim 1, wherein the mobile phone glass cover plate image acquisition specifically adopts a 16K line camera to respectively acquire images of the mobile phone cover plate of the upper half and the mobile phone cover plate of the lower half, and the image size exceeds 1/2 of the whole mobile phone screen during each shooting.

3. The machine vision-based method for detecting defects of silk-screen areas of glass cover plates of smart phones according to claim 1, wherein the window detection step specifically comprises:

the method comprises the steps of manufacturing a detection frame which is 2-5 pixels smaller than a window in advance, then primarily screening out an area of a mobile phone cover plate by utilizing a global threshold, then judging whether the area exists or not, then judging whether the area and the area of the detection frame have an intersection or not, and if the intersection exists, indicating that the area exceeds the window and needs to be placed again.

4. The method for detecting defects of silk-screen areas of a glass cover plate of a smart phone based on machine vision as claimed in claim 3, wherein the step of preliminarily screening out the areas of the cover plate of the smart phone by using the global threshold specifically sets a numerical value within a range of 0-255 by using the characteristic that the gray value of a single-channel image is between 0-255, and divides the image into two parts, namely, the unnecessary areas in the image can be shielded.

5. The machine vision-based defect detection method for the silk-screen area of the glass cover plate of the smart phone according to claim 1, wherein the step of extracting the large defects of the outline of the silk-screen area of the mobile phone cover plate comprises the following steps:

extracting a contour region of the mobile phone cover plate by global threshold segmentation by using the acquired standard mobile phone cover plate image, and manufacturing a standard template of the mobile phone cover plate; selecting a target area by manual framing, respectively segmenting a hole area and a character area by utilizing a local threshold value, manufacturing a standard template of holes and a standard template of characters, and storing all the standard templates as template files;

during online detection, a template matching method is utilized to match the contour extracted by the same mode from the detected image of the mobile phone cover plate with a standard template of the mobile phone cover plate, a matching score threshold is set, if the matching score reaches the threshold, corresponding position information and rotation information are obtained, otherwise, detection is finished and error reporting information is output;

and performing affine transformation on the standard template of the mobile phone cover plate to corresponding coordinates and rotation angles by using the acquired information, comparing the coordinates and rotation angles with the outline of the picture of the mobile phone cover plate in detection, obtaining the difference between the coordinates and the rotation angles by using Boolean operation, and outputting large defect information when the minimum circumscribed circle of the difference region reaches the minimum standard of the large defect to finish the detection.

6. The method for extracting the large defects of the outline of the silk-screen area of the mobile phone cover plate as claimed in claim 5, wherein the template matching method is to use the input template to translate pixel by pixel in the test image and set a certain search angle range to find the position and the deflection angle with the highest quality coefficient of the overlap with the test image, and record the information.

7. The machine vision-based defect detection method for the silk-screen area of the glass cover plate of the smart phone according to claim 6, wherein the step of dividing the detection area, including the silk-screen area, the holes and the character area, and the optical band area and the interference area, specifically comprises the following steps:

acquiring holes and character areas in a template matching mode, finding the holes and the character areas one by one, respectively moving the standard templates of the holes and the standard templates of the characters to corresponding positions in an affine transformation mode, acquiring images of the areas on an original image, and extracting corresponding real areas on the original image by using corresponding gray value thresholds, so that the areas of the holes and the characters are acquired more accurately;

acquiring an optical band region, acquiring an angle deviated from the horizontal direction by using the minimum external moment of the outline of the mobile phone cover plate, correcting the outline by using affine transformation, and creating a rectangle which takes the numerical value of the height as the length and the width of the outline as the width by taking the top edge of the outline as the starting point according to the height and the thickness of the optical band region of the interference region information in the read information; intersecting the rectangle with the outline to obtain the outline of the top, obtaining the optical band area by using the area expansion operation according to the obtained thickness information of the optical band area, and finally converting the inverse affine transformation back to the original position;

removing the overlapping area, manufacturing a rectangle with the length being half of the length of the mobile phone cover plate as the length and the width being the width of the mobile phone cover plate as the starting point according to the corrected outline, intersecting the outline to obtain an outline area of the upper half, and finally carrying out inverse affine transformation to the original position;

obtaining an interference area, and performing area expansion on the outline of the inner frame and the outline of the outer frame according to the thickness of the inner frame and the thickness of the outer frame of the interference area information in the read information to obtain the interference area of the inner frame and the interference area of the outer frame;

and obtaining a silk-screen area, and removing the light band area, the interference area, the hole and the character area in the outline area of the original position to obtain the silk-screen area.

8. The machine vision-based method for detecting defects of silk-screen areas of glass cover plates of smart phones according to claim 1, wherein the step of acquiring the defects of each detection area specifically comprises the following steps:

extracting the defects of the silk-screen area, and adding 20-40 gray values on the basis of the average gray value in the read information to extract the defects of the silk-screen area, wherein the gray value can be set by a user, and the default value is 30;

the defects of the light band region are extracted, the light reflection band region of the light band region can be extracted by adding 40-60 gray values on the basis of the average gray value in the read information, and in order to completely extract the light band, the intermittent region needs to be subjected to closed operation, the adjacent regions are connected, and false detection is avoided; screening out the largest actual area, namely the light band, according to the length of the minimum circumscribed rectangle of each area, subtracting the area from the light band area to obtain a non-interference area, and adding a gray value of 20-40 on the basis of the average gray value to extract the defects in the light band area;

all extracted defects are merged.

9. The method for detecting the defects of the silk-screen area of the smart phone glass cover plate based on the machine vision as claimed in claim 1, wherein the step of correcting the outline of the smart phone cover plate and acquiring the defect information of edge breakage specifically comprises the steps of:

on the basis of the corrected profile degree, the profile is divided into the profiles of an outer frame and an inner frame, firstly, a long-strip rectangle with the length of 30-100 pixels and the width of 1-5 pixels is used for opening operation, and edge bulges in the vertical direction of less than 0-100 pixels are removed; performing opening operation by using a long rectangular with the length of 1-5 pixels and the width of 30-100 pixels, and removing edge bulges in the horizontal direction of less than 0-100 pixels; respectively performing closing operation by using the rectangles with the two shapes, and respectively filling edge pits in the vertical direction and edge pits in the horizontal direction which are smaller than 0-100 pixels;

after processing vertical and horizontal edges, processing a circular arc part, creating a minimum positive external moment of the outline, performing difference operation on the minimum positive external moment and the original outline, performing opening operation on a circle with the radius of 20-30 pixels to obtain a circular arc area, creating the minimum positive external moment on the basis of the area, taking 0.2 time of the length of the rectangle as the radius of the circular opening operation, and removing a convex part of the circular arc part; closing operation is carried out in the same way, and the pit part of the circular arc is filled;

combining the corrected outlines of the vertical, horizontal and circular arc parts to form a corrected outline, performing the same treatment on the internal and external outlines to obtain a complete outline, and finally performing inverse affine transformation to the original position;

and performing Boolean operation on the obtained corrected outline and the original detection outline to obtain the defect information of edge breakage of the edge.

10. The method for detecting the defects of the silk-screen area of the glass cover plate of the smart phone based on the machine vision as claimed in claim 1, wherein the step of classifying the defects in a point shape, a linear shape and a planar shape by using a neural network classifier specifically comprises the following steps:

obtaining feature vectors comprising equivalent ellipse major axis radius length, equivalent ellipse minor axis radius length, roundness, maximum diameter, maximum inscribed rectangle width, minimum circumscribed circle radius, maximum inscribed circle radius, tightness, contour line total length, convexity, rectangularity, filling power and ratio features of major axis and minor axis of the defects by using the extracted sample defects, and labeling the feature vector group of each sample defect according to the type of a point line surface shape to form a sample library;

inputting the sample library into a neural network model, and training a point-line planar neural network classifier capable of identifying the defects;

during online detection, the extracted defects are directly input into the neural network classifier one by one, and the type of the defects belonging to the point line surface is identified.

11. The method for detecting defects in silk-screen area of glass cover plate of smart phone based on machine vision as claimed in claim 10, wherein when the defects are extracted from the silk-screen area, before classifying the defects by neural network classifier into point, linear and planar defects, further comprising the steps of:

and realizing clustering processing, and connecting all the areas in the silk-screen area to obtain connected domain characteristics, namely clustering and connecting all the areas with the clustering radius of R pixels, the area of more than S pixels and the number of small defect areas of more than N.

12. The method for detecting the defects of the silk-screen area of the glass cover plate of the smart phone based on the machine vision as claimed in claim 1, wherein the step of screening the defects according to the criteria defined by the defects specifically comprises the steps of:

aiming at point-like defects:

if D is less than 0.1mm, neglecting;

if D is more than or equal to 0.1mm and less than or equal to 0.2mm and D is more than or equal to 10mm, the mobile phone screen is good within 2, and the mobile phone screen is defective within 3;

if D is larger than 0.2mm, the mobile phone screen is a defective product;

wherein D is the minimum circumscribed circle diameter of the point defect, and D is the distance between a point and the point;

for linear defects:

if W is less than 0.02mm, the product is treated as a good product;

if W is more than or equal to 0.02mm and less than or equal to 0.05mm, d is more than or equal to 5mm, L is less than or equal to 3, the mobile phone screen is good when the number is less than or equal to 2, and the mobile phone screen is defective when the number is less than or equal to 3;

if W is larger than 0.05mm or L is larger than 3mm, the mobile phone screen is a defective product;

wherein W is the linear defect width L and the linear defect length, d is the different linear defect interval;

aiming at the surface defect:

if D > 0.5mm or S > 0.2mm²Indicating that the mobile phone screen is a defective product;

wherein D is the maximum inscribed circle diameter of the planar defect, and S is the planar defect area.

13. The machine vision-based method for detecting defects of silk-screen areas of glass cover plates of smart phones according to claim 1, wherein deep defect classification of line defects, IR hole defects and character defects by means of deep learning specifically comprises the steps of:

selecting a pretreatment model, wherein the pretreatment model is a predicted _ dl _ classifier _ enhanced model provided by HALCON software;

if the width or the length of the minimum positive external moment of the defect is smaller than the standard material size, filling the part smaller than the minimum positive external moment of the defect with the gray value average value of a frame which is 2-20 pixels larger than the minimum positive external moment of the defect, mainly extracting the average gray value near the defect as a filling value, and keeping the average gray value unchanged if the average gray value is larger than the minimum positive external moment of the defect; expanding the sample size of the defective material by mirroring, rotating and changing the brightness of the picture; then, image preprocessing is carried out, wherein the image preprocessing mainly comprises the steps of standardization of an image channel, normalization of image size and labeling of samples according to types;

adjusting deep learning parameters, inputting the prepared sample into a pre-training model, and starting off-line training;

after the trained deep learning classifier is obtained, the image and the character image of the defect or the whole IR hole are directly input into the corresponding classifier one by one, and the class of the defect or the existence of the defect in the IR hole and the character can be identified.

Technical Field

The invention relates to the field of visual detection, in particular to a method for visually detecting defects of a silk-screen area of a glass cover plate of a smart phone based on machine vision.

Background

The glass cover plate has the advantages of high hardness, high strength, scratch resistance, high transmittance, excellent impact resistance and the like, is widely applied to the fields of smart phones, tablet computers and the like, and has a very wide application background, however, defects can be generated on the glass cover plate in the production process or the transportation process, the quality of products is reduced, the defects comprise pittings, scratches, dirt, edge breakage, broken filaments, dust and the like, before the finished products are produced, the defects need to be detected and identified, and then the corresponding processes are adopted to repair or directly scrap according to the characteristics of different defects, so that the defective products are prevented from flowing into the market.

At present, the defect detection of the glass cover plate adopts manual detection and Automatic Optical Inspection (AOI), but the manual detection cannot realize accurate measurement, and the detection result can be more stable and reliable by using professional rapid detection equipment; the machine vision can not generate fatigue errors, and on the contrary, even if the products are completely the same, human eyes can have slight difference when detecting the products each time; the automatic detection equipment can undertake tasks of a plurality of people, and the detection efficiency is high. Although the advantages of the AOI device are great, there are still more difficulties to overcome, such as setting of detection standards, reliability of the device is very dependent on programming of programs, and problems of universality of products need to be solved.

Disclosure of Invention

Aiming at the technical problems, the invention aims to provide a smart phone glass cover plate silk-screen area defect detection method based on machine vision, which is used for detecting defects of a mobile phone cover plate silk-screen area. The method is mainly characterized in that the method can realize the universal application of various types, can effectively shield interference areas, accurately extract defects according to different detection standards, effectively identify the types of the defects by adopting a method of combining a classifier and deep learning, and acquire the morphology information of the defects.

The purpose of the invention is realized by at least one of the following technical solutions.

A method for detecting defects of a silk-screen area of a glass cover plate of a smart phone based on machine vision comprises the following steps:

collecting an image of a glass cover plate of the mobile phone;

reading related parameter information, including template information of an outer contour, upper and lower limits of a global threshold, size information of a mobile phone cover plate, interference area information, the type of the cover plate, average gray value of a silk-screen area, pixel equivalent, hole and character and other template information and defect screening standard information of the silk-screen area;

window detection, detecting whether the mobile phone cover plate exceeds a window, if so, continuing to perform subsequent operation, otherwise, ending the detection and outputting error report information;

extracting large defects of the outline of the silk-screen area of the mobile phone cover plate;

dividing a detection area, including a silk screen area, a hole and character area, an optical band area and an interference area;

acquiring defects of each detection area;

the method comprises the steps of correcting the outline of a mobile phone cover plate, and acquiring defect information of edge breakage;

classifying the point-shaped, linear and surface-shaped defects by using a neural network classifier;

screening the defects according to the standard defined by the defects;

deep defect classification is carried out on line defects, IR hole defects and character defects by utilizing deep learning, wherein the line defect classification comprises broken filaments and scratches, and the IR hole defects and the character defects comprise broken edges and internal defects;

the shape information of various defects, including the area, the position, the length, the width, the minimum circumscribed circle, the maximum inscribed circle and the external ellipse, is counted, so that the characteristics of the defects are more digital and visualized, and a certain basis is provided for defect screening.

Further, the mobile phone glass cover plate image acquisition specifically adopts a 16K linear array camera to respectively acquire images of the upper half mobile phone cover plate and the lower half mobile phone cover plate, the image size exceeds 1/2 of the whole mobile phone screen during each shooting, the image resolution of the 16K linear array camera is 40000 multiplied by 16384, and the detection precision reaches 0.005 mm.

Further, the window detection step specifically includes:

the method comprises the steps of manufacturing a detection frame which is 1-10 pixels smaller than a window in advance, then primarily screening out an area of a mobile phone cover plate by utilizing a global threshold, then judging whether the area exists or not, then judging whether the area and the area of the detection frame have intersection or not, and if the intersection exists, indicating that the area exceeds the window and needs to be placed again.

Further, the step of preliminarily screening out the area of the mobile phone cover plate by using the global threshold specifically is to set a numerical value within a range of 0-255 by using the characteristic that the gray value of a single-channel image is between 0-255, and divide the image into two parts, namely, the unnecessary area in the image can be shielded.

Further, the method is characterized in that the step of extracting the large defects of the outline of the silk-screen area of the mobile phone cover plate comprises the following steps:

extracting a contour region of the mobile phone cover plate by global threshold segmentation by using the acquired standard mobile phone cover plate image, and manufacturing a standard template of the mobile phone cover plate; selecting a target area by manual framing, respectively segmenting a hole area and a character area by utilizing a local threshold value, manufacturing a standard template of holes and a standard template of characters, and storing all the standard templates as template files;

during online detection, a template matching method is utilized to match the contour extracted by the same mode from the detected image of the mobile phone cover plate with the standard template, a matching score threshold is set, if the matching score reaches the threshold, corresponding position information and rotation information are obtained, otherwise, detection is finished and error reporting information is output;

and performing affine transformation on the standard template to corresponding coordinates and rotation angles by using the acquired information, comparing the coordinates and rotation angles with the outline of the picture of the mobile phone cover plate in detection, acquiring the difference between the coordinates and the outline by using Boolean operation, and outputting large defect information when the minimum circumscribed circle of the difference region reaches the minimum standard of the large defect to finish the detection.

Further, the template matching method specifically comprises the steps of translating the input template pixel by pixel in a test image, setting a certain search angle range, searching a position and a deflection angle with the highest quality coefficient of the overlap in the test image, and recording the information.

Further, the segmentation detection area comprises a silk screen area, a hole and a character area, and the optical band area and the interference area specifically comprise the following steps:

acquiring holes and character areas in a template matching mode, finding the holes and the character areas one by one, then respectively moving the hole standard template and the standard template of the characters to corresponding positions in an affine transformation mode, acquiring an image of the area on an original image, and extracting a corresponding real area on the original image by using a corresponding gray value threshold value, so that the areas of the holes and the characters are more accurately acquired;

acquiring an optical band region, acquiring an angle deviated from the horizontal direction by using the minimum external moment of the outline of the mobile phone cover plate, correcting the outline by using affine transformation, and creating a rectangle which takes the numerical value of the height as the length and the width of the outline as the width by taking the top edge of the outline as the starting point according to the height and the thickness of the optical band region of the interference region information in the read information; intersecting the rectangle with the outline to obtain the outline of the top, obtaining the optical band area by using the area expansion operation according to the obtained thickness information of the optical band area, and finally converting the inverse affine transformation back to the original position;

removing the overlapping area, manufacturing a rectangle with the length being half of the length of the mobile phone cover plate as the length and the width being the width of the mobile phone cover plate as the starting point according to the corrected outline, intersecting the outline to obtain an outline area of the upper half, and finally carrying out inverse affine transformation to the original position;

obtaining an interference area, and performing area expansion on the outline of the inner frame and the outline of the outer frame according to the thickness of the inner frame and the thickness of the outer frame of the interference area information in the read information to obtain the interference area of the inner frame and the interference area of the outer frame;

and obtaining a silk-screen area, and removing the light band area, the interference area, the hole and the character area in the outline area of the original position to obtain the silk-screen area for better extracting the defects of the silk-screen area.

Further, the acquiring the defects of each detection area specifically includes the following steps:

extracting the defects of the silk-screen area, and adding 20-40 gray values on the basis of the average gray value in the read information to extract the defects of the silk-screen area, wherein the gray value can be set by a user, and the default value is 30; extracting the defects of the light band region, and adding 40-60 gray values on the basis of the average gray value in the read information to extract the light reflection band region of the light band region, wherein the gray value can be set by a user, and is preferably 50; in order to completely extract the light band, the intermittent areas need to be closed to connect the similar areas, so as to avoid false detection; screening out the largest actual area, namely the light band, according to the length of the minimum circumscribed rectangle of each area, subtracting the area from the light band area to obtain a non-interference area, and adding a gray value of 20-40 on the basis of an average gray value to extract the defects existing in the light band area, wherein the gray value can be set by a user, and is preferably 30;

all extracted defects are merged.

Further, the step of correcting the profile of the mobile phone cover plate and acquiring the defect information of edge breakage specifically includes the steps of:

on the basis of the corrected profile degree, the profile is divided into the profiles of an outer frame and an inner frame, firstly, a long-strip rectangle with the length of 30-100 pixels and the width of 1-5 pixels is used for opening operation, and edge bulges in the vertical direction of less than 0-100 pixels can be removed; then, performing opening operation by using a long rectangular with the length of 1-5 pixels and the width of 30-100 pixels, and removing edge bulges in the horizontal direction of less than 0-100 pixels; closing operation is respectively carried out by using the rectangles with the two shapes, and edge pits in the vertical direction and edge pits in the horizontal direction which are smaller than 0-100 pixels can be respectively filled;

after processing vertical and horizontal edges, processing a circular arc part, creating a minimum positive external moment of the outline, performing difference operation on the minimum positive external moment and the original outline, performing opening operation on a circle with the radius of 20-30 pixels to obtain a circular arc area, creating the minimum positive external moment on the basis of the area, taking 0.2 time of the length of the rectangle as the radius of the circular opening operation, and removing a convex part of the circular arc part; closing operation is carried out in the same way, and the pit part of the circular arc is filled;

combining the corrected outlines of the vertical, horizontal and circular arc parts to form a corrected outline, performing the same treatment on the internal and external outlines to obtain a complete outline, and finally performing inverse affine transformation to the original position;

and performing Boolean operation on the obtained corrected outline and the original detection outline to obtain the defect information of edge breakage of the edge.

Further, the classifying the point-like, linear and planar defects of the defect by using the neural network classifier specifically comprises the following steps:

obtaining feature vectors comprising equivalent ellipse major axis radius length, equivalent ellipse minor axis radius length, roundness, maximum diameter, maximum inscribed rectangle width, minimum circumscribed circle radius, maximum inscribed circle radius, tightness, contour line total length, convexity, rectangularity, filling power and ratio features of major axis and minor axis of the defects by using the extracted sample defects, and labeling the feature vector group of each sample defect according to the type of a point line surface shape to form a sample library;

inputting the sample library into a neural network model, and training a point-line planar neural network classifier capable of identifying the defects;

during online detection, the extracted defects are directly input into the neural network classifier one by one, and the type of the defects belonging to the point line surface is identified.

Further, when the defect is extracted from the silk-screen area, before the defect is classified into a point-like defect, a linear defect and a planar defect by using a neural network classifier, the method further comprises the following steps:

and realizing clustering processing, and connecting all the areas in the silk-screen area to obtain connected domain characteristics, namely clustering and connecting all the areas with the clustering radius of R pixels, the area of more than S pixels and the number of small defect areas of more than N.

Further, the screening of the defects according to the defect definition standard specifically includes the steps of:

aiming at point-like defects:

if D is less than 0.1mm, neglecting;

if D is more than or equal to 0.1mm and less than or equal to 0.2mm and D is more than or equal to 10mm, the mobile phone screen is good within 2, and the mobile phone screen is defective within 3;

if D is larger than 0.2mm, the mobile phone screen is a defective product;

wherein D is the minimum circumscribed circle diameter of the point defect, and D is the distance between a point and the point;

for linear defects:

if W is less than 0.02mm, the product is treated as a good product;

if W is more than or equal to 0.02mm and less than or equal to 0.05mm, d is more than or equal to 5mm, L is less than or equal to 3, the mobile phone screen is good when the number is less than or equal to 2, and the mobile phone screen is defective when the number is less than or equal to 3;

if W is larger than 0.05mm or L is larger than 3mm, the mobile phone screen is a defective product;

wherein W is the linear defect width L and the linear defect length, d is the different linear defect interval;

aiming at the surface defect:

if D > 0.5mm or S > 0.2mm²Indicating that the mobile phone screen is a defective product;

wherein D is the maximum inscribed circle diameter of the planar defect, and S is the planar defect area.

Further, the deep defect classification of line defects, IR hole defects and character defects by deep learning specifically includes the steps of:

selecting a pretreatment model, wherein the pretreatment model is a predicted _ dl _ classifier _ enhanced model provided by HALCON software;

if the width or the length of the minimum positive external moment of the defect is smaller than the standard material size, filling the part smaller than the minimum positive external moment of the defect with the gray value average value of a frame which is 2-20 pixels larger than the minimum positive external moment of the defect, and mainly extracting the average gray value near the defect as a filling value; if the value is larger than the preset value, the value is kept unchanged; expanding the sample size of the defective material by mirroring, rotating and changing the brightness of the picture; then, image preprocessing is carried out, wherein the image preprocessing mainly comprises the steps of standardization of an image channel, normalization of image size and labeling of samples according to types;

adjusting deep learning parameters, inputting the prepared sample into a pre-training model, and starting off-line training;

after the trained deep learning classifier is obtained, the image and the character image of the defect or the whole IR hole are directly input into the corresponding classifier one by one, and the class of the defect or the existence of the defect in the IR hole and the character can be identified.

Compared with the prior art, the invention has the beneficial effects that:

the method can realize the universal application of various models, effectively shield interference areas, accurately extract defects according to different detection standards, effectively identify the types of the defects by adopting a method of combining a classifier and deep learning, and acquire the appearance information of the defects.

The accuracy of defect extraction is improved by a region blocking method, and the interference of an optical band region and a transition band is avoided;

by means of edge correction, during edge detection, slight deviation of template matching caused by large image resolution can be avoided, large deviation still occurs, and edge breakage defects can be accurately extracted;

the defect identification is carried out by combining the traditional neural network classifier and the deep learning classifier, so that the defect types can be effectively identified by combining the reality under the condition of low time cost;

the whole defect detection method has strong universality, can be adjusted according to different products, can also adapt to different detection standards, and is convenient to modify parameters.

Drawings

Fig. 1 is a schematic diagram of a defect extraction process according to an embodiment of the present invention.

FIG. 2 is a schematic view of window detection according to an embodiment of the present invention.

Fig. 3 is a schematic view of a circular arc correction process according to an embodiment of the present invention.

Fig. 4 is a schematic diagram of a defect extraction condition of a silk-screen area according to an embodiment of the present invention.

FIG. 5 is a diagram illustrating defect types according to an embodiment of the present invention.

Fig. 6 is a schematic diagram of a defect classification process of a silk-screen area according to an embodiment of the present invention.

FIG. 7 is a schematic diagram of defect classification flow of hole sites, characters and edge breakouts according to an embodiment of the present invention.

FIG. 8 is a schematic diagram of IR hole defect classification according to an embodiment of the present invention.

FIG. 9 is a schematic diagram of line defect classification according to an embodiment of the present invention.

Detailed Description

The present invention will be described in further detail with reference to examples and drawings, but the present invention is not limited thereto.

FIG. 1 is a schematic diagram of a defect extraction process according to the present invention. The method for detecting the defects of the silk-screen area of the glass cover plate of the smart phone based on machine vision comprises the following steps:

s1, acquiring the mobile phone glass cover plate image: the method is characterized in that a 16K linear camera is adopted to respectively acquire images of the upper half mobile phone cover plate and the lower half mobile phone cover plate, the size is approximately controlled to be 2/3 of the whole mobile phone screen, the image resolution is 40000 multiplied by 16384, and the detection precision can reach 0.005 mm.

S2, reading related parameter information including template information of the outer contour, upper and lower limits of a global threshold, dimension information of a mobile phone cover plate, interference area information, the type of the cover plate, template information of a silk screen area, such as average gray value, pixel equivalent, holes and characters, and the like, and defect screening standard information of the silk screen area.

S3, detecting the window, detecting whether the mobile phone cover plate exceeds the window, if so, continuing the subsequent operation, otherwise, ending the detection and outputting error information, wherein the detection method is implemented as follows: the method comprises the steps of manufacturing a detection frame which is 1-10 pixels smaller than a window in advance, then primarily screening out an area of a mobile phone cover plate by utilizing a global threshold, then judging whether the area exists or not, then judging whether the area and the area of the detection frame have intersection or not, and if the intersection exists, indicating that the area exceeds the window and needs to be placed again. The step of preliminarily screening out the area of the mobile phone cover plate by using the global threshold specifically is to set a numerical value within the range of 0-255 by using the characteristic that the gray value of a single-channel image is between 0-255, and divide the image into two parts, namely, the area which is not needed in the image can be shielded.

S4, extracting large defects of the outline of the silk-screen printing area of the mobile phone cover plate, and specifically comprising the following steps:

s41, extracting the outline area of the mobile phone cover plate by global threshold segmentation by using the acquired standard mobile phone cover plate image, and making a standard template of the mobile phone cover plate; selecting a target area by manual framing, respectively segmenting a hole area and a character area by utilizing a local threshold value, manufacturing a standard template of holes and a standard template of characters, and storing all the standard templates as template files;

s42, when in on-line detection, matching the contour extracted by the same method from the detected image of the mobile phone cover plate with the standard template by using a template matching method, setting a matching score threshold, if the matching score reaches the threshold, acquiring the position information and the rotation information corresponding to the matching, otherwise, ending the detection and outputting error reporting information; the template matching method specifically comprises the steps of translating the input template pixel by pixel in a test image, setting a certain search angle range, searching a position and a deflection angle with the highest overlapped quality coefficient, and recording the information.

And S43, affine transforming the standard template to the corresponding coordinate and rotation angle by using the obtained information, comparing the coordinate and rotation angle with the outline of the picture of the mobile phone cover plate in detection, obtaining the difference between the standard template and the rotation angle by using Boolean operation, and outputting the information of the large defect when the minimum circumscribed circle of the difference region reaches the minimum standard of the large defect so as to finish the detection.

S5, dividing the detection area, including silk screen area, holes and character area, optical zone area and interference area, including the steps:

s51, obtaining the hole and character areas by adopting a template matching mode, after finding the areas one by one, respectively moving the standard template of the hole and the standard template of the character to corresponding positions by utilizing an affine transformation mode, obtaining the image of the area on the original image, and extracting the corresponding real area on the original image by utilizing the corresponding gray value threshold value, thereby obtaining the hole and character areas more accurately.

S52, obtaining an optical tape area, wherein the optical tape area is positioned at the top of the mobile phone cover plate, the angle deviated from the horizontal direction can be obtained by using the minimum external moment of the outline of the mobile phone cover plate, then the outline is corrected by using affine transformation, and a rectangle which takes the numerical value of the height as the length and the width of the outline as the width is created by taking the top edge of the outline as the starting point according to the height and the thickness of the optical tape area of the interference area information in the read information. And intersecting the rectangle with the outline to obtain the outline of the top, obtaining the optical band area by using the area expansion operation according to the obtained thickness information of the optical band area, and finally, carrying out inverse affine transformation to the original position.

And S53, removing the overlapping area, wherein when the pictures are taken, the upper half piece and the lower half piece are respectively taken for one mobile phone cover plate, and each picture is approximately 2/3 of the whole mobile phone screen, so that the overlapping part is required to be removed to avoid re-inspection. And according to the corrected outline, taking the top edge as a starting point, making a rectangle which is half the length of the mobile phone cover plate as the length and wide as the width of the mobile phone cover plate, intersecting the outline to obtain an outline area of the upper half, and finally carrying out inverse affine transformation to return to the original position.

S54, obtaining an interference area, wherein the image of the picture has a transition zone at the edge of the mobile phone cover plate, which affects the defect extraction of the silk screen area, and the area needs to be shielded, so that the outline of the inner frame and the outline of the outer frame are respectively expanded according to the thickness of the inner frame and the outer frame of the interference area information in the read information, and the interference area of the inner frame and the interference area of the outer frame are obtained.

S55, obtaining a silk-screen area, and removing the light band area, the interference area, the hole and the character area in the outline area of the original position to obtain a pure silk-screen area, so that the defects of the silk-screen area can be better extracted.

S6, acquiring the defects of each detection area, specifically comprising the following steps:

and S61, extracting the defects of the silk-screen area, and adding 30 gray values on the basis according to the average gray value in the read information to extract the defects of the silk-screen area.

S62, extracting the defects of the light band region, adding 40-60 gray values on the basis of the average gray value in the read information to extract the light reflection band region of the light band region, wherein in order to completely extract the light band, the intermittent region needs to be subjected to closed operation, the adjacent regions are connected, and false detection is avoided; and screening out the largest actual area, namely the light band, according to the length of the minimum external moment of each area, subtracting the area from the light band area to obtain a non-interference area, and adding a gray value of 20-40 on the basis of the average gray value to extract the defects in the light band area.

And S63, merging all extracted defects.

S7, correcting the outline of the mobile phone cover plate and acquiring the defect information of edge breakage, comprising the following steps:

s71, on the basis of the corrected profile degree, dividing the contour into an outer frame contour and an inner frame contour, firstly performing opening operation by using a long-strip rectangle with the length of 30-100 pixels and the width of 1-5 pixels, and removing the edge bulge in the vertical direction of less than 0-100 pixels; then, performing opening operation by using a long rectangular with the length of 1-5 pixels and the width of 30-100 pixels, and removing edge bulges in the horizontal direction of less than 0-100 pixels; the two rectangles are used for respectively carrying out closing operation, and edge pits in the vertical direction and edge pits in the horizontal direction which are smaller than 0-100 pixels can be respectively filled. The opening operation is a process of corrosion first and then expansion, and is used for eliminating small objects, separating the objects at fine points, smoothing the boundary of a larger object and not obviously changing the area of the larger object. The closed operation is a process of expansion and corrosion. The filling material is used for filling fine holes in an object, connecting adjacent objects and smoothing the boundary of the objects without obviously changing the area of the objects.

S72, after processing vertical and horizontal edges, processing arc parts, as shown in the schematic diagram of the arc correction flow of FIG. 3, creating the minimum positive external moment of the outline, performing difference operation with the original outline, performing open operation with a circle with the radius of 20-30 pixels, thus obtaining an arc area, creating the minimum positive external moment on the basis of the area, and taking 0.2 times of the length of the rectangle as the radius of the open operation of the circle, thus preventing the overlarge radius and destroying the shape of the original outline, thus removing the convex part of the arc part; and then, the closed operation is carried out in the same way, so that the pit part of the circular arc can be filled.

And S73, combining the corrected vertical, horizontal and circular arc profiles to form a corrected profile, carrying out the same processing on the internal and external profiles to obtain a complete profile, and finally carrying out inverse affine transformation to the original position.

And S74, performing Boolean operation on the obtained corrected outline and the original detection outline to obtain the defect information of edge breakage.

S8, classifying the defects by using a neural network classifier, namely point-shaped defects, linear defects and surface-shaped defects,

fig. 4 is a schematic diagram showing a defect extraction condition of a silk-screen area, fig. 5 is a schematic diagram showing defect types, and the extracted defects of the silk-screen area are classified according to the defect classification process of the silk-screen area shown in fig. 6, which includes the following steps:

s81, realizing clustering processing, and connecting the regions to obtain connected domain characteristics; all the regions with clustering radius of R pixels, area larger than S pixels, and number of small defect regions larger than N are clustered and connected, where R is 50, S is 15, and N is 50 in this embodiment.

S82, obtaining feature vectors composed of the features of the major axis radius length of the equivalent ellipse, the minor axis radius length of the equivalent ellipse, the roundness, the maximum diameter, the maximum inscribed rectangle width, the minimum circumscribed circle radius, the maximum inscribed circle radius, the tightness, the total length of the contour line, the convexity, the squareness, the filling power, the ratio of the major axis to the minor axis and the like of the defect by utilizing the extracted sample defects, and well labeling the feature vector group of each sample defect according to the type of the point line and the surface to form a sample library

And S83, inputting the sample library into the neural network model, and training a point-line planar neural network classifier capable of identifying the defects.

S84, when in on-line detection, the defects obtained by extraction can be directly input into the neural network classifier one by one, and the type of the defect belonging to the point line surface can be identified.

Wherein, the neural network classifier has the capability of self-adapting or learning to the surrounding environment. When the artificial neural network is in an input-output mode, it can minimize errors by self-adjustment, i.e., learning by training. For some factors which are difficult to parameterize, rules can be automatically summarized through training. The input/output mode is mixed with error information, and the whole system is not affected seriously. Compared with the traditional empirical curve fitting model, the sensitivity of the artificial neural network to noise and incomplete information is low. The artificial neural network has better extrapolation, namely, the knowledge learned from partial samples is popularized to the whole samples from training. Training of artificial neural networks can take a significant amount of time, but once training is complete, the results can be quickly calculated from the given inputs.

S9, screening the defects according to the defect definition standard, comprising the following steps:

aiming at point-like defects:

if D is less than 0.1mm, neglecting;

if D is more than or equal to 0.1mm and less than or equal to 0.2mm and D is more than or equal to 10mm, the mobile phone screen is good within 2, and the mobile phone screen is defective within 3;

if D is larger than 0.2mm, the mobile phone screen is a defective product;

wherein D is the minimum circumscribed circle diameter of the point defect, and D is the distance between a point and the point;

for linear defects:

if W is less than 0.02mm, the product is treated as a good product;

if W is more than or equal to 0.02mm and less than or equal to 0.05mm, d is more than or equal to 5mm, L is less than or equal to 3, the mobile phone screen is good when the number is less than or equal to 2, and the mobile phone screen is defective when the number is less than or equal to 3;

if W is larger than 0.05mm or L is larger than 3mm, the mobile phone screen is a defective product;

wherein W is the linear defect width L and the linear defect length, d is the different linear defect interval;

aiming at the surface defect:

if D > 0.5mm or S > 0.2mm²Indicating that the mobile phone screen is a defective product;

wherein D is the maximum inscribed circle diameter of the planar defect, and S is the planar defect area.

As shown in the schematic flow chart of defect classification of hole sites, characters and edge breakage in fig. 7, an original image is first cut from the previously obtained IR hole and character regions to obtain a corresponding image, and then the image is input to a corresponding deep learning classifier in the same manner as the operation of line defect classification.

S10, deep defect classification of line defects, IR hole defects and character defects by deep learning is realized because the background of the defects is very complex or the shape of the defects is similar, the defects are difficult to identify by using the traditional neural network classifier, the main characteristics of the sample can be automatically learned by deep learning, the problems can be accurately solved, but the time cost is higher than that of the traditional classifier, and therefore, the deep defect classification is more practical when being used for processing the problem which cannot be effectively processed by the traditional classifier. The line defect classification comprises broken filaments and scratches, the IR hole defect and the character defect classification comprises edge breakage and internal defects, and the method specifically comprises the following steps:

s101, selecting a preprocessing model, wherein the model is a predicted _ dl _ classifierce _ enhanced model provided by HALCON software, and the model is used for improving the training and learning rate and can be directly called without understanding the model structure. The model provides 224 multiplied by 224 image samples, and provides parameters of the classification number, the weights of a training set, a verification set and a test data set, network weight, learning rate, iteration period, the size of batch size, the updating period of the learning rate and the updating magnification of the learning rate for setting, thereby facilitating the adjustment of the recognition rate of the classifier.

S102, because the sizes of the defects are different, in order to avoid defect distortion caused in the scaling process, special treatment is needed, if the width or the length of the minimum positive external moment of the defects is smaller than the standard material size, the part smaller than the width or the length of the minimum positive external moment of the defects is filled with the gray value average value of a frame which is 2-20 pixels larger than the minimum positive external moment of the defects, and the average gray value near the defects is mainly extracted to be used as a filling value; if the value is larger than the preset value, the value is kept unchanged; the sample size of the defective material can be expanded by mirroring, rotating and changing the brightness of the picture; and then, image preprocessing, mainly the standardization of an image channel and the normalization of an image size. And labeling the samples according to types.

S103, adjusting deep learning parameters, inputting the prepared samples into a pre-training model, and starting off-line training. Specific parameter settings are shown in table 1:

TABLE 1 Defect Classification deep learning parameter settings

S104, after the trained deep learning classifier is obtained, the image and the character image of the defect or the whole IR hole are directly input into the corresponding classifier one by one, so that the type of the defect or the existence of the defect in the IR hole and the character can be identified. FIG. 8 is a schematic diagram of IR hole classification, and FIG. 9 is a schematic diagram of line defect classification.

The deep learning is used for making up the defects of the neural network classifier and classifying the defects more specifically, and the deep learning can be used for extracting effective characteristics from the gray value of the defect image for classification, so that defects which are similar in shape but are actually non-homogeneous can be distinguished. The deep learning is characterized in that the machine learning needs to be subjected to multi-step feature extraction and then input into a model to train a function, an optimal feature combination needs to be searched, and the adaptability is poor; and the deep learning can automatically learn effective characteristic information, which is equivalent to nesting of a plurality of functions, and the described information has high complexity and stronger adaptability.

S11, counting the shape information of each defect, including area, position, length, width, minimum circumcircle, maximum inscribed circle and external ellipse, to make the feature of the defect more digital and visual, and provide a certain basis for defect screening.

The invention has the following advantages:

the method can realize the universal application of various models, effectively shield interference areas, accurately extract defects according to different detection standards, effectively identify the types of the defects by adopting a method of combining a classifier and deep learning, and acquire the appearance information of the defects.

The accuracy of defect extraction is improved by a region blocking method, and the interference of an optical band region and a transition band is avoided;

by means of edge correction, during edge detection, slight deviation of template matching caused by large image resolution can be avoided, large deviation still occurs, and edge breakage defects can be accurately extracted;

the defect identification is carried out by combining the traditional neural network classifier and the deep learning classifier, so that the defect types can be effectively identified by combining the reality under the condition of low time cost;

the whole defect detection method has strong universality, can be adjusted according to different products, can also adapt to different detection standards, and is convenient to modify parameters.

The embodiments of the present invention are not limited to the above-described embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and they are included in the scope of the present invention.