阅读说明：本技术 一种基于深度学习算法的液体内异物与气泡智能视觉检测系统 (Intelligent visual detection system for foreign matters and bubbles in liquid based on deep learning algorithm ) 是由 王芳 潘慧敏 李新倩 于 2019-10-08 设计创作，主要内容包括：一种基于深度学习算法的液体内异物与气泡智能视觉检测系统,其特征在于：包括多个工站、多个光源和多个工业相机,以及图像采集模块、图像预处理及筛选模块、分组转发模块、两级分类模块和分类结果统计模块。本发明创造性地在流水线上直接检测自旋后运动中的液瓶,去掉了输液瓶旋转后急停的步骤,这样不仅能够简化灯检机硬件电路设计,而且由于不再需要旋转后急停再进行拍照,所以减少了时间损耗,使检测速度变快；首创瓶身水平放置后检测方法,大大提高异物检出概率,对漂浮异物也有较好的检出效果,使用多相机多工站,做到输液瓶的内外全方位检测,另外,针对快速沉底异物单独使用一个工站进行检测,检出效果可达100％。(The utility model provides a foreign matter and bubble intelligent vision detecting system in liquid based on degree of depth learning algorithm which characterized in that: the system comprises a plurality of work stations, a plurality of light sources, a plurality of industrial cameras, an image acquisition module, an image preprocessing and screening module, a grouping and forwarding module, a two-stage classification module and a classification result counting module. The invention creatively detects the liquid bottle in motion after spinning directly on the production line, and removes the step of sudden stop after the rotation of the liquid bottle, thus not only simplifying the design of a hardware circuit of a lamp inspection machine, but also reducing the time loss and speeding up the detection as the sudden stop after the rotation is not needed and the picture is taken; the detection method is initiated after the bottle body is horizontally placed, the foreign matter detection probability is greatly improved, the floating foreign matter is well detected, the multi-camera multi-station is used, all-around detection inside and outside the infusion bottle is achieved, in addition, one station is independently used for detecting the fast sinking foreign matter, and the detection effect can reach 100%.)

1. The utility model provides a foreign matter and bubble intelligent vision detecting system in liquid based on degree of deep learning algorithm which characterized in that: the system comprises a plurality of work stations, a plurality of light sources, a plurality of industrial cameras, an image acquisition module, an image preprocessing and screening module, a grouping and forwarding module, a two-stage classification module and a classification result counting module;

the work stations comprise a hanging ring work station, a bottle cap work station, an appearance particle work station, a simulated manual work station and a bottom sinking work station; the light sources are all connected with a light source controller; the flying ring work station is provided with an annular light source and a plurality of industrial cameras, the bottle cap work station is provided with an annular light source and a plurality of industrial cameras, the appearance particle work station is provided with two arc-surface light sources and a plurality of industrial cameras, the imitation human work station is provided with a large backlight source and a plurality of industrial cameras, the bottom sinking work station is provided with a small backlight source and a plurality of industrial cameras, and all targets are horizontally placed on all the work stations.

2. The intelligent visual detection system for foreign matters and bubbles in liquid based on the deep learning algorithm as claimed in claim 1, wherein: rings worker station with bottle lid worker station all includes rotatable manipulator, and the side at the manipulator is laid to the industry camera, and the light source configuration is in the lower part or the upper portion of manipulator, outward appearance particle worker station imitative worker station with heavy end worker station all includes a conveyer belt, and the both sides and the upper portion at the conveyer belt are laid to the industry camera, the cambered surface light source is laid in the both sides of conveyer belt, and big backlight and little backlight configuration are in the conveyer belt lower part.

3. The intelligent visual detection system for foreign matters and bubbles in liquid based on the deep learning algorithm as claimed in claim 1, wherein: the image acquisition module is connected with each industrial camera and acquires image data of each industrial camera;

the image preprocessing and screening module is used for denoising the image acquired by the image acquisition module by using median filtering, performing corresponding edge extraction on the region of interest by using second-order Sobel filtering, extracting an engineering threshold value by segmenting and calculating the image, and screening out abnormal image frames by using the engineering threshold value;

the grouping forwarding module uniformly encodes the image data processed by the image preprocessing and screening module and sends the image data to the two-stage classification module; the code sets unique ID including the group information, the current time information and the preprocessing result information; grouping the information of the group according to the abnormal type or the processing scheme;

the two-stage classification module carries out suspicious anomaly detection and second-stage anomaly target detection on the image data sent by the packet forwarding module;

the suspicious anomaly detection is used for primarily screening a specific target image to obtain an image frame of the specific target; the second-stage abnormal target detection is used for training a pre-collected actual sample image through a convolutional neural network, monitoring the appearance of a hanging ring or a bottle cap or a bottle body under the current target, or foreign matters at the bottom of the hanging ring or the bottle body, or particles and hair-shaped floating matters in liquid to obtain abnormal alarm information, and distinguishing the target abnormal degree under the current target from an image frame of a specific target obtained by suspicious abnormal detection;

the classification result counting module counts the classification results obtained by the two-stage classification module, makes classification result judgment through continuous time judgment of the classification results, and stores corresponding result data to the local;

the classification result judgment adopts time continuity judgment, namely a final judgment result is obtained by judging whether the same result appears in a set time period determined according to different requirements and different targets, and the obtained result is stored locally for model optimization under the line;

the image acquisition module adopts monitoring equipment with an output function or a processing unit capable of executing image data extraction;

the image preprocessing and screening module uses a monitoring device with an image data processing function or a processing unit capable of executing image data processing.

4. The intelligent visual detection system for foreign matters and bubbles in liquid based on the deep learning algorithm as claimed in claim 3, wherein: the two-stage classification module comprises a first-stage neural network and a second-stage neural network; the first-stage neural network consists of a neural network model, and the neural network model is RCNN, Fast R-CNN, Faster R-CNN, FPN, YOLOv1, YOLO v2, SSD or RetinaNet; the second-stage neural network uses a convolutional neural network model, and the number of the neural network models is determined according to the number of the detection targets.

5. The intelligent visual detection system for foreign matters and bubbles in liquid based on the deep learning algorithm as claimed in claim 3, wherein: the system also comprises a master control center, wherein the master control center comprises a message management module and an early warning and display module; the message management module is used for receiving and transmitting event messages among the modules; and the early warning and display module is used for receiving the classification result, sending the corresponding result to the abnormal target removing equipment and outputting the result to the display equipment.

6. The intelligent visual detection system for foreign matters and bubbles in liquid based on the deep learning algorithm as claimed in claim 5, wherein: the abnormal target removing equipment adopts a medicine bottle removing machine, and the display equipment is a computer client or a webpage end.

Technical Field

The invention relates to the technical field of intelligent visual detection, in particular to an intelligent visual detection system for foreign matters and bubbles in liquid based on a deep learning algorithm.

Background

In recent years, the variety and quantity of traditional Chinese medicine oral liquid on the market are rapidly increased. However, the currently studied method for detecting visible foreign matters is mainly a traditional static detection method for large infusion bottles and ampoule bottles. The traditional Chinese medicine liquid bottle body is small, equipment which continues to move on a production line after spinning can be realized in manufacturing, however, the technology for correspondingly detecting impurities in the liquid medicine bottle in linear motion on the production line does not keep pace with research.

At present, most of the existing methods still stay in a manual detection stage, and the detection method is that trained workers observe whether visible foreign matters exist in liquid medicine under a fluorescent lamp, if the visible foreign matters exist, the bottles are discarded in a large container specially containing unqualified products, and the method is called manual lamp detection. Because long-time work can make people tired of eyes, influence the detection effect to need every staff to keep high responsibility all the time and carefully pay attention to the work of oneself, consequently the manual testing result all is relevant with numerous subjective factors such as everyone's eyesight, the responsibility heart, mood even to a great extent, can not guarantee the objective accuracy of testing result. Some enterprises stipulate that a lamp inspection worker needs to rest for a period of time every time the lamp inspection worker works for a short period of time in order to ensure the inspection quality, so that the influence of subjective factors on the inspection result can be reduced to a certain extent, the production efficiency is greatly reduced, the final liquid medicine quality inspection link becomes a bottleneck link influencing the speed of the production process of medicine products, and the final liquid medicine quality inspection link also becomes an important factor influencing the benefits of the medicine production industry. In China, due to the production process, the quality of containers for containing liquid medicine is uneven, the thickness of the bottle wall is uneven, and the density distribution is uneven, so that although developed countries such as Germany and Japan already have mature full-automatic lamp inspection equipment, the equipment is high in price, and due to the difference of container standards at home and abroad, the defect rate is too high, so that enterprises are not paid.

Therefore, it is imperative to develop a full-automatic lamp inspection device suitable for the national conditions of China, that is, the device simulates the eyes of human beings for inspection by adopting a machine vision method, but in the aspect of vision, no full-automatic lamp inspection machine capable of objectively and accurately detecting foreign matters exists in the market at present due to the following difficulties:

(1) random noise in the image during imaging and other numerous disturbances. Interference dominant packets in imaging processes

The method comprises the following steps:

a) because the production environment is not clean, the air contains tiny suspended matters or dust particles and the like;

b) the defects of the container, such as cracks on a glass bottle, can cause strong reflection, and cause severe tracking and identification of foreign matters

Heavy interference;

(2) the carving characters, the dents and the like on the bottle wall can occasionally shield foreign matters, so that the identification error is caused;

(3) due to the influence of machining precision, a bottle for containing liquid medicine can move in the imaging process, and detection is increased

The difficulty of identification;

(4) different products contain different types and quantities of foreign matters, even different individuals of the same product can contain various foreign matters, and the number and the size of the foreign matters are different, so that a universal and effective detection and identification method needs to be found. The requirements of detection speed and detection precision increase the detection difficulty. Due to the requirement of detection speed, the number of the acquired image frames is limited on one hand, and the application of advanced detection algorithms is also limited on the other hand.

Disclosure of Invention

The invention mainly aims to provide an intelligent visual detection system for foreign matters and bubbles in liquid based on a deep learning algorithm, which can effectively solve the problems in the background technology.

In order to achieve the purpose, the invention adopts the technical scheme that: the utility model provides a foreign matter and bubble intelligent vision detecting system in liquid based on degree of deep learning algorithm which characterized in that: the system comprises a plurality of work stations, a plurality of light sources, a plurality of industrial cameras, an image acquisition module, an image preprocessing and screening module, a grouping and forwarding module, a two-stage classification module and a classification result statistical module; the work stations comprise a hanging ring work station, a bottle cap work station, an appearance particle work station, a simulated manual work station and a bottom sinking work station; the light sources are all connected with a light source controller; the flying ring work station is provided with an annular light source and a plurality of industrial cameras, the bottle cap work station is provided with an annular light source and a plurality of industrial cameras, the appearance particle work station is provided with two arc-surface light sources and a plurality of industrial cameras, the imitation human work station is provided with a large backlight source and a plurality of industrial cameras, the bottom sinking work station is provided with a small backlight source and a plurality of industrial cameras, and all targets are horizontally placed on all the work stations.

Further, rings worker station with bottle lid worker station all includes rotatable manipulator, and the side at the manipulator is laid to the industry camera, and the light source configuration is in the lower part or the upper portion of manipulator, outward appearance particle worker station imitative worker station with end sinking worker station all includes a conveyer belt, and the industry camera is laid in the both sides and the upper portion of conveyer belt, the cambered surface light source is laid in the both sides of conveyer belt, and big backlight and little backlight configuration are in the conveyer belt lower part.

Further, the image acquisition module is connected with each industrial camera and acquires image data of each industrial camera; the image preprocessing and screening module is used for denoising the image acquired by the image acquisition module by using median filtering, performing corresponding edge extraction on an interested region through second-order Sobel filtering, extracting an engineering threshold value through segmenting and calculating the image, and screening out abnormal image frames through the engineering threshold value; the grouping forwarding module uniformly encodes the image data processed by the image preprocessing and screening module and sends the image data to the two-stage classification module; the code comprises the information of the group to which the code belongs, the current time information and the preprocessing result information, and unique ID is set; grouping the information of the group according to the abnormal type or the processing scheme; the two-stage classification module carries out suspicious anomaly detection and second-stage anomaly target detection on the image data sent by the packet forwarding module; the suspicious anomaly detection is used for primarily screening a specific target image to obtain an image frame of the specific target; the second-stage abnormal target detection is used for training a pre-collected actual sample image through a convolutional neural network, monitoring the appearance of a hanging ring or a bottle cap or a bottle body under the current target, or foreign matters at the bottom of the hanging ring or the bottle body, or particles and hair-shaped floating matters in liquid to obtain abnormal alarm information, and distinguishing the target abnormal degree under the current target from an image frame of a specific target obtained by suspicious abnormal detection; the classification result counting module counts the classification results obtained by the two-stage classification module, makes classification result judgment through continuous time judgment of the classification results, and stores corresponding result data to the local; the classification result judgment adopts time continuity judgment, namely a final judgment result is obtained by judging whether the same result appears in a set time period determined according to different requirements and different targets, and the obtained result is stored locally for model optimization under the line; the image acquisition module adopts monitoring equipment with an output function or a processing unit capable of executing image data extraction; the image preprocessing and screening module uses a monitoring device with an image data processing function or a processing unit capable of executing image data processing.

Further, the two-stage classification module comprises a first-stage neural network and a second-stage neural network; the first-stage neural network consists of a neural network model, and the neural network model is RCNN, Fast R-CNN, FasterR-CNN, FPN, YOLOv1, YOLO v2, SSD or RetinaNet; the second-stage neural network uses a convolutional neural network model, and the number of the neural network models is determined according to the number of the detection targets.

The system further comprises a master control center, wherein the master control center comprises a message management module and an early warning and display module; the message management module is used for receiving and transmitting event messages among the modules; and the early warning and display module is used for receiving the classification result, sending the corresponding result to the abnormal target removing equipment and outputting the result to the display equipment.

Furthermore, the abnormal target removing device adopts a medicine bottle removing machine, and the display device is a computer client or a webpage end.

Compared with the prior art, the invention has the following beneficial effects:

the invention creatively detects the liquid bottle in motion after spinning directly on the production line, and removes the step of sudden stop after the rotation of the liquid bottle, thus not only simplifying the design of a hardware circuit of a lamp inspection machine, but also reducing time loss and quickening the detection speed because the sudden stop after the rotation is not needed and the shooting is carried out; the detection method is initiated after the bottle body is horizontally placed, the detection probability of foreign matters is greatly improved, the floating foreign matters are well detected, the multi-camera multi-station is used, the detection of all directions inside and outside the infusion bottle is realized, in addition, one station is independently used for detecting the fast sinking foreign matters, and the detection effect can reach 100%.

Detailed Description

In order to make the technical means, the creation characteristics, the achievement purposes and the effects of the invention easy to understand, the invention is further described with reference to the specific embodiments.