阅读说明：本技术 基于深度学习的纸巾产品包装检测方法 (Paper towel product packaging detection method based on deep learning ) 是由 邓中民 胡灏东 柯薇 于 2021-07-23 设计创作，主要内容包括：本发明涉及一种基于深度学习的纸巾产品包装检测方法,包括以下步骤：1、利用多种不同包装的纸巾盒五个面的图片分别对每一面对应的网络模型进行训练,得到每一面对应的最终网络模型；2、实际进行某一类纸巾盒的包装生产检测时,将该类标准纸巾盒样本和实际纸巾盒的五个面图像分别输入每一面相应的网络模型中,得到标准特征参数和实际特征参数,计算同一面的实际特征参数与标准特征参数之间的欧式距离β,若β小于预设阈值α则判断该面纸巾盒包装完好,若β大于预设阈值α则判断该面纸巾盒包装破损。本发明具有识别效率高、精准度可以自主调节、便于二次部署等优点。可针对多种纸巾盒包装进行检测,具有可移植性和广泛的应用性。(The invention relates to a paper towel product packaging detection method based on deep learning, which comprises the following steps: 1. respectively training the network model corresponding to each face by using pictures of five faces of the tissue boxes packaged in different ways to obtain a final network model corresponding to each face; 2. when the packaging production detection of a certain tissue box is actually carried out, five face images of a standard tissue box sample and an actual tissue box are respectively input into a corresponding network model of each face to obtain a standard characteristic parameter and an actual characteristic parameter, the Euclidean distance beta between the actual characteristic parameter and the standard characteristic parameter of the same face is calculated, if beta is smaller than a preset threshold value alpha, the tissue box is judged to be packaged perfectly, and if beta is larger than the preset threshold value alpha, the tissue box is judged to be packaged damaged. The invention has the advantages of high recognition efficiency, self-regulation of precision, convenience for secondary deployment and the like. Can detect various tissue box packages, and has portability and wide applicability.)

1. The product packaging detection method of the tissue box based on deep learning is characterized by comprising the following steps:

step 1, acquiring five face images of a plurality of tissue box products except for bottom faces, wherein n packaging types of the tissue boxes are set, namely A1 and A2 … Ai … An, and i is 1 … n, so that m pictures of each face of each tissue box Ai are acquired respectively, and m and n are natural numbers;

step 2, dividing the collected images and the standard tissues into two categories of damaged images and intact images, namely, each image of each tissue box has two categories of intact images and damaged images; the pictures of all kinds of tissue boxes on each face form a training set of the face;

step 3, respectively training each picture of the tissue box, when training a certain face, firstly randomly selecting the type of the tissue box, if the Ai type is selected, selecting one of intact pictures of the Ai type tissue box in the training set of the face as a standard, selecting one of intact pictures as a positive example, then randomly selecting one of damaged pictures of the Ai type tissue box as a negative example, the standard, the positive example and the negative example form a standard-good-bad composition triple, inputting triple network training, repeating the steps until a network loss function reaches a convergence condition, finishing the face training, wherein the final triple network is the corresponding network model of the face, and respectively training to obtain the network models corresponding to five faces;

step 4, when the packaging production detection of a certain type of tissue box is actually carried out, five face images of a sample of the standard tissue box are input into a corresponding network model of each face to obtain a standard characteristic parameter of each face; the method comprises the steps of collecting images of five surfaces of a produced tissue box, inputting the images of each surface into a corresponding network model respectively to obtain actual characteristic parameters of each surface, calculating Euclidean distance beta between the actual characteristic parameters and standard characteristic parameters of the same surface, judging that the tissue box is well packaged if beta is smaller than a preset threshold value alpha, and judging that the tissue box is damaged if beta is larger than the preset threshold value alpha.

2. The deep learning based detection method for packaging of tissue products according to claim 1, further comprising the step of 5: and (4) manually checking the judgment result in the step (4), and if the judgment result is wrong, inputting the corresponding tissue box picture and the corresponding network model to optimize and adjust the network model.

3. The method for detecting the package of the tissue product based on the deep learning of claim 1, wherein the step 2 further comprises the processing steps of removing uneven illumination and denoising of the collected image.

4. The method of claim 1, further comprising the step of augmenting the training set with image enhancement in step 2.

Technical Field

The patent belongs to the field of product packaging quality detection, and particularly relates to a tissue product packaging detection method based on deep learning.

Background

The main factors affecting the quality of the tissue package are the opening of the tissue which is cracked, the opening of the tissue which is not sealed and the side seal which is cracked, resulting in the tissue overflowing from the package. The general image processing based on computer vision is used for completing the package detection of paper towel products, the method only aims at a single product, the whole neural network needs to be retrained when the product is replaced, and the accuracy of rejecting defective products cannot be dynamically controlled. Or the migration network retrains also needs a larger amount of new packaged pictures, and the system lacks portability and wide applicability. The package detection is an indispensable ring of a product production line, and if the detection speed is not fast enough, the production efficiency of an enterprise is influenced, and the benefit of the enterprise is directly influenced. Moreover, a manual detection method is adopted in the traditional production line, and the manual detection is greatly influenced by environmental factors, has more subjective factors and has low efficiency. And manual detection can not check the quality condition of the assembly line in real time, which is not beneficial to the construction of informatization and intelligent factories.

Disclosure of Invention

The invention aims to provide a paper towel product packaging detection method based on deep learning, which is accurate in detection result and high in efficiency.

The product packaging detection method of the tissue box based on deep learning comprises the following steps:

step 1, acquiring five face images of a plurality of tissue box products except for bottom faces, wherein n packaging types of the tissue boxes are set, namely A1 and A2 … Ai … An, and i is 1 … n, so that m pictures of each face of each tissue box Ai are acquired respectively, and m and n are natural numbers;

step 2, dividing the collected images and the standard tissues into two categories of damaged images and intact images, namely, each image of each tissue box has two categories of intact images and damaged images; the pictures of all kinds of tissue boxes on each face form a training set of the face;

step 3, respectively training each picture of the tissue box, when training a certain face, firstly randomly selecting the type of the tissue box, if the Ai type is selected, selecting one of intact pictures of the Ai type tissue box in the training set of the face as a standard, selecting one of intact pictures as a positive example, then randomly selecting one of damaged pictures of the Ai type tissue box as a negative example, the standard, the positive example and the negative example form a standard-good-bad composition triple, inputting triple network training, repeating the steps until a network loss function reaches a convergence condition, finishing the face training, wherein the final triple network is the corresponding network model of the face, and respectively training to obtain the network models corresponding to five faces;

step 4, when the packaging production detection of a certain type of tissue box is actually carried out, five face images of a sample of the standard tissue box are input into a corresponding network model of each face to obtain a standard characteristic parameter of each face; the method comprises the steps of collecting images of five surfaces of a produced tissue box, inputting the images of each surface into a corresponding network model respectively to obtain actual characteristic parameters of each surface, calculating Euclidean distance beta between the actual characteristic parameters and standard characteristic parameters of the same surface, judging that the tissue box is well packaged if beta is smaller than a preset threshold value alpha, and judging that the tissue box is damaged if beta is larger than the preset threshold value alpha.

Further, the method also comprises the step 5: and (4) manually checking the judgment result in the step (4), and if the judgment result is wrong, inputting the corresponding tissue box picture and the corresponding network model to optimize and adjust the network model.

Further, step 2 includes a processing step of removing uneven illumination and denoising for the collected image.

Further, step 2 includes a step of expanding the training set for image enhancement.

The invention has the beneficial effects that: the invention combines the triple network framework in the deep learning with the residual error network Resnet deep learning network, and improves the traditional image processing technology based on computer vision. Let this system have that recognition efficiency is high, the precision can independently be adjusted, the secondary of being convenient for deploys advantages such as, train the back to every face of paper handkerchief box in earlier stage, when changing the kind of production paper handkerchief box, need not to train the model again, have portability and extensive application.

Drawings

FIG. 1 is a schematic flow diagram of the present invention;

fig. 2 is a schematic diagram of a triple network structure.

Detailed Description

The technical solution of the present invention will be described clearly and completely with reference to the accompanying drawings, and it should be understood that the described embodiments are only partial embodiments of the present invention, rather than full embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

As shown in fig. 1, a machine learning based detection method for tissue product packaging includes the following steps;

1. pre-training of the neural network. Five-face (except bottom face) images of all the tissue box packages which are possibly produced in the follow-up process of the factory assembly line are shot by the industrial camera and stored. The image of each surface is taken as a training set of neural networks.

2. The images of the five surfaces are distinguished and divided, and the image of each surface is divided into two categories of damaged images and intact images for training a neural network.

3. Preprocessing the acquired images such as uneven illumination, denoising and the like, and expanding a training set by adopting rotation, translation and histogram enhancement image enhancement algorithms.

4. The triple network deep learning algorithm is adopted to train the network entries, and the network framework is shown in fig. 2.

The data is in the form of triplets (X0, X1, X2), one picture X0 is selected from the broken (or good) training set, and the other two pictures X1, X2 are selected, one from the broken X1 (or good) training set as a homogeneous sample, and one from the good X2 (or good) training set as a heterogeneous sample. Inputting the three pictures X0, X1 and X2 into the same NET, wherein the NET selects a residual error neural network Resnet, the number of layers of the residual error network Resnet is deeper, and the residual error network Resnet can be used as a feature extractor to extract more feature values and simultaneously avoid overfitting of the network;

for example, for 3 types of existing A, B, C packages, a photo of A, B, C types of packages is taken, assuming that a 100B 100C 100 are intact and a 100B 100C 100 are broken, then one of the intact sides of a (or B or C) is selected as a standard, then one of the intact sides of a (or B or C) is selected as a positive example, then one of the broken sides of a (or B or C) is selected as a negative example, a standard-good-bad composition triple is formed, the training is performed by dropping the network, then the photo-selecting step is repeated and the training is performed again, and the steps are repeated n times until the network converges and is OK.

5. After passing through NET, the picture data X0, X1 and X2 are converted into one-dimensional arrays, X0 ', X1 ' and X2 ', and the one-dimensional vectors can reflect the characteristics of the pictures in a numerical mode. The differences between samples X1, X0, X2 were compared by euclidean distance (equation 1). (Note: Distance function is used to calculate the Euclidean Distance of two vectors)

Distance(X0’,X1’)＝||X0’-X1’||₂(formula 1)

Distance(X0’,X2’)＝||X0’-X2’||₂

Training the characteristic parameter W through a training set, so that the Euclidean Distance1(X0 ', X1') between the parameter X0 'of the sample X0 after passing through the residual network Resnet and the parameter X1' of the sample X1 of the same type after passing through the residual network Resnet is as small as possible. The euclidean Distance1(X0 ', X2') between the parameter X0 'of the sample X0 after passing through the residual network Resnet and the parameter X2' of the same type of sample X2 after passing through the residual network Resnet is made as large as possible. And training the characteristic parameters W through a training set to obtain ideal characteristic parameters W. And finishing the training of the neural network model.

6. Five faces of a picture of a kind of tissue package to be detected are led into the five network models, and the picture of the front face of the intact tissue package is X_{Is just}Obtaining fixed parameters X after leading in the neural network model_{Is just}', mixing X_{Is just}' stored in the system. Front photo X of the collected tissue package on the production line_{Test (experiment)}The information is guided into the system and then,obtaining a parameter X through a neural network model NET_{Test (experiment)}', comparison X_{Is just}' and X_{Test (experiment)}The euclidean distance between'. If the difference is within a, the two photos are proved to be small, the paper towel package to which the photos belong is judged to be intact (formula 2), otherwise, the paper towel package is considered to be broken (formula 3).

Distance(X_{Is just}’,X_{Test (experiment)}’)＝||X_{Is just}’-X_{Test (experiment)}’||₂>Alpha (formula 2)

Distance(X_{Is just}’,X_{Test (experiment)}’)＝||X_{Is just}’-X_{Test (experiment)}’||₂<Alpha (formula 3)

Alpha can be adjusted according to the accuracy required by the system.

7. The pictures of the paper towel package with the detection errors are led into the system again to train the neural network, so that the characteristic parameter W is continuously converged, and the detection accuracy of the system can be dynamically improved.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.