阅读说明：本技术 一种基于yolov3和CNN的盘头标识识别方法 (Pan head identification method based on yolov3 and CNN ) 是由 宣琦 赵佳康 朱城超 翔云 于 2019-09-09 设计创作，主要内容包括：一种基于yolov3和CNN的盘头标识识别方法,包括以下步骤：1)盘头图片获取：在实际的生产环境下,通过摄像头采集数据；2)盘头图片处理：先切割出原图中完整的盘头标识,用yolov3模型训练盘头标识的切割；3)再根据切割出的盘头标识图片切割出单个字符图片；4)用CNN卷积神经网络训练盘头标识字符的识别；5)系统调试：将盘头标识切割、字符切割和字符识别整合成一个系统；6)配置树莓派4的环境,移植系统,最后在树莓派上测试。本发明检测速度快和精度高,能提高生产效率和降低成本,具有较高的实际应用价值。(A disc head identification recognition method based on yolov3 and CNN comprises the following steps: 1) pan head picture acquisition: acquiring data through a camera in an actual production environment; 2) pan head picture processing: firstly, cutting out a complete pan head mark in an original image, and training the cutting of the pan head mark by using a yolov3 model; 3) cutting out a single character picture according to the cut-out coiled hair identification picture; 4) training the identification of the pan head identification character by using a CNN convolutional neural network; 5) debugging the system: integrating the cutting of the coiled head mark, the cutting of characters and the recognition of the characters into a system; 6) and (4) configuring the environment of the raspberry pi 4, transplanting the system, and finally testing on the raspberry pi. The invention has the advantages of high detection speed and high precision, can improve the production efficiency and reduce the cost, and has higher practical application value.)

1. A disc head identification recognition method based on yolov3 and CNN, characterized in that the method comprises the following steps:

s1: pan head picture acquisition: in an actual production environment, data are collected through a camera, and the pan heads printed with red marks in different light rays in the actual environment are respectively collected;

s2: pan head picture processing: firstly, cutting out a complete pan head mark in an original image, and training the cutting of the pan head mark by using a yolov3 model;

s3: cutting out a single character picture according to the cut-out coiled hair identification picture;

s4: training the identification of the pan head identification character by using a CNN convolutional neural network;

s5: debugging the system: integrating the cutting of the coiled head mark, the cutting of characters and the recognition of the characters into a system;

s6: and (4) configuring the environment of the raspberry pi 4, transplanting the system, and finally debugging on the raspberry pi.

2. The disc head identification recognition method based on yolov3 and CNN as claimed in claim 1, wherein in step S1, an actual system platform is built, the position of the camera is fixed, the camera is made to take a picture according to a triggering instruction that the disc head rolls over, and the picture is stored in JPG format.

3. The disc head identification recognition method based on yolov3 and CNN as claimed in claim 1 or 2, wherein said step S2 comprises the steps of:

s2-1: firstly, making a data set for training, collecting pictures containing pan heads, taking partial pictures as a training set, manually marking a frame of a pan head identification through a manual marking software Labelimg, marking the label type as 'biaozhi', generating a corresponding xml file by each picture with a corresponding name, making the pictures and the xml file into a VOC data set format, and generating a train.txt, val.txt, test.txt and train.txt files for training and testing;

s2-2: in the yolov3 model, a feature extraction network used by yolov3 is DarkNet53, the network is formed by overlapping residual error units, a pan head original image is used as the input of a neural network, firstly, a convolution feature map of N x N is output through the feature extraction network to generate N x N gird cells, and then, the type and the coordinates of an object are output through a prediction layer anchors, and a Bounding box coordinate (z coordinate)_x,z_y,z_w,z_h) The predictive calculation formula is as follows:

z_x＝σ(q_x)+c_x

z_y＝σ(q_y)+c_y

wherein q is_x,q_y,q_w,q_hCoordinates of the predicted center point and width and height of the prediction box for each Bounding box of yolov3, c_x,c_yIs the coordinate offset, p, of the grid cell grid where the center coordinate of the frame is located_w,p_hTo predict the width and height of the anterior anchor; firstly, determining which Grid cell the target center coordinate in the grountruth falls in, then predicting the target according to the Grid cell, wherein each Grid cell in yolov3 has three anchors, and calculating the predicted coordinate z_x,z_y,z_w,z_hWhen the loss is less, selecting three anchors with highest confidence level to calculate the loss of the group truth;

target class penalty L_cla(o, c) use is made of a binary cross-entropy loss, where o_ijE.g. {0,1}, which represents whether the target bounding box i is predicted to have a target or not,representing sigmoid probability of the target in the network prediction target boundary box i, and the calculation formula is as follows:

finally, NMS processing is carried out on the predicted target, and finally the rectangular box with the highest target probability is reserved, and the vertex coordinates of the rectangular box are output;

s2-3: cutting out pan head marks and output vertex coordinates according to the trained model, screening target pictures according to the approximate length-width ratio of the pan head marks, and correcting the screened pictures through OpenCV perspective transformation, wherein the calculation formula of the perspective transformation is as follows:

where src () is the vertex coordinates of the picture before input correction, dst () is the vertex coordinates after output correction, M is a 3 × 3 transformation matrix, and the transformation matrix M is calculated by applying the getpropertransform () function in OpenCV.

4. The disc head identification recognition method based on yolov3 and CNN as claimed in claim 1 or 2, wherein said step 3 comprises the steps of:

s3-1: firstly, carrying out noise reduction processing on a corrected target identification picture;

s3-2: superposing the denoised picture through binarization and HSV red extraction to obtain a final binarization picture;

s3-3: and dividing the obtained binary image into single characters, wherein the image is a 0-1 matrix, 0 is a background 1 and is a character, the characters are divided according to the number of 0-1 in each column, the number of 1 and 0 in each column is respectively counted, the characters are divided according to the 0-1 sum transformation of each column, and the divided 5 characters are sequentially added into a list to ensure that the sequence is not disordered.

5. The disc head identification recognition method based on yolov3 and CNN as claimed in claim 1 or 2, wherein said step 4 comprises the steps of:

s4-1: taking the cut single character as a data set for CNN character recognition model training, wherein the data set comprises 10 numbers and 24 capital letters of 34 classes, and the letters are not included in the letters O and I;

s4-2: and training the CNN network according to the prepared data set.

6. The method for identifying disc head identification based on yolov3 and CNN as claimed in claim 1 or 2, wherein the process of step S5 is to integrate the acquisition of disc head identification, the cutting of identification single character and the character identification into a system, first acquire disc head identification in original image according to trained yolov3 model, retain the best identification picture by screening the aspect ratio of the cut identification, cut out each character of the acquired disc head identification, acquire the sequence of the cut single character, then transfer into trained CNN model according to the sequence of disc head characters, identify each character, and obtain the complete disc head identification output.

7. The coiled hair identification recognition method based on yolov3 and CNN as claimed in claim 1 or 2, wherein in step S6, Python3, Tensorflow and OpenCV environments required by Raspy pie 4 are configured, the integrated system is transplanted to the Raspy pie, the debugging is performed on the Raspy pie, pictures are taken through a camera connected with the Raspy pie, and the coiled hair identification is output after algorithm processing.

Technical Field

The invention relates to deep learning, computer vision, target detection, character recognition and an embedded system, in particular to a pan head identification recognition method based on yolov3 and CNN.

Background

With the rapid development of scientific technology, the modern warp knitting industry develops towards high-speed production, intelligent control, multifunction, simple use, internet of things design and the like, and under the promotion of modern science and technology, the production efficiency of warp knitting products is improved, and the product quality is also improved. The pan head of the warp knitting machine is an important part on the warp knitting machine, and the selection of the pan head is important when the warp knitting machine is required to work efficiently.

Because the yarn types are more, in order to facilitate management and improve production efficiency, 5-shaped marks consisting of capital English letters and Arabic numerals are printed on each pan head, and different types of yarns can be distinguished by identifying the pan heads. The identification of the pan head in the actual factory production is also to identify the classification by manual work, and the technical defects exist: the accuracy is lower, production efficiency is lower.

Disclosure of Invention

In order to overcome the defects of low accuracy and low production efficiency of the existing pan head identification and classification through manual work, the invention provides the pan head identification method based on yolov3 and CNN, which can reduce the labor cost and effectively improve the automatic production efficiency under the condition of ensuring the identification accuracy.

The technical scheme adopted by the invention for solving the technical problems is as follows:

a disc head identification recognition method based on yolov3 and CNN comprises the following steps:

s1: pan head picture acquisition: in an actual production environment, data are collected through a camera, and the pan heads printed with red marks in different light rays in the actual environment are respectively collected;

s2: pan head picture processing: firstly, cutting out a complete pan head mark in an original image, and training the cutting of the pan head mark by using a yolov3 model;

s3: cutting out a single character picture according to the cut-out coiled hair identification picture;

s4: training the identification of the pan head identification character by using a CNN convolutional neural network;

s5: debugging the system: integrating the cutting of the coiled head mark, the cutting of characters and the recognition of the characters into a system;

s6: and (4) configuring the environment of the raspberry pi 4, transplanting the system, and finally debugging on the raspberry pi.

Further, in step S1, an actual system platform is built, the position of the camera is fixed, the camera is made to take a picture according to the trigger instruction that the pan head rolls, and the picture is stored in the JPG format.

Still further, the step S2 includes the steps of:

s2-1: firstly, making a data set for training, collecting pictures containing pan heads, taking partial pictures as a training set, manually marking a frame of a pan head identification through a manual marking software Labelimg, marking the label type as 'biaozhi', generating a corresponding xml file by each picture with a corresponding name, making the pictures and the xml file into a VOC data set format, and generating a train.txt, val.txt, test.txt and train.txt files for training and testing;

s2-2: in the yolov3 model, a feature extraction network used by yolov3 is DarkNet53, the network is formed by overlapping residual error units, the original coiled hair is used as the input of a neural network, firstly, a convolution feature map of N x N is output through the feature extraction network to generate N x N gird cells, and then, the category and the coordinate of a target are output through a prediction layer anchors. Bounding box coordinate (z)_x,z_y,z_w,z_h) The predictive calculation formula is as follows:

z_x＝σ(q_x)+c_x

z_y＝σ(q_y)+c_y

wherein q is_x,q_y,q_w,q_hFor yolov3 each BounCoordinate of center point predicted by the ding box and width and height of prediction frame, c_x,c_yIs the coordinate offset, p, of the grid cell grid where the center coordinate of the frame is located_w,p_hIn order to predict the width and height of anchors before predicting, firstly, determining which Grid cell the target center coordinate in the groudtruth falls into, then predicting the target according to the Grid cell, wherein each Grid cell in yolov3 has three anchors, and calculating the predicted coordinate z_x,z_y,z_w,z_hWhen the loss is less, selecting three anchors with highest confidence level to calculate the loss of the group truth;

target class penalty L_cla(o, c) use is made of a binary cross-entropy loss, where o_ijE.g. {0,1}, which represents whether the target bounding box i is predicted to have a target or not,

representing sigmoid probability of the target in the network prediction target boundary box i, and the calculation formula is as follows:

finally, NMS processing is carried out on the predicted target, and finally the rectangular box with the highest target probability is reserved, and the vertex coordinates of the rectangular box are output;

s2-3: cutting out pan head marks and output vertex coordinates according to the trained model, screening target pictures through the length-width ratio of the pan head marks, and correcting the screened pictures through OpenCV perspective transformation, wherein the calculation formula of the perspective transformation is as follows:

where src () is the vertex coordinates of the picture before input correction, dst () is the vertex coordinates after output correction, M is a 3 × 3 transformation matrix, and the transformation matrix M is calculated by applying the getpropertransform () function in OpenCV.

Still further, the step 3 includes the steps of:

s3-1: firstly, carrying out noise reduction processing on a corrected target identification picture;

s3-2: superposing the denoised picture through binarization and HSV red extraction to obtain a final binarization picture;

s3-3: and dividing the obtained binary image into single characters, wherein the image is a 0-1 matrix, 0 is a background 1 and is a character, the characters are divided according to the number of 0-1 in each column, the number of 1 and 0 in each column is respectively counted, the characters are divided according to the 0-1 sum transformation of each column, and the divided 5 characters are sequentially added into a list to ensure that the sequence is not disordered.

Still further, the step 4 includes the steps of:

s4-1: taking the cut single character as a data set for CNN character recognition model training, wherein the data set comprises 10 numbers and 24 capital letters of 34 classes, and the letters are not included in the letters O and I;

s4-2: and training the CNN network according to the prepared data set.

Still further, the process of step S5 is as follows: acquiring a pan head mark, cutting a mark single character and recognizing the character into a system, acquiring the pan head mark in an original image according to a trained yolov3 model, screening the length-width ratio of the cut mark, reserving an optimal mark picture, cutting the acquired pan head mark into each character, acquiring the sequence of the cut single character, transmitting the sequence of the cut single character into a trained CNN model according to the sequence of the pan head character, recognizing each character, and acquiring and outputting the complete pan head mark.

In the step S6, the environments such as Python3, tenswafw, OpenCV, and the like required by the raspberry group 4 are configured, and the integrated system is transplanted to the raspberry group and debugged on the raspberry group.

According to the invention, a set of embedded system capable of automatically identifying the pan head identification is designed according to the computer technology, so that the labor cost can be reduced, and the automatic production efficiency can be improved. The identification system is mainly based on a Yolov3 model of a Tensorflow deep learning framework to automatically cut out an identification target, and then trains a Convolutional Neural Network (CNN) for classification to identify.

The invention has the beneficial effects that: under the condition of ensuring the speed and the accuracy, yolov3 can adapt to the change of light rays to acquire pan head marks, which is obviously superior to the traditional image processing method, the classification of the CNN neural network is better than the traditional method in precision, and the whole set of system can improve the production efficiency and reduce the production cost.

Drawings

FIG. 1 is a system flow diagram of an embodiment of the present invention;

FIG. 2 is a flow chart of the structure of yolov3 according to an embodiment of the present invention;

fig. 3 is a flow chart of the CNN algorithm according to the embodiment of the present invention.

Detailed Description

The following detailed description of embodiments of the invention is provided in connection with the accompanying drawings.

Referring to fig. 1 to 3, a disc head identification recognition method based on yolov3 and CNN includes the following steps:

s1: pan head picture acquisition: in an actual production environment, data are collected through a camera, and the pan heads printed with red marks in different light rays in the actual environment are respectively collected;

s2: pan head picture processing: firstly, cutting out a complete pan head mark in an original image, and training the cutting of the pan head mark by using a yolov3 model;

s3: cutting out a single character picture according to the cut-out coiled hair identification picture;

s4: training the identification of the pan head identification character by using a CNN convolutional neural network;

s5: debugging the system: integrating the cutting of the coiled head mark, the cutting of characters and the recognition of the characters into a system;

s6: and (4) configuring the environment of the raspberry pi 4, transplanting the system, and finally debugging on the raspberry pi.

Further, in step S1, an actual system platform is built, the position of the camera is fixed, the camera is made to take a picture according to the trigger instruction that the pan head rolls, and the picture is stored in the JPG format.

Still further, the step S2 includes the steps of:

s2-1: firstly, making a data set for training, collecting pictures containing pan heads, taking partial pictures as a training set, manually marking a frame of a pan head identification through a manual marking software Labelimg, marking the label type as 'biaozhi', generating a corresponding xml file by each picture with a corresponding name, making the pictures and the xml file into a VOC data set format, and generating a train.txt, val.txt, test.txt and train.txt files for training and testing;

s2-2: in the yolov3 model, a feature extraction network used by yolov3 is DarkNet53, and the network is formed by overlapping residual error units. Taking the original coiled hair image as the input of the neural network, firstly outputting an N x N convolution feature map through the feature extraction network to generate N x N gird cells, and then outputting the category and the coordinate of the target through a prediction layer anchors. Bounding box coordinate (z)_x,z_y,z_w,z_h) The predictive calculation formula is as follows:

z_x＝σ(q_x)+c_x

z_y＝σ(q_y)+c_y

wherein q is_x,q_y,q_w,q_hCoordinates of the predicted center point and width and height of the prediction box for each Bounding box of yolov3, c_x,c_yIs the coordinate offset, p, of the grid cell grid where the center coordinate of the frame is located_w,p_hTo predict the width and height of the anterior anchor; firstly, determining the order in the grountruthMarking the Grid cell in which the central coordinate falls, predicting a target according to the Grid cell, wherein each Grid cell in yolov3 has three anchors, and calculating a predicted coordinate z_x,z_y,z_w,z_hWhen the loss is less, selecting three anchors with highest confidence level to calculate the loss of the group truth;

target class penalty L_cla(o, c) use is made of a binary cross-entropy loss, where o_ijE.g. {0,1}, which represents whether the target bounding box i is predicted to have a target or not,

representing sigmoid probability of the target in the network prediction target boundary box i, and the calculation formula is as follows:

finally, NMS processing is carried out on the predicted target, and finally the rectangular box with the highest target probability is reserved, and the vertex coordinates of the rectangular box are output;

s2-3: cutting out a pan head mark and an output vertex coordinate according to a trained model, screening a target picture through the approximate length-width ratio of the pan head mark, and correcting the screened picture through OpenCV perspective transformation, wherein a calculation formula of the perspective transformation is as follows:

where src () is the vertex coordinates of the picture before input correction, dst () is the vertex coordinates after output correction, M is a 3 × 3 transformation matrix, and the transformation matrix M is calculated by applying the getpropertransform () function in OpenCV.

Still further, the step 3 includes the steps of:

s3-1: firstly, carrying out noise reduction processing on a corrected target identification picture;

s3-2: superposing the denoised picture through binarization and HSV red extraction to obtain a final binarization picture;

s3-3: and dividing the obtained binary image into single characters, wherein the image is a 0-1 matrix, 0 is a background 1 and is a character, the characters are divided according to the number of 0-1 in each column, the number of 1 and 0 in each column is respectively counted, the characters are divided according to the 0-1 sum transformation of each column, and the divided 5 characters are sequentially added into a list to ensure that the sequence is not disordered.

Still further, the step 4 comprises the following steps:

s4-1: taking the cut single character as a data set for CNN character recognition model training, wherein the data set comprises 10 numbers and 24 capital English letters, and the letters are not included in the letters 'O' and 'I';

s4-2: and training the CNN network according to the prepared data set.

Still further, the process of step 5 is as follows: acquiring a pan head mark, cutting a mark single character and recognizing the character into a system, acquiring the pan head mark in an original image according to a trained yolov3 model, screening the length-width ratio of the cut mark, reserving an optimal mark picture, cutting the acquired pan head mark into each character, acquiring the sequence of the cut single character, transmitting the sequence of the cut single character into a trained CNN model according to the sequence of the pan head character, recognizing each character, and acquiring and outputting the complete pan head mark.

In the step 6, environments such as Python3, tensoflow, OpenCV and the like required by the raspberry pi 4 are configured, the integrated system is transplanted to the raspberry pi, and the raspberry pi is debugged.

As described above, in the embodiment of the invention, based on yolov3 target detection and the method for identifying the pan head identifier by using the neural network, the invention identifies the pan head identifier by training the yolov3 model and the CNN model, so that the production efficiency is greatly improved and the production cost is reduced under the condition of ensuring the speed and the precision.

The above-mentioned embodiments are only preferred embodiments of the present invention, which are merely illustrative and not restrictive, and the technical solutions and the patent concepts according to the present invention disclosed in the specification are equally replaced or modified by those skilled in the art within the scope of the present invention.