阅读说明：本技术 一种基于双目视觉的中药小包装定位方法 (Binocular vision-based small traditional Chinese medicine package positioning method ) 是由 何新 马轩 陈琛 于 2021-07-09 设计创作，主要内容包括：本发明提供了一种基于双目视觉的中药小包装定位方法,该方法包括：双目相机采集中药小包装的图像；利用基于深度学习的YOLO-V4目标识别算法对左图像进行识别,得到中药小包装中心点的坐标；选择ORB算法进行特征点提取,并使用改进的RANSAC算法进行单应性矩阵计算,以获得中心点在右图像中的匹配坐标；根据双目视觉测距原理,通过计算出左、右两张图像的视差,计算深度信息,对中药小包装的中心点进行三维重建,得到中药小包装的三维坐标；将三维坐标传递给PLC,控制真空吸附泵完成药包的抓取。本发明基于深度学习的目标检测算法来实现对中药小包装在图像中位置的识别,具有定位精度高、检测速度快的特点。(The invention provides a binocular vision-based small traditional Chinese medicine package positioning method, which comprises the following steps: the binocular camera collects images of the small traditional Chinese medicine packages; identifying the left image by using a YOLO-V4 target identification algorithm based on deep learning to obtain the coordinates of the center point of the small traditional Chinese medicine package; selecting an ORB algorithm to extract feature points, and performing homography matrix calculation by using an improved RANSAC algorithm to obtain a matching coordinate of a central point in a right image; according to a binocular vision ranging principle, calculating the parallax of the left image and the right image, calculating depth information, and performing three-dimensional reconstruction on the central point of the small traditional Chinese medicine package to obtain three-dimensional coordinates of the small traditional Chinese medicine package; and transmitting the three-dimensional coordinates to the PLC, and controlling the vacuum adsorption pump to complete the grabbing of the medicine package. The invention realizes the recognition of the position of the small Chinese medicine package in the image based on the target detection algorithm of deep learning, and has the characteristics of high positioning precision and high detection speed.)

1. The utility model provides a traditional chinese medicine small bag positioning method based on binocular vision, is applied to the small traditional chinese medicine package positioning system based on binocular vision, the system is including the conveyer belt of the transport traditional chinese medicine small bag and set up the binocular camera on the conveyer belt and be used for snatching the vacuum adsorption pump of traditional chinese medicine small bag, binocular camera and vacuum adsorption pump all with PLC controller electric connection, its characterized in that includes following step:

s1, the binocular camera collects images of the small traditional Chinese medicine packages;

s2, identifying the left image by using a deep learning-based YOLO-V4 target identification algorithm to obtain the coordinates of the center point of the small traditional Chinese medicine package;

s3, stereo matching is carried out on the left image and the right image: selecting an ORB algorithm to extract feature points, and performing homography matrix calculation by using an improved RANSAC algorithm to obtain a matching coordinate of a central point in a right image;

s4, calculating depth information by calculating the parallax of the left image and the right image according to a binocular vision ranging principle, and performing three-dimensional reconstruction on the central point of the small traditional Chinese medicine package to obtain three-dimensional coordinates of the small traditional Chinese medicine package;

and S5, transmitting the three-dimensional coordinates to the PLC, and controlling the vacuum adsorption pump to complete the grabbing of the medicine package.

2. The binocular vision-based small traditional Chinese medicine package positioning method according to claim 1, wherein the step S1 further comprises binocular camera calibration, specifically comprising:

and calibrating the binocular camera by using a Zhang friend calibration method and a three-dimensional camera calibration module in Matlab to obtain a distortion matrix and internal and external parameters of the camera.

3. The binocular vision-based small traditional Chinese medicine package positioning method according to claim 2, wherein the calibrating of the binocular camera by using the stereoscopic camera calibration module in Matlab specifically comprises:

1) respectively shooting 33 checkerboards with different angles by using a left camera and a right camera to serve as calibration pictures, and respectively storing the calibration pictures in two folders;

2) respectively importing two folders in a Matlab toolbox, namely inputting the grid size of a calibration board;

3) automatically extracting angular points, starting calibration, and deleting pictures with large errors;

4) and repeating the steps until the ideal result is obtained.

4. The binocular vision-based small traditional Chinese medicine package positioning method of claim 1, wherein in the step S3, an ORB algorithm is selected for feature point extraction, and specifically comprises:

1) setting P as a characteristic point, and selecting a square neighborhood with the size of S x S by taking the P as a center to perform Gaussian smoothing;

2) generating N point pairs by using a random algorithm in the field, and defining a decision function T;

wherein p (x) and p (y) represent gray scale values of x and y;

3) selecting a pair (x, y) from the randomly generated point pairs, and comparing the gray value of the pair (x, y);

4) repeating the step 3) for N times to obtain an N-dimensional descriptor consisting of binary codes:

a sampling matrix S composed of coordinates to be used for calculation around the key point is expressed by the following formula (3):

the formula (3) gives the characteristic point direction parameter theta and the corresponding rotation matrix R_θ:

Correcting the matrix S:

S_θ＝R_θS (5)

the last rotated descriptor is represented as:

G_n(P,θ)＝f_N(P)|(x_i,y_i)∈S_θ (6)

5. the binocular vision-based small traditional Chinese medicine package positioning method of claim 1, wherein in the step S3, a modified RANSAC algorithm is used for homography matrix calculation, specifically comprising:

1) defining the matching quality E:

E＝γ*r (7)

in the formula, gamma is the ratio of the nearest neighbor distance and the secondary neighbor distance calculated by Lowe's algorithm; r is the distance of the feature vector extracted by the ORB algorithm;

2) sorting the characteristic point pairs in an ascending order according to the matching quality E to form a characteristic point pair matching set;

3) selecting four points with the minimum E value to initialize a homography matrix, and then sequentially selecting 4 points from the ordered characteristic point set to calculate a homography matrix H;

4) and if the number of the inner points obtained by calculation by using the homography matrix is greater than m, considering that H is the optimal homography matrix, wherein m is alpha N, alpha is the ratio of the number of the inner points to the total number of the characteristic points, and taking the value as 0.8.

6. The binocular vision-based positioning method for the small traditional Chinese medicine packages according to claim 1, wherein in the step S4, according to a binocular vision ranging principle, by calculating parallax between a left image and a right image, calculating depth information, and performing three-dimensional reconstruction on a central point of the small traditional Chinese medicine package, three-dimensional coordinates of the small traditional Chinese medicine package are obtained, and the method specifically comprises the following steps:

let the pixel coordinate P in the left image plane_left(X_left,Y_left) Matched pixel coordinate P in the right image plane_right(X_right,Y_right) And calculating the Z-axis coordinate of the point in the camera coordinate system, namely depth information according to the following formula:

wherein B is the distance between the line points in the projection of the left and right image planes, f represents the focal length of the camera, and parallel binocular vision is achievedThe coordinates of the P points in the model on the Y axis in the left and right image planes are equal, i.e. Y_left＝Y_rightY, parallax d X_left-X_right。

Technical Field

The invention relates to the technical field of small traditional Chinese medicine package identification, in particular to a small traditional Chinese medicine package positioning method based on binocular vision.

Background

Along with the continuous development of science and technology, the automation level in the fields of medicine production and processing is continuously improved, and at present, on the production line for transporting and grabbing small traditional Chinese medicine packages, the image recognition algorithm is adopted to replace manual recognition, so that the production efficiency is improved. However, the small traditional Chinese medicine packages are easy to stack and deform in the placing process due to the plastic materials, and the traditional image recognition algorithm is used for solving the problem that the recognition result is poor. The traditional algorithm obtains the category by extracting the features of the image, such as texture, color and the like, and then training the extracted features by using a classifier, and the convolutional neural network can extract the features by using convolution operation, so that the feature extraction and classification can be completed in one step, and the method is widely applied to the identification and classification of the image. The main task of identifying the small traditional Chinese medicine packages is to identify the medicine bags with complete packages on the surface layer for subsequent grabbing, and specific to identifiable and unidentifiable targets, no clear boundary exists between the identifiable targets and the unidentifiable targets, so that the medicine bags with fuzzy boundaries are difficult to identify. In addition, the traditional algorithm has poor real-time performance, and the position of the medicine package cannot be well correctly positioned from a large number of medicine packages.

Disclosure of Invention

The invention aims to provide a binocular vision-based small traditional Chinese medicine package positioning method, which is used for realizing the recognition of the position of a small traditional Chinese medicine package in an image based on a deep learning target detection algorithm and has the characteristics of high positioning precision and high detection speed.

In order to achieve the purpose, the invention provides the following scheme:

the utility model provides a traditional chinese medicine small bag positioning method based on binocular vision, is applied to the small traditional chinese medicine package positioning system based on binocular vision, the system is including the conveyer belt of the transportation traditional chinese medicine small bag and set up the binocular camera on the conveyer belt and be used for snatching the vacuum adsorption pump of traditional chinese medicine small bag, binocular camera and vacuum adsorption pump all with PLC controller electric connection, the method includes following step:

s1, the binocular camera collects images of the small traditional Chinese medicine packages;

s2, identifying the left image by using a deep learning-based YOLO-V4 target identification algorithm to obtain the coordinates of the center point of the small traditional Chinese medicine package;

s3, stereo matching is carried out on the left image and the right image: selecting an ORB algorithm to extract feature points, and performing homography matrix calculation by using an improved RANSAC algorithm to obtain a matching coordinate of a central point in a right image;

s4, calculating depth information by calculating the parallax of the left image and the right image according to a binocular vision ranging principle, and performing three-dimensional reconstruction on the central point of the small traditional Chinese medicine package to obtain three-dimensional coordinates of the small traditional Chinese medicine package;

and S5, transmitting the three-dimensional coordinates to the PLC, and controlling the vacuum adsorption pump to complete the grabbing of the medicine package.

Further, step S1 further includes calibrating a binocular camera, which specifically includes:

and calibrating the binocular camera by using a Zhang friend calibration method and a three-dimensional camera calibration module in Matlab to obtain a distortion matrix and internal and external parameters of the camera.

Further, the calibrating the binocular camera by using the stereo camera calibration module in Matlab specifically includes:

1) respectively shooting 33 checkerboards with different angles by using a left camera and a right camera to serve as calibration pictures, and respectively storing the calibration pictures in two folders;

2) respectively importing two folders in a Matlab toolbox, namely inputting the grid size of a calibration board;

3) automatically extracting angular points, starting calibration, and deleting pictures with large errors;

4) and repeating the steps until the ideal result is obtained.

Further, in step S3, selecting an ORB algorithm to perform feature point extraction specifically includes:

1) setting P as a characteristic point, and selecting a square neighborhood with the size of S x S by taking the P as a center to perform Gaussian smoothing;

2) generating N point pairs by using a random algorithm in the field, and defining a decision function T;

wherein p (x) and p (y) represent gray scale values of x and y;

3) selecting a pair (x, y) from the randomly generated point pairs, and comparing the gray value of the pair (x, y);

4) repeating the step 3) for N times to obtain an N-dimensional descriptor consisting of binary codes:

a sampling matrix S composed of coordinates to be used for calculation around the key point is expressed by the following formula (3):

the formula (3) gives the characteristic point direction parameter theta and the corresponding rotation matrix R_θ:

Correcting the matrix S:

S_θ＝R_θS (5)

the last rotated descriptor is represented as:

G_n(P,θ)＝f_N(P)|(x_i,y_i)∈S_θ (6)

further, in step S3, performing homography matrix calculation using an improved RANSAC algorithm specifically includes:

1) defining the matching quality E:

E＝γ*r

in the formula, gamma is the ratio of the nearest neighbor distance and the secondary neighbor distance calculated by Lowe's algorithm; r is the distance of the feature vector extracted by the ORB algorithm;

2) sorting the characteristic point pairs in an ascending order according to the matching quality E to form a characteristic point pair matching set;

3) selecting four points with the minimum E value to initialize a homography matrix, and then sequentially selecting 4 points from the ordered characteristic point set to calculate a homography matrix H;

4) and if the number of the inner points obtained by calculation by using the homography matrix is greater than m, considering that H is the optimal homography matrix, wherein m is alpha N, alpha is the ratio of the number of the inner points to the total number of the characteristic points, and taking the value as 0.8.

Further, in step S4, according to the binocular vision distance measurement principle, calculating parallax between the left and right images, calculating depth information, and performing three-dimensional reconstruction on the central point of the small package of the traditional Chinese medicine to obtain three-dimensional coordinates of the small package of the traditional Chinese medicine, specifically including:

let the pixel coordinate P in the left image plane_left(X_left,Y_left) Matched pixel coordinate P in the right image plane_right(X_right,Y_right) And calculating the Z-axis coordinate of the point in the camera coordinate system, namely depth information according to the following formula:

where B is the distance between the line points in the left and right image plane projection, f represents the camera focal length, and the coordinates of the P point on the Y axis in the left and right image planes are equal in the parallel binocular vision model, i.e., Y is_left＝Y_rightY, parallax d X_left-X_right。

According to the specific embodiment provided by the invention, the invention discloses the following technical effects: according to the binocular vision-based small traditional Chinese medicine package positioning method, after the images of the small traditional Chinese medicine packages are collected by using a binocular camera, the left images are identified by using a target identification algorithm based on deep learning, and the YOLO-V4 algorithm is selected as a core algorithm of image identification to obtain the coordinates of the central points of the small traditional Chinese medicine packages, so that the binocular vision-based small traditional Chinese medicine package positioning method has the characteristics of high detection precision and high detection speed; the two-dimensional coordinates of the center point of the small traditional Chinese medicine package output by the image recognition part are matched with the right image in the binocular vision system to obtain the coordinates of the point in space.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.

FIG. 1 is a schematic flow chart of a binocular vision-based small package positioning method for traditional Chinese medicines in the embodiment of the invention;

FIG. 2 shows the result of the Matlab camera calibration according to the embodiment of the present invention;

FIG. 3 is a diagram of an imaging model of a binocular camera according to an embodiment of the present invention;

fig. 4 is a flow chart of the RANSAC algorithm modified in the embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The invention aims to provide a binocular vision-based small traditional Chinese medicine package positioning method, which is used for realizing the recognition of the position of a small traditional Chinese medicine package in an image based on a deep learning target detection algorithm and has the characteristics of high positioning precision and high detection speed.

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.

As shown in fig. 1, the binocular vision-based positioning method for small traditional Chinese medicine packages provided by the embodiment of the present invention is applied to a binocular vision-based positioning system for small traditional Chinese medicine packages, the system includes a conveyor belt for transporting small traditional Chinese medicine packages, a binocular camera arranged on the conveyor belt, and a vacuum adsorption pump for grabbing small traditional Chinese medicine packages, the binocular camera and the vacuum adsorption pump are both electrically connected to a PLC controller, and the method includes the following steps:

s1, the binocular camera collects images of the small traditional Chinese medicine packages;

s2, identifying the left image by using a deep learning-based YOLO-V4 target identification algorithm to obtain the coordinates of the center point of the small traditional Chinese medicine package;

s3, stereo matching is carried out on the left image and the right image: selecting an ORB algorithm to extract feature points, and performing homography matrix calculation by using an improved RANSAC algorithm to obtain a matching coordinate of a central point in a right image;

s4, calculating depth information by calculating the parallax of the left image and the right image according to a binocular vision ranging principle, and performing three-dimensional reconstruction on the central point of the small traditional Chinese medicine package to obtain three-dimensional coordinates of the small traditional Chinese medicine package;

and S5, transmitting the three-dimensional coordinates to the PLC, and controlling the vacuum adsorption pump to complete the grabbing of the medicine package.

Wherein, the step S1 further includes binocular camera calibration: calibrating a binocular camera by using a Zhang friend calibration method and a three-dimensional camera calibration module in Matlab to obtain a distortion matrix and internal and external parameters of the camera; the method specifically comprises the following steps:

1) respectively shooting 33 checkerboards with different angles by using a left camera and a right camera to serve as calibration pictures, and respectively storing the calibration pictures in two folders;

2) respectively importing two folders in a Matlab toolbox, namely inputting the grid size of a calibration board;

3) automatically extracting angular points, starting calibration, and deleting pictures with large errors;

4) the steps are repeated until the desired results are obtained, as shown in figure 2.

In step S3, selecting the ORB algorithm to extract the feature points specifically includes:

1) setting P as a characteristic point, and selecting a square neighborhood with the size of S x S by taking the P as a center to perform Gaussian smoothing;

2) generating N point pairs by using a random algorithm in the field, and defining a decision function T;

wherein p (x) and p (y) represent gray scale values of x and y;

3) selecting a pair (x, y) from the randomly generated point pairs, and comparing the gray value of the pair (x, y);

4) repeating the step 3) for N times to obtain an N-dimensional descriptor consisting of binary codes:

a sampling matrix S composed of coordinates to be used for calculation around the key point is expressed by the following formula (3):

the formula (3) gives the characteristic point direction parameter theta and the corresponding rotation matrix R_θ:

Correcting the matrix S:

S_θ＝R_θS (5)

the last rotated descriptor is represented as:

G_n(P,θ)＝f_N(P)|(x_i,y_i)∈S_θ (6)

the orb (organized Fast and rotaed BRIEF) algorithm, which can be used to quickly create feature vectors for keypoints in images, detects the root cause of higher speed than SIFT and SURF, is the use of Fast keypoints and BRIEF descriptors. After the feature points are extracted, feature descriptors need to be established, the ORB algorithm adopts an rBRIEF descriptor to describe the extracted feature points, and rotation information is added to the descriptors by utilizing the main direction of the feature points, so that the rotation invariance can be obtained. The ORB algorithm describes the feature points by using the binary string, and compared with the 128-dimensional feature vector of the SURF algorithm, the ORB algorithm reduces the memory resource due to the binary string description, and obviously improves the calculation speed.

In terms of algorithm operation efficiency, the ORB algorithm is superior in efficiency, and the matching speed is faster than the SURF algorithm. Considering that the characteristic points detected by the ORB algorithm are more gathered near the small traditional Chinese medicine package and the real-time performance of the ORB algorithm is higher, the invention selects the ORB algorithm to carry out characteristic point matching.

As shown in fig. 4, in step S3, the performing the homography matrix calculation by using the improved RANSAC algorithm specifically includes:

1) defining the matching quality E:

E＝γ*r

in the formula, gamma is the ratio of the nearest neighbor distance and the secondary neighbor distance calculated by Lowe's algorithm; r is the distance of the feature vector extracted by the ORB algorithm;

2) sorting the characteristic point pairs in an ascending order according to the matching quality E to form a characteristic point pair matching set;

3) selecting four points with the minimum E value to initialize a homography matrix, and then sequentially selecting 4 points from the ordered characteristic point set to calculate a homography matrix H; the homography matrix H is defined as follows:

4) and if the number of the inner points obtained by calculation by using the homography matrix is greater than m, considering that H is the optimal homography matrix, wherein m is alpha N, alpha is the ratio of the number of the inner points to the total number of the characteristic points, and taking the value as 0.8.

Randomly selecting 4 points in the RANSAC algorithm to calculate the homography matrix, judging the points in the set to be inner points and outer points by using the homography matrix, and if the probability of randomly selecting the four points to be the inner points is more than 95%, constructing a correct homography matrix, otherwise, continuously and iteratively calculating the homography matrix. The increasing of the iteration times can influence the calculation efficiency of the RANSAC algorithm, so the invention uses the PROSAC idea for the improved optimization of the RANSAC algorithm.

The PROSAC algorithm is used for calculating the homography matrix by sequencing the matching quality and using the matching points with high matching quality. Therefore, the calculated homography matrix can calculate the inner point and the outer point of the algorithm more quickly, and the calculation speed of the algorithm is effectively accelerated.

When performing feature matching, the Lowe's algorithm screens mismatching points by calculating the ratio gamma of the nearest neighbor distance to the next nearest neighbor distance, and if gamma is less than T (T is a set threshold), the mismatching points are considered as correct matching points. When the Lowe's algorithm is used, the ratio r of the two is expected to be small, and when the matching quality measurement is carried out, the distance r of the feature vector extracted by the ORB algorithm is considered to be very small if the two feature points are matched, so that the matching quality E is defined by combining the two as the matching quality basis.

As shown in fig. 3, the binocular stereo vision is mainly divided into parallel binocular vision and non-parallel binocular vision, and in this mode, the imaging planes of the two cameras are parallel, that is, the left and right cameras are in the same plane, the embodiment of the present invention selects the parallel binocular vision as a research object. One point P (X) in space_C,Y_C,Z_C) And its projected points on the left and right imaging planes form a triangle, the model of which is shown in fig. 2. If the pixel coordinates of the point in the left and right imaging images are known, the three-dimensional coordinates of the point P in the world coordinate system can be calculated based on the principle of triangle geometry.

The improved RANSAC algorithm is tested, the matching effect of the improved RANSAC algorithm on the characteristic points is relatively small, the iteration times of the improved RANSAC algorithm are obviously reduced in the running time, the running time is less, and the running result is shown in a table 1.

TABLE 1 RANSAC Algorithm before and after improvement

The invention tracks the calculation homography matrix and counts the iteration times and the running time before and after the improvement of the step. Statistical results show that the improved RANSAC algorithm can basically obtain an accurate homography matrix in one-time calculation, is faster than the original algorithm in the running time, and meets the real-time requirement of the method.

In step S4, according to the binocular vision distance measurement principle, calculating parallax between the left and right images, calculating depth information, and performing three-dimensional reconstruction on the central point of the small package of the traditional Chinese medicine to obtain three-dimensional coordinates of the small package of the traditional Chinese medicine, specifically including:

let the pixel coordinate P in the left image plane_left(X_left,Y_left) Matched pixel coordinate P in the right image plane_right(X_right,Y_right) And calculating the Z-axis coordinate of the point in the camera coordinate system, namely depth information according to the following formula:

where B is the distance between the line points in the left and right image plane projection, f represents the camera focal length, and the coordinates of the P point on the Y axis in the left and right image planes are equal in the parallel binocular vision model, i.e., Y is_left＝Y_rightY, parallax d X_left-X_right。

According to the parallel binocular vision principle, if a point corresponding to a picture acquired by a right camera is found for a certain point in the picture acquired by a left camera, the three-dimensional coordinate of the point in a camera coordinate system can be calculated, so that the binocular vision model is used as the basic principle of the positioning method.

In addition, the YOLO-V4 algorithm adopted by the invention is an algorithm with excellent comprehensive performance in the field of target detection so far, not only is the detection precision high, but also the detection speed is higher than other algorithms, the real-time requirement of the invention can be ensured, and the YOLO-V4 algorithm has excellent performance under the conditions of small object detection, target distribution and target distribution concentration, and meets the requirement of the invention on multi-scale and medicine bag detection, so that the YOLO-V4 is selected as a core algorithm for identifying small traditional Chinese medicine packages.

The Anchor box of the YOLO-V4 is obtained by clustering on a COCO data set, the sample of the embodiment of the invention is a small package of traditional Chinese medicine in a pharmacy, and the sample difference is small, so the Anchor box obtained by the YOLO-V4 is not suitable for the invention. Aiming at the problem, the embodiment of the invention uses a K-means + + algorithm to re-cluster the medicine bag samples to obtain the Anchor box suitable for the application scene of the invention. The invention adopts K-means + + to recalculate the width and height of the Anchorbox, and the specific steps are as follows:

step 1: randomly selecting a sample from the normalized label samples GT as a first clustering center;

step 2: for the location boxes in the remaining labels in the label sample GT, calculate its IoU distance d (j) from the existing cluster center;

step 3: selecting IoU a coordinate frame with a larger distance d (j) as a new clustering center;

step 4: repeating steps 2 and Step3 until K cluster centers are determined;

step 5: traversing the other sample labels GT except the K clustering centers, and dividing the coordinate frame into the class to which the clustering center with the minimum clustering distance d belongs;

step 6: calculating the width-height mean value of each type of coordinate frame to serve as a new clustering center;

step 7: and repeating the steps of Step5 and Step 6. The repartitioning is stopped until the new cluster center is equal to the original cluster center or the distance between the new cluster center and the original cluster center is less than a threshold value.

The medicine package identification method aims at medicine package identification, and mainly aims at identifying medicine packages which are on the surface layer and are packaged completely so as to be convenient for follow-up grabbing, and aiming at identifiable and unidentifiable targets, no clear boundary exists between the identifiable targets and the unidentifiable targets, so that pictures with fuzzy boundaries are difficult to identify. In the data set samples manufactured by the method, the number of the labels which can not be grabbed is large, and when the data set is manufactured, the samples which meet the grabbing conditions but are slightly shielded are all set as the labels which can not be grabbed, so that the YOLO-V4 algorithm can accurately identify the samples which can not be grabbed, and the identification effect of the samples which can be grabbed is poor. In addition, due to the fact that the medicine packages are stacked, the number of the medicine packages which cannot be grabbed in an actual scene is larger than that of the medicine packages which can be grabbed, and the classes of the training samples are not balanced. Aiming at the problem, the Loss function of YOLO-V4 is optimized based on Focal local, and the optimized Loss function is expressed as follows:

the alpha factor enables training to balance the problem that the proportion of the grippable samples is not uniform with that of the non-grippable samples. The addition of a factor γ (γ > 0) can make training focus on these more difficult to correctly identify packages, γ adjusting the rate at which the weight of simple samples decreases, which is a cross-loss entropy function when γ is 0, and the influence of the adjustment factor increases when γ increases.

The embodiment of the invention optimizes and realizes the YOLO-V4 algorithm from two aspects: firstly, the Anchor box of the self-defined data set is reset by using a k-means + + algorithm, so that the loss of training is reduced more quickly, and the accuracy is improved; secondly, a loss function of a YOLO-V4 algorithm is improved, the problems of an original model on a user-defined data set are analyzed, and the loss function of the YOLO-V4 is optimized by using Focal local, so that the classified loss is reduced, and the classification of the medicine bags which can be grabbed and cannot be grabbed is more accurate.

According to the binocular vision-based small traditional Chinese medicine package positioning method, after the images of the small traditional Chinese medicine packages are collected by using a binocular camera, the left images are identified by using a target identification algorithm based on deep learning, and the YOLO-V4 algorithm is selected as a core algorithm of image identification to obtain the coordinates of the central points of the small traditional Chinese medicine packages, so that the binocular vision-based small traditional Chinese medicine package positioning method has the characteristics of high detection precision and high detection speed; the two-dimensional coordinates of the center point of the small traditional Chinese medicine package output by the image recognition part are matched with the right image in the binocular vision system to obtain the coordinates of the point in space.

The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.