阅读说明：本技术 一种基于机器视觉的白砂糖袋自动计数装置及方法 (Machine vision-based white granulated sugar bag automatic counting device and method ) 是由 赵东哲 张雷 靳晔 张震 秦亚敏 邓菲 王勇 庞增拴 于 2020-05-14 设计创作，主要内容包括：本发明公开了一种基于机器视觉的白砂糖袋自动计数装置及方法。所述装置包括传送带架体、传送皮带、设备支架、工业计算机、线激光器固定头、工业相机固定头、线激光器、工业相机。线激光器、工业相机分别通过线激光器固定头、工业相机固定头安装于设备支架,用于激光轮廓生成和采集,工业计算机将数帧白砂糖袋切片化分割轮廓图像中的相对包罗面积,经综合化分析后计算出袋数,方法过程包括图像预处理、激光条纹中心线提取、中心线断点补偿、相对包罗面积计算和袋数综合计算。本发明实现了输送白砂糖袋过程的自动计数,有效解决了白砂糖连袋、叠袋等复杂姿态下糖袋计数困难且人力消耗较大的问题,有效提高了计袋的准确率,降低了企业人力成本。(The invention discloses a machine vision-based white granulated sugar bag automatic counting device and method. The device comprises a conveyor belt frame body, a conveyor belt, an equipment support, an industrial computer, a line laser fixing head, an industrial camera fixing head, a line laser and an industrial camera. The line laser and the industrial camera are respectively installed on the equipment support through a line laser fixing head and an industrial camera fixing head and used for generating and collecting laser outlines, the industrial computer slices a plurality of frames of white granulated sugar bags to segment relative envelope areas in outline images, and bag numbers are calculated after comprehensive analysis. The automatic counting device realizes automatic counting in the process of conveying the white granulated sugar bags, effectively solves the problems of difficult counting and large labor consumption of the white granulated sugar bags in complex postures such as bag connection and bag folding, effectively improves the bag counting accuracy and reduces the labor cost of enterprises.)

1. The utility model provides a white granulated sugar bag automatic counting device based on machine vision, the device includes conveyer belt support body (1), conveyer belt (2), equipment support (3), industrial computer (4), line laser fixed head (5), line laser fixed head two (6), industry camera fixed head (7), line laser (8), line laser two (9), industry camera (10).

2. The machine vision-based white granulated sugar bag automatic counting device of claim 1, characterized in that: the equipment support (3) is fixed on the side of the conveyor belt frame body (1).

3. The machine vision-based white granulated sugar bag automatic counting device of claim 1, characterized in that: line laser fixed head one (5), line laser fixed head two (6) with industry camera fixed head (7) are fixed be triangular distribution on equipment support (3), and are located conveyer (2) top.

Furthermore, the first line laser fixing head (5) and the second line laser fixing head (6) are symmetrically and vertically distributed at the positions of the two sides of the upper part of the conveying belt (2) in the conveying direction according to the central line, and the industrial camera fixing head (7) is positioned at the position above the lower part of the central line of the conveying belt (2) in the conveying direction.

4. The machine vision-based white granulated sugar bag automatic counting device of claim 3, characterized in that: the line laser I (8) and the line laser II (9) are respectively fixed on the line laser fixing head I (5) and the line laser fixing head II (6).

Further, line laser instrument one (8) with line laser instrument two (9) are partial to conveyer belt (2) conveying center line direction installation, the laser line part overlaps and is a word cover conveyer belt (2) width, and with conveyer belt (2) direction of transfer central line is perpendicular.

5. The machine vision-based white granulated sugar bag automatic counting device of claim 1, characterized in that: the industrial camera (10) is installed and fixed at an oblique lower angle through the industrial camera fixing head (7), the horizontal direction is opposite to the conveying direction of the conveying belt (2), and the visual field range covers the laser line outline appearing area.

6. A machine vision-based white granulated sugar bag automatic counting method comprises the following steps:

a. acquiring a laser outline image of the white granulated sugar bag in real time and carrying out image preprocessing;

b. extracting a laser outline center line of the white granulated sugar bag in the image;

c. breakpoint compensation of the laser outline center line of the white granulated sugar bag;

d. calculating the relative enveloping area of the laser contour center line of the white granulated sugar bag;

e. and calculating and displaying the number of the white granulated sugar bags.

7. The machine vision-based white granulated sugar bag automatic counting method according to claim 6, characterized in that: the image preprocessing in the step a comprises image correction, image graying, image filtering and image binarization.

8. The machine vision-based white granulated sugar bag automatic counting method according to claim 6, characterized in that: and b, adopting a Hessian matrix algorithm to realize the extraction of the laser stripe center line.

9. The machine vision-based white granulated sugar bag automatic counting method according to claim 6, characterized in that: and e, carrying out current bag number statistics by adopting single-frame and multi-frame relative inclusionand comprehensive analysis.

Technical Field

The invention belongs to the technical field of machine vision counting, and particularly relates to a white granulated sugar bag automatic counting device and method based on machine vision.

Background

White granulated sugar is an important additive in the food processing industry, and is an indispensable seasoning for home life. Most of domestic white granulated sugar is packaged by adopting 50Kg specification woven bags in production places and distributed nationwide. At the storage, transportation and processing points of grain depots, logistics centers, food processing plants and the like, the processes of loading and unloading the white granulated sugar by the conveyor belt relate to the statistical task of the quantity of a large number of sugar bags.

In patent [ CN201610813722.8], a counting and detecting method of a belt conveyer is proposed, in patent [ CN201710814559.1], a statistical method of the number of bags of quantitative bag type packaging materials is proposed, and in patent [ CN201410633496.6], a packaging bag counting system of a sugar refinery is proposed. However, the sensors in the above three patents all adopt correlation photoelectric switches, and the shape of the recognized target is relatively fixed, and the posture of transmission is ideal. In the process of conveying sugar bags, because the sugar bags are soft and have different stacking postures, the counting method and other conventional counting sensors in the patent are difficult to be suitable for the conveying conditions of non-ideal stacking postures such as bag connection and bag stacking of the sugar bags. At present, in actual bag counting, a method combining manual counting and weighbridge weighing is mainly adopted, but the weighbridge has a large weighing error on the whole vehicle, so that manual counting is mainly used as the standard, a large amount of manpower is consumed in repeated counting work with low value, and the consumption of manpower and time cost aggravates the improvement of the logistics cost of the white granulated sugar. Therefore, the invention provides a non-contact unmanned automatic counting device and method under the complex posture of the sugar bag, which has great significance for solving the practical problems.

Disclosure of Invention

The invention aims to provide a machine vision-based white granulated sugar bag automatic counting device and a machine vision-based white granulated sugar bag automatic counting method, which can realize a non-contact and non-human-participation real-time accurate counting function in a white granulated sugar conveying process and solve the problem of overlarge labor cost consumption in the existing counting process.

The invention relates to a machine vision-based white granulated sugar bag automatic counting device which comprises a conveyor belt frame body, a conveyor belt, an equipment support, an industrial computer, a first line laser fixing head, a second line laser fixing head, an industrial camera fixing head, a first line laser, a second line laser and an industrial camera.

In some embodiments, the equipment rack is secured to the side of the conveyor belt frame.

In some embodiments, the first line laser fixing head, the second line laser fixing head and the industrial camera fixing head are fixed on the equipment bracket in a triangular distribution and are positioned above the conveying belt.

Furthermore, the first line laser fixing head and the second line laser fixing head are symmetrically and vertically distributed at the two sides of the upper part of the first line laser fixing head and the second line laser fixing head according to the central line of the conveying direction of the conveying belt, and the industrial camera fixing head is positioned at the position above the lower stream of the central line of the conveying direction of the conveying belt.

In some embodiments, the first line laser and the second line laser are respectively fixed on the first line laser fixing head and the second line laser fixing head.

Furthermore, the first line laser and the second line laser are installed in a direction deviating from the transmission center line of the transmission belt, and the laser lines are partially overlapped to form a straight line to cover the width range of the transmission belt and are vertical to the center line of the transmission direction of the transmission belt.

In some embodiments, the industrial camera is fixed by the industrial camera fixing head in an oblique downward angle, the horizontal direction is opposite to the conveying direction of the conveying belt, the visual field range covers the laser line outline appearance area, and the industrial camera and the industrial computer are connected through six types of gigabit lines.

According to the white granulated sugar bag automatic counting method based on machine vision, laser line outline images under the conditions of no white granulated sugar bag and white granulated sugar bag transmission are respectively collected at high speed through the industrial camera, so that the measured white granulated sugar bag is sliced and segmented in a program, and after calculation of the relative inclusional total area and comparison analysis of the relative inclusional unit area are completed, real-time counting of the quantity in the white granulated sugar bag transmission process is realized.

In some embodiments, a machine vision-based white granulated sugar bag automatic counting method comprises the following steps:

a. acquiring a laser outline image of the white granulated sugar bag in real time and carrying out image preprocessing;

b. extracting a laser outline center line of the white granulated sugar bag in the image;

c. breakpoint compensation of the laser outline center line of the white granulated sugar bag;

d. calculating the relative enveloping area of the laser contour center line of the white granulated sugar bag;

e. and calculating and displaying the number of the white granulated sugar bags.

In certain embodiments, the image preprocessing in step a comprises image correction, image graying, image filtering and image binarization processes.

In certain embodiments, a Hessian matrix algorithm is used in step b to achieve laser fringe centerline extraction.

In some embodiments, the current bag number statistics is performed in step e by using single-frame and multi-frame relative inclusionary area analysis

The beneficial technical effects of the invention comprise:

(1) by adopting a machine vision detection mode combining line laser and an industrial camera, the bag number statistics of non-ideal complex conveying postures such as white granulated sugar bag connection, bag folding and the like can be realized, the accuracy rate is extremely high, and the damage to the white granulated sugar bag is avoided by non-contact measurement;

(2) the device and the method have the advantages of low hardware cost, convenient deployment, low requirement on corresponding field application environment, and high adaptability and portability.

Drawings

FIG. 1 is a schematic diagram of the overall structure of the white granulated sugar bag automatic counting device based on machine vision;

fig. 2 is a schematic flow chart of the machine vision-based white granulated sugar bag automatic counting method of the invention;

FIG. 3 is a schematic diagram of the breakpoint compensation logic algorithm of the present invention.

Detailed Description

The invention will now be described by way of example with reference to a 50Kg white sugar bag, which will be described in detail with reference to the accompanying figures 1-3.

Fig. 1 shows a schematic diagram of the overall structure of the present invention, which includes a conveyor belt frame 1, a conveyor belt 2, an equipment frame 3, an industrial computer 4, a first line laser fixing head 5, a second line laser fixing head 6, a industrial camera fixing head 7, a first line laser 8, a second line laser 9, an industrial camera 10, and a white granulated sugar bag 11 to be tested.

In this embodiment, the white granulated sugar conveyer belt is composed of a conveyer belt frame body 1 and a conveyer belt 2, a white granulated sugar bag 11 to be tested is placed on the upper surface of the conveyer belt 2 for conveying, the equipment support 3 is fixed at the side position of the conveyer belt frame body 1, the line laser fixing head I5, the line laser fixing head II 6 and the industrial camera fixing head 7 are respectively fixed at the top of the equipment support 3 and are symmetrically distributed in a triangular shape with the center line of the conveyer belt 2 in the conveying direction, the line laser I8, the line laser II 9 and the industrial camera 10 are respectively fixed in the corresponding fixing heads, the installation angle of the line laser satisfies that the laser lines are partially overlapped to cover the width of the conveyer, and is perpendicular with 2 direction of transfer central lines of conveyer, and industrial camera 10 slant angle of installation down satisfies the laser line profile and is in the image acquisition within range, and industrial camera 10 links to each other with the industrial computer 4 of equipment support 3 side installation through six types of giga net twines.

In the embodiment, the selection of the industrial camera 10 is comprehensively determined according to the conditions of the size, the measurement precision, the imaging speed and the like of the measured white granulated sugar bag, the weight of the applied measured white granulated sugar bag is 50Kg, the maximum length of full sugar is 900mm, the maximum width is 500mm, the width of the conveying belt is 1000mm, the measurement precision is 1mm, and the distance between the camera and the white granulated sugar bag is more than 1000 mm. Firstly, pixel estimation is carried out, the target surface of the camera is generally in a 4:3 rectangular shape, the short edge of the target surface is taken as a reference, the pixels of the short edge are greater than 1000/1 and equal to 1000, and a Haokawav CMOS camera MV-CA013-20GC with the target surface size of 1/2 ″ (6.4 × 4.8mm), the resolution of 1280 × 1024, the pixel size of 5 μm and the maximum frame rate of 90fps can be selected according to the estimated pixels. The focal length f is calculated by f Wh/W, wherein the target surface width W is 4.8mm, the spacing h is 1000mm, the acquisition width W is 1000mm, and the focal length f is 4.8mm through calculation, and a Haekangwei MVL-C0420-5MP lens with the focal length of 4mm can be selected.

The line laser is comprehensively determined according to conditions such as irradiation distance, area length, laser line quality and the like, according to comparison of various wavelengths and lenses, the 650nm wavelength Boville prism line laser can eliminate central hot spots and fading edge distribution phenomena of Gaussian beams, has excellent performances in the aspects of light forming linearity, stability, density uniformity and the like, can select a HW650AB100-16GD-WLD line laser, has a line emission angle of 60 degrees and a minimum uniform line width of 1.5 mm.

Fig. 2 is a schematic flow chart of the method of the present invention, which specifically includes image preprocessing, laser stripe centerline extraction, stripe centerline breakpoint compensation, outline relative envelope area calculation, and white granulated sugar bag counting and displaying. The method specifically comprises the following steps:

(1) the normal operation of the conveyor belt 2 and the state that the sugar bag does not start to be conveyed, the laser line forms a belt surface contour line on the conveyor belt 2, an image is collected by an industrial camera 10 and loaded into an industrial computer 4, the image is subjected to image correction, image graying, image filtering and image binarization, laser stripe center line extraction is realized by a Hessian algorithm, and the center line breakpoint compensation algorithm and the relative envelope area S are performed₀And (5) calculating to finish the initial acquisition process.

(2) The conveyor belt 2 runs normally to have no sugar bag and finishes the initial acquisition state, the initial acquisition process is carried out again in the next frame, and S is updated₀So as to eliminate the influence caused by the change of the conveyor belt or the equipment bracket.

(3) The conveying belt 2 runs normally and is in a sugar bag conveying state, the white sugar bag is conveyed from the line laser end to the camera direction, the laser line forms a stripe profile initial frame on the outer surface of the sugar bag, a stripe profile original image is subjected to image correction, image graying, image filtering and image binarization processing, then the central line extraction is carried out through a Hessian algorithm, and S is obtained through a central line breakpoint compensation algorithm and relative envelope area calculation₁。

(4) When S is₁-S₀When the threshold value is set to be more than or equal to S, the process judges that the sugar bag contour is effective, and the threshold value is set to be S₁-S₀Assignment pocket block assemblyAnd relatively covering the area S, and acquiring a next frame of image. Otherwise, judging that the outline of the sugar bag is invalid, and acquiring a next frame of image.

(5) When continuous effective sugar bag contour image acquisition is carried out, every frame S₁-S₀And overlapping the total relative envelope area S of the bag blocks, and acquiring a next frame of image.

(6) When the pocket leaves the initial frame of the laser line, i.e. S₁-S₀S and S>When 0, judging that the outline of the sugar bag is invalid, calculating the nearest integer quotient of the total relative covered area S' of the S bag and the single bag and assigning the number N of the sugar bags in the bag block₁。

(7) Updating the total bag number N to N₁And+ N, updating and displaying the screen of the industrial computer 4.

(8) And after the counting of the bag blocks is finished, assigning the value of S to be 0, judging whether the bag counting process is finished or not, if not, acquiring the next frame of image, and if not, stopping the process.

(9) Conveyor Belt 2 No sugar bag conveying Interval State, S₁-S₀And when S is less than S and S is equal to 0, directly judging whether the bag counting process is finished or not, if not, carrying out next frame image acquisition, otherwise, terminating the process.

1. Image correction

The image correction mainly adopts a geometric correction method to solve the geometric distortion phenomenon of a laser line outline image generated by the problem of the shooting angle of the industrial camera 10, the geometric distortion image correction is carried out according to a reference image, the reference image f (x, y) is set as a manual distortion-removed image, g (x ', y') represents the real-time acquisition of the geometric distortion image, and the conversion form isThe implementation sequentially calculates the correction coordinates of each pixel point of the distorted image according to the relation based on the known point coordinates according to the linear transformation with n being 1, and endows the pixel values with corresponding coordinates, namely finishing the geometric distortion correction of the laser line profile image.

2. Image graying

To reduce the effect of natural ambient light on the 650nm red laser line and reduce the amount of data for image analysis, weighting is selected in this implementationThe average algorithm is used for carrying out image graying treatment to the weight coefficient P of R, G, B component_r、P_g、P_bThe weight ratio is carried out to make each pixel value be 256 levels

3. Image filtering

In order to reduce interference generated by image noise and improve reliability and accuracy of laser line profile extraction, a Gaussian filter is adopted in the implementation, the profile characteristics of an original image can be kept while noise is removed, a linear smoothing filter of a template coefficient is determined according to a Gaussian function form, namely a final pixel value of each pixel point in an image is obtained by weighted averaging of a neighborhood of the pixel value and a pixel value of the pixel value, the weighted value is monotonically decreased along with the increase of the distance from a central point, and the value of a filter coefficient at (x, y) position in a two-dimensional Gaussian filter template is equal to the value of the filter coefficient at (x, y) positionWherein, σ is a Gaussian distribution parameter and determines the smoothing degree of the Gaussian function.

4. Image binarization

In order to further highlight the profile characteristics of the laser line, reduce the data volume of the image and improve the overall operation real-time performance of the process, the filtered gray-scale image is subjected to binarization processing, and a threshold value method is adopted to perform binarization processing on the gray-scale image through the gray value of each pixel point and a set threshold value T_bBy comparison, classifying 256-level gray scale image pixels into 0 and 255 values, the process can be represented asWherein I_i,jGraying the (I, j) th pixel of the image for M × N I_i,j' As the (i, j) th pixel of the binarized image, i ∈ [0, M-1]，j∈[0,N-1]Normal threshold value T_bThe selection method comprises a minimum error fraction method, a maximum inter-class variance method and a maximum entropy method, and the maximum inter-class variance method is selected in the implementation.

5. Extraction of laser stripe center line by Hessian algorithm

Laser line stripe center line lifterThe method comprises an edge contrast method, a geometric center method, a threshold value method, an extreme value method, a gray scale gravity center method, a Hessian matrix algorithm and the like, wherein the Hessian matrix algorithm is selected for implementation, namely, the normal direction of the stripes is determined firstly, then the position of the central line is solved by Taylor expansion in the normal direction, and the Hessian matrix expression isWhereing (x, y) is a two-dimensional Gaussian convolution template, I (x, y) is a matrix which takes the central point (x, y) of the image as the center and has the same size with the two-dimensional Gaussian convolution template, and the image (x, y)₀,y₀) The normal direction of the point is the direction of the maximum absolute value of the second-order directional derivative of the point, and the eigenvector (n) corresponding to the absolute value of the maximum characteristic of the Hessian matrix of the point_x,n_y) Given that the tangential direction is perpendicular to the normal direction, (x)₀,y₀) The gray distribution function in the normal direction of the point is expanded by second-order Taylor to obtain the available I (x)₀+tn_x,y₀+tn_y) Indicating that the characteristic value of the laser stripe is Gaussian distributed in the normal direction, and the higher the light intensity is, the closer the characteristic value is to the center of the light bar, so that I (x)₀+tn_x,y₀+tn_y) Is taken as the point where the first derivative of (x) is zero₀,y₀) Center pixel point (x) in the normal direction of the point₀+tn_x,y₀+tn_y) Can obtainAndthenAnd calculating in sequence to obtain the coordinates of each pixel point of the central line of the laser stripe.

6. Breakpoint compensation

For solving the problem of collection caused by complex postures such as folding and folding of sugar bagsThe laser contour line image has a breakpoint region problem, a centerline breakpoint compensation logic algorithm is adopted, fig. 3 is a local breakpoint region in an M × N image, and P1 and P2 are coordinates (X)_P1,Y_P1) And (X)_P2,Y_P2) The horizontal column difference is i and the vertical row difference is j. Let quotient step length A be rounded down for the quotient of i, j maximum and minimum, remainder step length B be i, j maximum and minimum remainder + A, and the expression isB ═ max (i, j) -a · (min (i, j) -1), the breakpoint compensation logic rule table is as follows:

7. relative inclusionary area calculation

The calculation of the relative covered area includes S₀And S₁Statistics of the area covered by the center line, S₀Earlier stage is S₁Therefore, S is used as follows₁Calculating, performing column-by-column and row-by-row analysis by using M × N image i equal to 0 and j equal to 0 pixel point, and when (i, j) is equal to 255, comparing S with S₁Is updated to S₁+ M-i continue the process; when (i, j) is 0, i! When the value is equal to M-1, updating i to i +1 and continuing the process; when (i, j) is 0, i is M-1, j! When the value is equal to N-1, i is updated to 0, j is updated to j +1, and the process is continued; when (i, j) is equal to 0, i is equal to M-1, j is equal to N-1, ending the relative inclusionary area calculation process of the frame and recording S₁。

The above embodiments are merely examples of the present invention which has been made in practice and good effects, and the present invention is not limited to the above specific forms, and any person skilled in the art of the present invention can make equivalent modifications by using the method or technique described in the present patent without departing from the scope of the main technical solution of the present invention, so any equivalent modifications without departing from the essence of the above technical solution still fall within the protection scope of the technical solution of the present invention.