阅读说明：本技术 一种基于视频分析的袋装物料出入库计量方法 (Video analysis-based method for metering bagged materials into and out of warehouse ) 是由 张耿城 梁尔祝 张日强 肖成勇 田思雨 耿威 于 2021-01-06 设计创作，主要内容包括：本发明公开一种基于视频分析的袋装物料出入库计量方法,步骤为：通过视频传感设备采集视频图像数据；选取视频图像数据中的感兴趣区域作为分析对象并进行预处理；对预处理的数据进行训练,构建袋装物料图像实时分割模型；利用袋装物料图像实时分割模型构建物料实时分割体系,计算每一个分割结果的面积大小以确定跟踪物料区域；计算跟踪物料区域的中心点；绘制跟踪物料区域中心坐标的曲线；计算曲线的斜率和收敛范围并设置阈值,进行物料统计；构建循环判定体系,对于不符合条件的情况进行滤除,实现袋装物料的连续计量。本发明可从系统中看到当前仓库中物料的情况,形成一个袋装物料出入库计量系统,方便后续调度,使得工厂运行实现自动化。(The invention discloses a video analysis-based method for metering bagged materials in and out of a warehouse, which comprises the following steps: collecting video image data through video sensing equipment; selecting an interested area in video image data as an analysis object and carrying out pretreatment; training the preprocessed data, and constructing a bagged material image real-time segmentation model; constructing a material real-time segmentation system by utilizing a bagged material image real-time segmentation model, and calculating the area of each segmentation result to determine a material tracking region; calculating the central point of the tracked material area; drawing a curve for tracking the center coordinates of the material area; calculating the slope and the convergence range of the curve, setting a threshold value, and carrying out material statistics; and constructing a circulation judgment system, and filtering the condition that the condition is not met to realize the continuous metering of the bagged materials. The invention can see the current material condition in the warehouse from the system to form a bagged material warehouse-in and warehouse-out metering system, thereby facilitating subsequent scheduling and realizing automation of factory operation.)

1. A video analysis-based method for metering bagged materials in and out of a warehouse is characterized by comprising the following steps:

1) collecting video image data through video sensing equipment;

2) selecting an interested area in video image data as an analysis object and carrying out pretreatment;

3) training the preprocessed data, and constructing a bagged material image real-time segmentation model;

4) constructing a material real-time segmentation system by utilizing a bagged material image real-time segmentation model, and calculating the area of each segmentation result to determine a material tracking region;

5) calculating a central point of the tracked material area, wherein the coordinate of the central point is a required tracking target;

6) drawing a curve for tracking the center coordinates of the material area;

7) calculating the slope and the convergence range of the curve, setting a threshold value, and carrying out material statistics;

8) and constructing a circulation judgment system, and filtering the condition that the condition is not met to realize the continuous metering of the bagged materials.

2. The video analysis-based bagged material warehousing/warehousing metering method of claim 1, wherein: in the step 2), the region of interest is selected by selecting a region from the video data as an analysis object, and the selection principle is to remove the region with light leakage in the video and remove the irrelevant region as much as possible; the video preprocessing operation is to convert the image into an HSV space, balance the overall brightness of the image by using histogram equalization, preprocess video data and facilitate subsequent work.

3. The video analysis-based bagged material warehousing/warehousing metering method of claim 1, wherein: training the preprocessed data in the step 3) comprises the following steps: training the marked data by using a VGG16-U-NET deep learning model, and segmenting the outline of the bagged material; the materials are marked in white, and other irrelevant areas are marked in black, so that video data formed by a section of binary image is generated, and noise points appearing in the data are processed by adopting opening and closing operation.

4. The video analysis-based bagged material warehousing/warehousing metering method of claim 1, wherein: calculating the area of each segmentation result in the step 4) to determine a material tracking area as follows: and calculating the number of pixels of each segmentation result as the area size by taking the pixel as a unit, judging a tracked target area through the calculated area, and sequencing the areas of all the segmentation results in the image in a descending order, thereby selecting the area with the largest area as a tracked material area.

5. The video analysis-based bagged material warehousing/warehousing metering method of claim 1, wherein: calculating the central point of the material tracking area in the step 5) as follows: for each frame of data in the video, calculating coordinates of the center point of the bagged materials, and screening the coordinates; and discarding the point with the coordinate change smaller than the specified threshold value within a certain time to ensure that the subsequent steps are smoothly carried out.

6. The video analysis-based bagged material warehousing/warehousing metering method of claim 5, wherein: step 5) also comprises the following steps: and discarding the central points by a slope judging method aiming at the condition that the curve is oscillated due to the fact that the warehousing angle of the material transporting forklift needs to be adjusted for multiple times.

7. The video analysis-based bagged material warehousing/warehousing metering method of claim 1, wherein: step 6) drawing a curve for tracking the center coordinates of the material area as follows: and establishing a coordinate system by using the original coordinate axis of the image, drawing two curves by using the coordinates of all points meeting the conditions, and using the variation trend of the x coordinate as one of the bases for judging the bagged materials entering and exiting the warehouse.

8. The video analysis-based bagged material warehousing/warehousing metering method of claim 1, wherein: step 7) calculating the slope and the convergence range of the curve and setting a threshold value as follows: fitting the curve by using a least square method, solving the average slope of the curve, and discarding the coordinate points if the material does not move or the moving amplitude is small at the point with the slope smaller than the threshold value; and judging the initial range and the final convergence range of the coordinates of the acquired points, setting a specified threshold value, and finally determining whether the material enters or exits the warehouse or not so as to count the material.

9. The video analysis-based bagged material warehousing/warehousing metering method of claim 1, wherein: and 8) judging once when the whole video needs to be circularly judged, emptying the stored data, judging when the stored data is larger than N and continuous values, and filtering the condition which does not meet the conditions to realize the metering of the bagged materials.

Technical Field

The invention relates to a material warehouse-in and warehouse-out metering technology, in particular to a video analysis-based method for metering bagged materials in warehouse-in and warehouse-out.

Background

Industrial production requires many chemical products, such as ammonium nitrate, which are flammable and explosive, are dangerous and endanger workers, and thus all fall under the strict national regulations. It becomes important for plant enterprises to be able to meter these chemicals precisely. Because chemical articles have the characteristics of flammability, explosiveness and the like, the scheme based on RFID and forklift weighing has strict explosion-proof requirements and is higher in cost.

Materials such as ammonium nitrate belong to dangerous goods regulated by the state, and the use condition needs to be strictly controlled. Because the doors of the warehouse are more, the materials are frequently fed and discharged, and the mode of manually counting bags at present is easy to make mistakes, so that the dangerous chemicals are inaccurate in inventory and easy to run off, thereby causing a series of problems.

Disclosure of Invention

Aiming at the defects in the prior art, the invention aims to provide a video analysis-based method for metering the material in and out of a bag, which is convenient for a supervisor to manage.

In order to solve the technical problems, the invention adopts the technical scheme that:

the invention provides a video analysis-based metering method for warehousing and warehousing of bagged materials, which comprises the following steps:

1) collecting video image data through video sensing equipment;

2) selecting an interested area in video image data as an analysis object and carrying out pretreatment;

3) training the preprocessed data, and constructing a bagged material image real-time segmentation model;

4) constructing a material real-time segmentation system by utilizing a bagged material image real-time segmentation model, and calculating the area of each segmentation result to determine a material tracking region;

5) calculating a central point of the tracked material area, wherein the coordinate of the central point is a required tracking target;

6) drawing a curve for tracking the center coordinates of the material area;

7) calculating the slope and the convergence range of the curve, setting a threshold value, and carrying out material statistics;

8) and a circulation judgment system is constructed, and reasonable filtering is carried out on the condition that the condition is not met, so that the continuous metering of the bagged materials is realized.

In the step 2), the region of interest is selected by selecting a region from the video data as an analysis object, and the selection principle is to remove the region with light leakage in the video and remove the irrelevant region as much as possible; the video preprocessing operation is to convert the image into an HSV space, balance the overall brightness of the image by using histogram equalization, preprocess video data and facilitate subsequent work.

Training the preprocessed data in the step 3) comprises the following steps: training the marked data by using a VGG16-U-NET deep learning model, and segmenting the outline of the bagged material; the materials are marked in white, and other irrelevant areas are marked in black, so that video data formed by a section of binary image is generated, and noise points appearing in the data are processed by adopting opening and closing operation.

Calculating the area of each segmentation result in the step 4) to determine a material tracking area as follows: and calculating the number of pixels of each segmentation result as the area size by taking the pixel as a unit, judging a tracked target area through the calculated area, and sequencing the areas of all the segmentation results in the image in a descending order, thereby selecting the area with the largest area as a tracked material area.

Calculating the central point of the material tracking area in the step 5) as follows: for each frame of data in the video, calculating coordinates of the center point of the bagged materials, and screening the coordinates; discarding the point with the coordinate change smaller than a specified threshold value within a certain time to ensure that the subsequent steps are smoothly carried out;

step 5) also comprises the following steps: and discarding the central points by a slope judging method aiming at the condition that the curve is oscillated due to the fact that the warehousing angle of the material transporting forklift needs to be adjusted for multiple times.

Step 6) drawing a curve for tracking the center coordinates of the material area as follows: and establishing a coordinate system by using the original coordinate axis of the image, drawing two curves by using the coordinates of all points meeting the conditions, and using the variation trend of the x coordinate as one of the bases for judging the bagged materials entering and exiting the warehouse.

Step 7) calculating the slope and the convergence range of the curve and setting a threshold value as follows: fitting the curve by using a least square method, solving the average slope of the curve, and discarding the coordinate points if the material does not move or the moving amplitude is small at the point with the slope smaller than the threshold value; and judging the initial range and the final convergence range of the coordinates of the acquired points, setting a specified threshold value, and finally determining whether the material enters or exits the warehouse or not so as to count the material.

And 8) judging once when the whole video needs to be circularly judged, emptying the stored data, judging when the stored data is larger than N and continuous values, and filtering the condition which does not meet the conditions to realize the metering of the bagged materials.

The invention has the following beneficial effects and advantages:

1. the video analysis-based bagged material warehouse-in and warehouse-out metering method saves labor cost, frees supervisory personnel, can conveniently see the current material condition in a warehouse from the system, forms a bagged material warehouse-in and warehouse-out metering system, facilitates subsequent factory scheduling, and further realizes automation of factory production.

2. The method can simultaneously monitor a plurality of doors of the storehouse, thereby helping a factory to better measure the bagged materials and facilitating subsequent dispatching management.

Drawings

FIG. 1 is a flow chart of a video analysis-based metering method for the warehousing and delivery of bagged materials;

FIG. 2A is a block diagram of a bagged material metering system employed in the method of the present invention;

FIG. 2B is a block diagram of a data transmission module shown in FIG. 2A;

fig. 3 is a diagram of a structure of a partitioned network model in the method of the present invention.

Detailed Description

The invention is further elucidated with reference to the accompanying drawings.

The invention provides a video analysis-based metering method for warehousing and warehousing of bagged materials, which comprises the following steps:

1) collecting video image data through video sensing equipment;

2) selecting an interested area in video image data as an analysis object and carrying out pretreatment;

3) training the preprocessed data, and constructing a bagged material image real-time segmentation model;

4) constructing a material real-time segmentation system by utilizing a bagged material image real-time segmentation model, and calculating the area of each segmentation result to determine a material tracking region;

5) calculating a central point of the tracked material area, wherein the coordinate of the central point is a required tracking target;

6) drawing a curve for tracking the center coordinates of the material area;

7) calculating the slope and the convergence range of the curve, setting a threshold value, and carrying out material statistics;

8) and a circulation judgment system is constructed, and reasonable filtering is carried out on the condition that the condition is not met, so that the continuous metering of the bagged materials is realized.

As shown in fig. 2A-2B, the method of the present invention is implemented in a video analysis-based metering system for warehousing and warehousing of bagged materials, and the system includes:

the video data acquisition module is used for acquiring video data of the warehouse door; in the embodiment, a DS-2CD4024F-SDI network camera of Haikangwei is used as the acquisition equipment;

the data transmission module transmits the acquired video data to a local server through a router and other related hardware; in this embodiment, a router with a network port for supplying power is used as a transit terminal, and the server device acquires data through the IP address of the camera.

The data preprocessing module is used for receiving the data transmitted by the router and preprocessing each frame of video data by using an image processing method; in this embodiment, an OpenCV-based graphics processing algorithm is adopted, and a threshold is set to perform histogram equalization on an image, so as to perform preprocessing on data.

The material metering module acquires a segmentation graph through a deep learning segmentation method, calculates the center point of the graph, and draws the analysis of a curve through points obtained by each frame, so that the types of the curve are divided into 3 types: the 3 curves are used for judging whether the materials pass through the warehouse or not and judging the flowing direction of the materials so as to help a factory to monitor the flowing of the materials; in the embodiment, a U-Net segmentation model is adopted to segment the images of the materials, so that the environment background is filtered out, and the tracking of the materials is realized;

the material metering display module displays the current material inlet and outlet conditions in real time, displays the inlet and outlet number on a monitoring interface, automatically fills form data, records the material inlet and outlet time, and is convenient for an enterprise supervisor to check; in the embodiment, an OpenCV interface design is adopted, and the number of the current accesses is displayed in the upper left corner of a monitoring picture in real time;

and the data storage module uses database counting and is used for storing the material data which are put in and out of the warehouse. In the embodiment, the MySql database is adopted, the in-out time and the in-out number of the materials are stored in the database which is built in advance, sharing is realized through the local area network, and calling of the master monitoring station is facilitated.

In the step 1), video image data are collected through video sensing equipment, firstly, a high-definition camera is installed above a warehouse door, the camera is opposite to the warehouse door, and the video data are uploaded to a local server for storage in real time through a router;

in the step 2), the region of interest is selected by selecting a region from the video data as an analysis object, and the selection principle is to remove the region with light leakage in the video and remove the irrelevant region as much as possible; the video preprocessing operation is to convert the image into an HSV space, balance the overall brightness of the image by using histogram equalization, preprocess video data and facilitate subsequent work.

The training process of the preprocessed data in the step 3) is as follows: training the marked data by using a VGG16-U-NET deep learning model, and segmenting the outline of the bagged material; in the embodiment, materials are marked in white, and other irrelevant areas are marked in black, so that video data formed by a section of binary image is generated, and noise points appearing in the data are processed by adopting opening and closing operation; the structure of the split network model is shown in fig. 3.

In the step 4), an image processing method is used, the area of each segmentation result is calculated by taking a pixel as a unit, namely the number of the included pixels is determined, the tracked material area is determined according to the calculated area, and the areas of all the segmentation results in the image are sorted in a descending order, so that the largest area is selected as a tracking target;

determining the center of the area by calculating the moment of the graph in the step 5), wherein the coordinate of the central point is the area of the material to be tracked in the design, solving the coordinate of the central point of the bagged material for each frame of data in the video, and screening the coordinates; discarding points with coordinate changes smaller than a certain degree within a certain time; screening points with obvious outliers, and properly abandoning the points to ensure that the subsequent steps are smoothly carried out; in addition, the forklift may need to adjust the warehousing angle for many times to cause the oscillation of the curve, so the points also need to be discarded by a method for judging the slope in the algorithm;

step 6) drawing a curve for tracking the center coordinates of the material area as follows: establishing a coordinate system by using the original coordinate axis of the image, and drawing two curves by using the coordinates of all the central points meeting the conditions; in the embodiment, the variation trend of the x coordinate is used as one of the bases for judging the bagged materials entering and exiting the warehouse.

Step 7) calculating the slope and the convergence range of the curve and setting a threshold value as follows: fitting the curve by using a least square method, solving the average slope of the curve, and discarding the coordinate points if the material does not move or the moving amplitude is small at the point with the slope smaller than the threshold value; and judging the initial range and the final convergence range of the coordinates of the central point, setting a proper threshold value, and finally determining whether the material enters or exits the warehouse or not so as to count the material.

In the step 8), for the situation that the whole video needs to be circularly judged, judging is carried out once, stored data are emptied at the same time, and when the stored data are set to be larger than N (in the embodiment, N = 180) and continuous values, judging is carried out again, and reasonable filtering is carried out on the situation that the conditions are not met, so that the metering of bagged materials is realized.

In this embodiment, for example, bagged ammonium nitrate stored in a blasting warehouse is taken, four products in one batch are arranged in 4 layers, the four layers are placed on a prepared tray, the tray is dragged into the warehouse by a forklift, when the tray enters a camera view at a warehouse door, the invention performs track analysis on the material, when the slope and the convergence range of the track are within a specified range, and the material enters a gate, the warehousing number of the warehouse is increased by 1 or decreased by 1, the monitoring picture is displayed at the upper left corner, and the monitoring picture is stored in a pre-established database.

The invention adopts the computer vision technology, judges whether the warehouse-in and warehouse-out occur or not by dividing and tracking the bagged materials and combining the motion trail of the materials, namely the slope and the convergence range of the motion curve of the materials, and simultaneously calculates the times of warehouse-in and warehouse-out of the materials, thereby accurately measuring the stored goods in the warehouse, helping factory enterprises to better manage the number of the goods and facilitating the management of a supervisor. The method of the invention can liberate the supervisory personnel, can conveniently see the current material condition in the warehouse from the system, form a bagged material warehouse-in and warehouse-out metering system, and is convenient for subsequent factory scheduling, thereby further realizing automation of factory production.