阅读说明：本技术 一种基于视觉技术和深度学习的存余垃圾识别方法和装置 (Remaining garbage identification method and device based on visual technology and deep learning ) 是由 袁志业 张文涛 宫文龙 于 2020-06-16 设计创作，主要内容包括：本发明公开了一种基于视觉技术和深度学习的存余垃圾识别方法和装置,包括匀料器、视觉单元、执行单元、输送线和总控机,输送线为电机驱动的传送带装置,第一传送带的上方支架上连接有匀料器,物料从匀料器的上端进入,从匀料器的下端均匀的落入第一传送带上,第一传送带旁的地面上固定连接有总控机,物料顺着第一传送带运动到第二传送带上,第二传送带将物料运转到第三传送带上,本发明将替代人工劳动力,排除对人体危害,并且相比单纯依靠基于深度学习和机器视觉完成物体识别检测来讲,本研究先通过得到物体密度和坐标,再将少部分图像中物体进行算法识别,可以大幅度缩短识别耗时,提高识别精准度达到99.5％。(The invention discloses a method and a device for identifying remaining garbage based on a visual technology and deep learning, which comprises a material homogenizer, a visual unit, an execution unit, a conveying line and a master controller, wherein the conveying line is a conveying belt device driven by a motor, the material homogenizer is connected on a bracket above a first conveying belt, materials enter from the upper end of the material homogenizer and uniformly fall onto the first conveying belt from the lower end of the material homogenizer, a master controller is fixedly connected on the ground beside the first conveying belt, the materials move onto a second conveying belt along the first conveying belt, and the second conveying belt conveys the materials onto a third conveying belt. The time consumption of recognition can be greatly shortened, and the recognition accuracy is improved to 99.5%.)

1. A method for identifying remaining garbage based on visual technology and deep learning is characterized in that materials on a heavy material conveying line are basically inorganic materials such as 'bricks, stones, rubbles, concrete blocks' and the like, but the materials are generally mixed with 1% -10% of impurities, the inorganic materials such as 'plastics, rubbers, cotton fabrics, woods', 'bricks, stones, rubbles, concrete blocks' and the like have high dry density which is larger than m0, if the cotton fabrics contain a large amount of muddy water materials, the density of the cotton fabrics is increased and may be larger than m0, an a1 object is in a visual area, the mass b1 is obtained by weighing, the volume v1 is obtained by a binocular camera, the density d1 is b1/v1, and an object image and coordinates are obtained by a monocular camera; determining the size of a threshold value m0, judging d1, if d1 is less than m0, directly judging the object to be determined as an impurity, if d1 is greater than m0, judging the object to be determined, extracting an image where the object to be determined is located through an interested region, entering a visual unit, completing the identification of a target object image by a deep learning training model and machine vision fusion, completing the training of the deep learning training model after preprocessing an image sample of a residual garbage sample database, identifying the image of the object to be determined through vision after preprocessing, and if the image is correctly identified, outputting a detection result to an end effector, wherein the result comprises an object type and coordinates; if the pictures are not recognized, the pictures are stored in a designated folder for later supplementary training, so that the database is continuously promoted.

2. The method for identifying the remaining garbage based on the visual technology and the deep learning as claimed in claim 1, wherein the working process comprises the steps that inorganic heavy objects obtained through front end mechanical sorting firstly enter the material homogenizer (100), the materials are scattered and spread on the first conveyor belt (401), then the scattered materials are conveyed to the visual unit (200) through the second traditional conveyor belt (402), when a single material enters the third conveyor belt (403), the single material is firstly sensed by the infrared sensor (407) at the feeding end, the first conveyor belt (401), the second conveyor belt (402) and the material homogenizer (100) stop working, after the detection is finished, the material is sensed by the infrared sensor (407) at the discharging end of the third conveyor belt (403), the first conveyor belt (401), the second conveyor belt (402) and the material homogenizer (100) start to act again, and the materials finish the binocular camera (201) and the binocular camera (201) in sequence in the visual unit (200), The monocular camera (202) and the conveying and weighing module (204) detect the object and then enter the execution unit (300), the object is grabbed or rejected by the actuator (301), all signal control communication and transmission in the visual unit (200) are fed back to the master controller (500), and the master controller (500) sends real-time signals to control the actuator (301) to complete grabbing or rejection.

3. The device for identifying the remaining garbage based on the visual technology and the deep learning is characterized by comprising a material homogenizer (100), a visual unit (200), an execution unit (300), a conveying line (400) and a master controller (500), wherein the conveying line (400) is a conveying belt device driven by a motor, the material homogenizer (100) is connected on a support above a first conveying belt (401), materials enter from the upper end of the material homogenizer (100) and uniformly fall onto the first conveying belt (401) from the lower end of the material homogenizer (100), the master controller (500) is fixedly connected to the ground beside the first conveying belt (401), the materials move onto a second conveying belt (402) along the first conveying belt (401), the second conveying belt (402) conveys the materials onto a third conveying belt (403), the area where the third conveying belt (403) is located is the visual unit (200), and finally the materials are analyzed by the visual unit (200) and then run onto a fourth conveying belt (404) from the third conveying belt (403), the area where the fourth conveyor belt (404) is located is an execution unit (300), and collecting boxes (302) are placed on two sides of the fourth conveyor belt (404).

4. A device for identifying remaining garbage based on visual technique and deep learning according to claim 3, wherein the visual unit (200) comprises: binocular camera (201), monocular camera (202), light source (203) and transport weighing module (204), carry weighing module (204) to be connected with third conveyer belt (403), all be connected with light source (203) on the feed end of third conveyer belt (403) and the top support of discharge end, between two light sources (203), the feed end that is close to third conveyer belt (403) is connected with binocular camera (201), be close to third conveyer belt (403) discharge end and be connected with monocular camera (202).

5. A device for identifying remaining garbage based on visual technique and deep learning according to claim 4, wherein the conveying and weighing module (204) comprises: the conveying belt conveyor comprises a base (405), a pressure sensor (406) and a third conveying belt (403), wherein a pressure sensor (406) is connected to a support for supporting the second conveying belt (402) and the fourth conveying belt (404), the base (405) is connected to the upper end of the pressure sensor (406), and the third conveying belt (403) is connected to the base (405) in a rotating mode.

6. A device for identifying remaining garbage based on visual technique and deep learning as claimed in claim 5, wherein the bracket near the feeding end and the discharging end of the third conveyor belt (403) is connected with infrared sensors (407).

7. A device for identifying remaining garbage based on visual technique and deep learning as claimed in claim 3, wherein the execution unit (300) comprises an actuator (301) and a collection box (302), the actuator (301) is connected to the upper end of the fourth conveyor belt (404) through a bracket, and the collection box (302) is placed at the two ends of the fourth conveyor belt (404) and is located where the actuator (301) can reach.

8. The device for identifying the remaining garbage based on the visual technology and the deep learning as claimed in claim 3, wherein a second conveyor belt (402) is arranged, so that the light and heavy materials can be smoothly conveyed from the first conveyor belt (401) to the third conveyor belt (403), the feeding end of the second conveyor belt (402) is slightly lower than the discharging end of the first conveyor belt (401), the discharging end of the second conveyor belt (402) is slightly higher than the feeding end of the third conveyor belt (403), and the second conveyor belt (402) is obliquely connected to a bracket for supporting and conveying.

Technical Field

The invention relates to the field of sorting of remaining garbage, in particular to a remaining garbage identification method and device based on a visual technology and deep learning.

Background

In recent years, on the industrial assembly line technology for sorting household garbage and construction garbage, sorting work is mainly completed through traditional mechanical sorting and manual assistance in China, and meanwhile, technologies based on photoelectricity, electromagnetism, wind power and the like are researched, developed and applied in the industry; developed countries develop a robot vision sorting technology based on deep learning on a domestic garbage, construction garbage and general industrial garbage sorting production line, and most typically abroad, such as a ZenRobotics robot in Finland, which is applied to the construction garbage sorting, and an MAX _ AI mechanical arm of a BHS company in America utilizes and identifies garbage materials to sort garbage and the like. But the investment is high, and the characteristics of the garbage left in China are not applicable.

The left garbage in China is mostly mixed and piled for years, the irregular garbage pile body mainly containing domestic garbage is provided with buildings, general industry and other garbage, the environment is complex, the traditional mechanical sorting is adopted, and the sorting effect with high requirement purity is difficult to achieve through single functional equipment combinations such as a rotary screen, wind power, photoelectricity, a vibrating screen and a bouncing screen, the manual assistance capability is limited, and the labor force is gradually lacking.

The introduction of foreign robots for sorting has high investment, and the database needs to be established again according to the garbage characteristics of China, so that the difficulty is high, and the algorithm needs to be continuously optimized according to the garbage characteristics of China. The robot sorting technology of each country is applied to the field with high garbage recycling value due to high investment, such as building garbage, industrial garbage or domestic garbage with high plastic and metal content, and no application case exists in the field of sorting the remaining garbage at present, so that the robot sorting technology is innovative and applied to the field of intelligently sorting the remaining garbage, is matched with a traditional intelligent garbage sorting assembly line to replace manual auxiliary sorting, and can solve the problem of high impurity screening rate to the maximum extent.

Disclosure of Invention

The invention aims to provide a method and a device for identifying remaining garbage based on a visual technology and deep learning, so as to solve the problems in the background technology.

In order to achieve the purpose, the invention provides the following technical scheme:

a remaining garbage identification method based on vision technology and deep learning is characterized in that materials (the grain diameter is more than 40 mm and less than 200 mm) on a heavy material conveying production line are basically inorganic materials such as bricks, stones, rubbles and concrete blocks, but 1% -10% of impurities are generally mixed, the dry density of the inorganic materials such as plastics, rubbers, cotton fabrics and wood, the dry density of the inorganic materials such as the bricks, the stones, the rubbles and the concrete blocks is larger and is larger than m0, if the cotton fabrics contain a large amount of muddy water, the density of the cotton fabrics is increased and may be larger than m0, an a1 object is weighed in a vision area to obtain the mass b1, a binocular camera obtains the volume v1, a computer calculates the density d1 to b1/v1, and a monocular camera obtains an object image and coordinates of the cotton fabrics; determining the size of a threshold value m0, judging d1, if d1 is less than m0, directly judging the object to be determined as an impurity, if d1 is greater than m0, judging the object to be determined, extracting an image where the object to be determined is located through an interested region, entering a visual unit, completing the identification of a target object image by a deep learning training model and machine vision fusion, completing the training of the deep learning training model after preprocessing an image sample of a residual garbage sample database, identifying the image of the object to be determined through vision after preprocessing, and if the image is correctly identified, outputting a detection result to an end effector, wherein the result comprises an object type and coordinates; if the pictures are not recognized, the pictures are stored in a designated folder for later supplementary training, so that the database is continuously promoted.

A method for recognizing the left garbage based on visual technique and deep learning includes such steps as mechanically sorting the inorganic heavy objects (mainly brick, stone, or gravel) from front end, spreading them on the first conveyer belt, sending them to visual unit via the second conveyer belt, sensing by infrared sensor at feeding end when a single object is fed to the third conveyer belt, stopping the first, second and uniform conveyers, detecting the object by infrared sensor at discharging end of the third conveyer belt, starting the first, second and uniform conveyers, picking up the object by the executor, all signal control communication and transmission in the visual unit are fed back to the master controller, and the master controller sends real-time signals to control the actuator to complete grabbing or rejecting actions.

The utility model provides a surplus rubbish recognition device based on vision technique and degree of depth study, including the evener, the vision unit, the execution unit, transfer chain and total accuse machine, the transfer chain is motor drive's conveyer belt device, be connected with the evener on the top support of first conveyer belt, the material gets into from the upper end of evener, fall into on the first conveyer belt from the lower extreme of evener, subaerial fixedly connected with master control machine by the first conveyer belt, the material moves to the second conveyer belt along first conveyer belt on, the second conveyer belt transports the material to the third conveyer belt on, the region in third conveyer belt place is the vision unit, last material transports to the fourth conveyer belt from the third conveyer belt after the vision unit analysis on, the region in fourth conveyer belt place is the execution unit, the collecting box has been placed to the both sides of fourth conveyer belt.

As a further scheme of the invention: the visual unit includes: binocular camera, monocular camera, light source and carry weighing module, carry weighing module and be connected with the third conveyer belt, all be connected with the light source on the feed end of third conveyer belt and the top support of discharge end, between two light sources, the feed end that is close to the third conveyer belt is connected with binocular camera, is close to the third conveyer belt discharge end and is connected with monocular camera.

As a still further scheme of the invention: the conveying and weighing module comprises: the base, pressure sensor and third conveyer belt are connected with pressure sensors on the support that supports second conveyer belt and fourth conveyer belt, and pressure sensors's upper end is connected with the base, rotates on the base and is connected with the third conveyer belt.

As a still further scheme of the invention: and infrared sensors are connected to the brackets beside the feeding end and the discharging end of the third conveyor belt.

As a still further scheme of the invention: the execution unit comprises an actuator and a collection box, the actuator is connected to the upper end of the fourth conveyor belt through a support, and the collection box is placed at the two ends of the fourth conveyor belt and is located at a position where the actuator can reach.

As a still further scheme of the invention: the second conveyor belt is arranged, the feeding end of the second conveyor belt is slightly lower than the discharging end of the first conveyor belt, the discharging end of the second conveyor belt is slightly higher than the feeding end of the third conveyor belt, and the second conveyor belt is obliquely connected to the support for supporting and conveying the light and heavy materials.

Compared with the prior art, the invention has the beneficial effects that: (1) the manual labor force is replaced, and the harm to the human body is eliminated; (2) compared with the method of simply finishing object identification and detection based on deep learning and machine vision, the method has the advantages that the object density and coordinates are obtained, and then the objects in a small part of images are subjected to algorithm identification, so that the identification time consumption can be greatly shortened, and the identification accuracy is improved to 99.5%.

Drawings

Fig. 1 is a block diagram of a method for identifying remaining garbage based on vision technology and deep learning.

Fig. 2 is a schematic structural diagram of a remaining garbage recognition device based on a visual technology and deep learning.

Fig. 3 is a top view of a remaining garbage recognition apparatus based on a visual technique and deep learning.

Fig. 4 is an enlarged view of the structure a in fig. 2.

As shown in the figure: 100. a material homogenizer; 200. a vision unit; 201. a binocular camera; 202. a monocular camera; 203. a light source; 204. a conveying and weighing module; 300. an execution unit; 301. an actuator; 302. a collection box; 400. a conveying line; 401. a first conveyor belt; 402. a second conveyor belt; 403. a third conveyor belt; 404. a fourth conveyor belt; 405. a base; 406. a pressure sensor; 407. an infrared sensor; 500. and a master control machine.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Referring to fig. 1 to 4, in an embodiment of the present invention, a method for identifying remaining garbage based on a vision technology and deep learning is implemented in such a manner that materials (40 mm < particle size < 200 mm) on a heavy material conveying line are substantially inorganic materials such as "bricks, stones, rubbles, concrete blocks" and the like, but are generally mixed with 1% to 10% of impurities, the dry densities of the inorganic materials such as "plastics, rubbers, cotton fabrics, woods", "bricks, stones, rubbles, concrete blocks" and the like are generally large and are all > m0, if the cotton fabrics contain a large amount of muddy water, the densities of the cotton fabrics are increased and may also be > m0, an a1 object is in a vision area, the mass b1 is obtained by weighing, a binocular camera obtains a volume v1, and a density d1 is b 1/1, and a monocular camera obtains an object image and coordinates of the object; determining the size of a threshold value m0, judging d1, if d1 is less than m0, directly judging the object to be determined as an impurity, if d1 is greater than m0, judging the object to be determined, extracting an image where the object to be determined is located through an interested region, entering a visual unit, completing the identification of a target object image by a deep learning training model and machine vision fusion, completing the training of the deep learning training model after preprocessing an image sample of a residual garbage sample database, identifying the image of the object to be determined through vision after preprocessing, and if the image is correctly identified, outputting a detection result to an end effector, wherein the result comprises an object type and coordinates; if the pictures are not recognized, the pictures are stored in a designated folder for later supplementary training, so that the database is continuously promoted.

A method for recognizing the left garbage based on visual technique and deep learning includes such steps as mechanically sorting the front end of inorganic heavy objects, loading them in a material homogenizer 100, spreading them on a first conveyer belt 401, sending them to a visual unit 200 via a second conveyer belt 402, sensing them by an infrared sensor 407 at the feeding end when a single material is fed in a third conveyer belt 403, stopping the first conveyer belt 401, the second conveyer belt 402 and the homogenizer 100, detecting the material by the infrared sensor 407 at the discharging end of the third conveyer belt 403, restarting the first conveyer belt 401, the second conveyer belt 402 and the homogenizer 100, detecting the material by a binocular camera 201, a monocular camera 202 and a conveying and weighing module 204 in the visual unit 200, and sending it to an execution unit 300, the target object is grabbed or rejected by the executor 301, all signal control communication and transmission in the visual unit 200 are fed back to the master controller 500, and the master controller 500 sends real-time signals to control the executor 301 to complete grabbing or rejection.

A device for identifying the remaining garbage based on visual technology and deep learning comprises a material homogenizer 100 and a visual unit 200, the material conveying device comprises an execution unit 300, a conveying line 400 and a general control machine 500, wherein the conveying line 400 is a motor-driven conveyor belt device, a material homogenizer 100 is connected to a support above a first conveying belt 401, materials enter from the upper end of the material homogenizer 100 and uniformly fall onto the first conveying belt 401 from the lower end of the material homogenizer 100, the general control machine 500 is fixedly connected to the ground beside the first conveying belt 401, the materials move onto a second conveying belt 402 along the first conveying belt 401, the second conveying belt 402 conveys the materials onto a third conveying belt 403, an area where the third conveying belt 403 is located is a visual unit 200, the materials are finally analyzed by the visual unit 200 and then conveyed onto a fourth conveying belt 404 from the third conveying belt 403, an area where the fourth conveying belt 404 is located is the execution unit 300, and collecting boxes 302 are placed on two sides of the fourth conveying belt 404.

The visual unit 200 includes: binocular camera 201, monocular camera 202, light source 203 and carry weighing module 204, carry weighing module 204 to be connected with third conveyer belt 403, all be connected with light source 203 on the top support of the feed end of third conveyer belt 403 and discharge end, between two light sources 203, the feed end that is close to third conveyer belt 403 is connected with binocular camera 201, is close to third conveyer belt 403 discharge end and is connected with monocular camera 202.

The conveyor weighing module 204 includes: the conveyer belt comprises a base 405, a pressure sensor 406 and a third conveyer belt 403, wherein a pressure sensor 406 is connected to a bracket for supporting the second conveyer belt 402 and the fourth conveyer belt 404, the upper end of the pressure sensor 406 is connected with the base 405, and the third conveyer belt 403 is rotatably connected to the base 405.

And infrared sensors 407 are connected to the brackets beside the feeding end and the discharging end of the third conveyor belt 403.

The execution unit 300 includes an actuator 301 and a collection box 302, the actuator 301 is connected to the upper end of the fourth conveyor belt 404 by a bracket, and the collection box 302 is placed at both ends of the fourth conveyor belt 404 at a place where the actuator 301 can reach.

In order to facilitate the smooth transportation of light and heavy materials from the first conveyor belt 401 to the third conveyor belt 403, the second conveyor belt 402 is provided, the feeding end of the second conveyor belt 402 is slightly lower than the discharging end of the first conveyor belt 401, the discharging end of the second conveyor belt 402 is slightly higher than the feeding end of the third conveyor belt 403, and the second conveyor belt 402 is obliquely connected to a bracket for supporting transportation.

In the aspect of identification algorithm, through test simulation under the condition of indoor equivalent computers, the logic judgment method identifies the target object with the speed of 250-450 milliseconds; and a sample model is trained by a large data set, and the model is called to identify the speed of the target object in 680-1150 milliseconds. In the aspect of hardware devices, a simulated environment is built in an indoor test, and the material can be uniformly scattered and uniformly spread on a conveying belt by using the material loading of a material homogenizer. The visual area can be respectively realized by using monocular and binocular cameras, the object coordinate is positioned, the interest area is extracted, and the volume is obtained by three-dimensional reconstruction of the target object. The object obtains real-time mass data in the conveying process of the weighing module.

In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art through specific situations.

Although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that various changes in the embodiments and/or modifications of the invention can be made, and equivalents and modifications of some features of the invention can be made without departing from the spirit and scope of the invention.