阅读说明：本技术 基于图像处理的用于环卫车的智能控制机械臂及控制方法 (Intelligent control mechanical arm for sanitation vehicle based on image processing and control method ) 是由 程磊 单龙 孙进 宋洁 张凯 许海川 陈诚 于 2020-05-06 设计创作，主要内容包括：本发明公开了一种基于图像处理的用于环卫车的智能控制机械臂及控制方法,所述的控制方法包括以下步骤：实时采集图像,从图像中识别出垃圾桶的图像；判断环卫车与垃圾桶的位置关系,驱动环卫车移动到可抓取垃圾桶的位置；根据停车时环卫车相对于垃圾桶的位置关系,控制机械臂执行抓取垃圾桶的操作。所述机械臂包括图像识别模块,行车指示模块、机械臂和机械臂控制单元。本发明通过采用视觉传感器对垃圾桶进行图像识别,并运算其空间三维坐标,为环卫车辆的停放和智能机械臂的伸出距离和抓取位置提供数据,从而确定智能机械臂伸出机构伸出距离的方法。(The invention discloses an intelligent control mechanical arm for a sanitation vehicle based on image processing and a control method, wherein the control method comprises the following steps: acquiring images in real time, and identifying images of the trash can from the images; judging the position relation between the sanitation vehicle and the garbage can, and driving the sanitation vehicle to move to a position where the garbage can be grabbed; and controlling the mechanical arm to execute the operation of grabbing the garbage can according to the position relation of the sanitation vehicle relative to the garbage can during parking. The mechanical arm comprises an image recognition module, a driving indication module, a mechanical arm and a mechanical arm control unit. According to the method, the image recognition is carried out on the garbage can by adopting the visual sensor, the spatial three-dimensional coordinate of the garbage can is calculated, data are provided for the parking of sanitation vehicles and the extending distance and the grabbing position of the intelligent mechanical arm, and therefore the extending distance of the extending mechanism of the intelligent mechanical arm is determined.)

1. A control method for realizing mechanical arms of an intelligent sanitation truck based on an image acquisition system is characterized by comprising the following steps:

acquiring images in real time, and identifying images of the trash can from the images;

judging the position relation between the sanitation vehicle and the garbage can, and driving the sanitation vehicle to move to a position where the garbage can be grabbed;

and controlling the mechanical arm to execute the operation of grabbing the garbage can according to the position relation of the sanitation vehicle relative to the garbage can during parking.

2. The control method according to claim 1, characterized in that: the step of identifying the images of the trash can comprises the steps of comparing the collected images with a database for storing all trash can picture information, and identifying whether the images collected in real time contain the trash can or not from the comparison result.

3. The control method according to claim 1, characterized in that:

representing the spatial position relation of the sanitation vehicle and the garbage can by adopting three-dimensional coordinates (x, y, z), wherein the three-dimensional coordinates take the installation point of the mechanical arm on the sanitation vehicle as an origin of coordinates;

and judging whether the garbage can is in a grippable range or not according to the distance between the garbage can and the mechanical arm calculated according to the three-dimensional coordinate (x, y, z), if so, executing a gripping operation, otherwise, calculating a proper parking position of the sanitation vehicle according to the coordinate value, driving the sanitation vehicle to a driving direction in a specified direction, and executing the gripping operation when the sanitation vehicle is parked in an area of the grippable position of the garbage can.

4. The control method according to claim 1, characterized in that: the process of grabbing the garbage can comprises the step of controlling the extending distance and the speed of a telescopic oil cylinder connected with the mechanical arm to grab the garbage can according to the position relation of the sanitation vehicle relative to the garbage can when the garbage can is parked.

5. The utility model provides an intelligent arm for sanitation car which characterized in that: the system comprises an image recognition module, a driving indication module, a mechanical arm and a mechanical arm control unit;

the image identification module is configured to acquire images around the sanitation vehicle in real time and identify images of the garbage can from the images;

the driving indicating module is configured to drive the sanitation vehicle to move to a position where the garbage can be grabbed according to the judged position relation between the sanitation vehicle and the garbage can;

and the mechanical arm control unit is configured to control the mechanical arm to execute the operation of grabbing the garbage can according to the position relation of the sanitation vehicle relative to the garbage can during parking.

6. A robotic arm as claimed in claim 5, in which: the intelligent traffic control system is characterized by further comprising a main controller, wherein the image recognition module, the driving indication module and the mechanical arm control unit are connected with the main controller, transmit information received in real time to the main controller and receive instructions of the main controller.

7. A robotic arm as claimed in claim 6, in which: the image recognition module comprises a visual sensor and an operation host;

at least two visual sensors are arranged on the outer side of each of the front, rear, left and right side surfaces of the sanitation vehicle;

the operation host is provided with a database for storing all the garbage can picture information, and is used for comparing the image information captured by the vision sensor with the information in the database in real time and judging whether the garbage can is captured in the acquired image.

8. A robotic arm as claimed in claim 6, in which: the mechanical arm control unit comprises a telescopic oil cylinder, an extending electromagnetic proportional valve and a retracting electromagnetic proportional valve which are connected with the telescopic oil cylinder, the on-off and valve core opening of the extending electromagnetic proportional valve and the retracting electromagnetic proportional valve are adjusted through a PID closed-loop control method, and the extending distance and the extending speed of the telescopic oil cylinder are controlled.

9. A robotic arm as claimed in claim 8, in which: the mechanical arm control unit further comprises a displacement sensor for recording the displacement distance of the mechanical arm extending out for the first time and controlling the mechanical arm to stretch and contract by adopting the length equal to the first distance at each time in the process of carrying the same garbage can.

10. A robotic arm as claimed in claim 8, in which: the mechanical arm control unit further comprises a garbage unloading unit, the garbage unloading unit comprises a bucket holding oil cylinder, a bucket turning oil cylinder, a pressure detector, an upward turning in-place detector and a downward turning in-place detector, the bucket holding oil cylinder is connected with the pressure detector, the upward turning in-place detector and the downward turning in-place detector are connected with the bucket turning oil cylinder, the garbage unloading unit is connected with the main controller, and the garbage collection and the garbage placing back of the mechanical arm are controlled.

Technical Field

The invention belongs to the technical field of intelligent sanitation vehicles, and particularly relates to an intelligent control mechanical arm for a sanitation vehicle based on image processing and a control method.

Background

When garbage truck of present domestic use collects rubbish, all need have the sanitary worker to push away garbage bin to collection device department to on fixed collection device, operating personnel passes through operating button switch control collection device's lift, goes into the garbage carriage with the rubbish in the garbage bin in, this kind of traditional collection mode, and it is big to need operating personnel many and intensity of labour, is unfavorable for reducing the human cost.

Disclosure of Invention

Aiming at the problems, the invention provides an intelligent control mechanical arm for a sanitation vehicle based on image processing and a control method.

The technical purpose is achieved, the technical effect is achieved, and the invention is realized through the following technical scheme:

the invention provides a control method for realizing a mechanical arm of an intelligent sanitation truck based on an image acquisition system, which comprises the following steps:

acquiring images in real time, and identifying images of the trash can from the images;

judging the position relation between the sanitation vehicle and the garbage can, and driving the sanitation vehicle to move to a position where the garbage can be grabbed;

and controlling the mechanical arm to execute the operation of grabbing the garbage can according to the position relation of the sanitation vehicle relative to the garbage can during parking.

As a further improvement of the invention, the step of identifying the images of the trash can comprises comparing the acquired images with a database storing all trash can picture information, and identifying whether the images acquired in real time contain the trash can or not from the comparison result.

As a further improvement of the invention, a three-dimensional coordinate (x, y, z) is adopted to represent the spatial position relationship between the sanitation vehicle and the garbage can, and the three-dimensional coordinate takes the installation point of the mechanical arm on the sanitation vehicle as the origin of coordinates; and judging whether the garbage can is in a grippable range or not according to the distance between the garbage can and the mechanical arm calculated according to the three-dimensional coordinate (x, y, z), if so, executing a gripping operation, otherwise, calculating a proper parking position of the sanitation vehicle according to the coordinate value, driving the sanitation vehicle to a driving direction in a specified direction, and executing the gripping operation when the sanitation vehicle is parked in an area of the grippable position of the garbage can.

As a further improvement of the invention, the process of grabbing the garbage can comprises the step of controlling the extending distance and the speed of a telescopic oil cylinder connected with the mechanical arm to grab the garbage can according to the position relation of the sanitation vehicle relative to the garbage can when the garbage can is parked.

Based on the control method, the invention also provides an intelligent mechanical arm for the sanitation vehicle, which comprises an image recognition module, a driving indication module, a mechanical arm and a mechanical arm control unit;

the image identification module is configured to acquire images around the sanitation vehicle in real time and identify images of the garbage can from the images;

the driving indicating module is configured to drive the sanitation vehicle to move to a position where the garbage can be grabbed according to the judged position relation between the sanitation vehicle and the garbage can;

and the mechanical arm control unit is configured to control the mechanical arm to execute the operation of grabbing the garbage can according to the position relation of the sanitation vehicle relative to the garbage can during parking.

As a further improvement of the invention, the intelligent traffic control system further comprises a main controller, wherein the image recognition module, the driving indication module and the mechanical arm control unit are connected with the main controller, transmit the information received in real time to the main controller and receive the instruction of the main controller.

As a further improvement of the invention, the image recognition module comprises a vision sensor and an operation host;

at least two visual sensors are arranged on the outer side of each of the front, rear, left and right side surfaces of the sanitation vehicle;

the operation host is provided with a database for storing all the garbage can picture information, and is used for comparing the image information captured by the vision sensor with the information in the database in real time and judging whether the garbage can is captured in the acquired image.

As a further improvement of the invention, the mechanical arm control unit comprises a telescopic oil cylinder, an extending electromagnetic proportional valve and a retracting electromagnetic proportional valve which are connected with the telescopic oil cylinder, and the extension distance and the extension speed of the telescopic oil cylinder are controlled by adjusting the on-off and the valve core opening of the extending electromagnetic proportional valve and the retracting electromagnetic proportional valve through a PID closed-loop control method.

As a further improvement of the invention, the mechanical arm control unit further comprises a displacement sensor for recording the extended displacement distance of the telescopic oil cylinder when the mechanical arm grabs the garbage can for the first time, and controlling the mechanical arm to adopt the same extended displacement distance of the oil cylinder in the process of putting back the garbage can.

As a further improvement of the invention, the mechanical arm control unit also comprises a garbage unloading unit, the garbage unloading unit comprises a bucket holding cylinder, a bucket turning cylinder, a pressure detector, an upturning in-place detector and a downturning in-place detector, the bucket holding cylinder and the pressure detector are connected with the bucket holding cylinder, the upturning in-place detector and the downturning in-place detector are connected with the bucket turning cylinder, and the garbage unloading unit is connected with the main controller and controls the mechanical arm to collect garbage and put the garbage can back.

The invention has the beneficial effects that:

(1) the vision sensor is adopted to carry out image recognition on the garbage can, the three-dimensional space coordinate of the garbage can is calculated, and data are provided for parking of sanitation vehicles, the extending distance of the intelligent mechanical arm and the grabbing position of the intelligent mechanical arm.

(2) The method for determining the extending distance of the intelligent mechanical arm extending mechanism according to the spatial three-dimensional coordinates of the garbage can is provided.

(3) According to feedback displacement data of a displacement sensor arranged in the oil cylinder, a PID closed-loop control method is adopted to control the on-off of the electromagnetic proportional valve and the opening degree of a valve core, so that the extending distance and the extending speed of the telescopic oil cylinder are controlled, and the intelligent mechanical arm telescopic mechanism is ensured to quickly and accurately reach a set displacement value.

(4) Provides a method for replacing the garbage can by memorizing the extending displacement of the oil cylinder.

Drawings

FIG. 1 is a schematic view of the overall structure of the sanitation truck;

FIG. 2 is a schematic view of a robotic arm;

fig. 3 is a structural connection diagram of a control system arranged on the sanitation vehicle.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The following detailed description of the principles of the invention is provided in connection with the accompanying drawings.

An intelligent sanitation vehicle as shown in fig. 1 comprises a sanitation vehicle 1, an image recognition module, a driving indication module, a mechanical arm 4 and a mechanical arm control unit.

The intelligent mechanical arm 4 is arranged on a sanitation vehicle 1, the image recognition module (ICP 6600-7) is a visual sensor 3(ISV200-I-01) arranged on the front side, the rear side, the left side and the right side of a garbage box body 2 of the vehicle, each side at least comprises two visual sensors 3, and all the visual sensors 3 on the same side are taken as a group. When the operation host (ARK-1124) is used for positioning the garbage can, under the premise of determining the three-dimensional coordinates (x, y, z) of each vision sensor 3 relative to the mechanical arm, the operation host extracts and processes the received image, and the operation host firstly compares the image with a database to identify the garbage can; after identifying the trash can, the operation host computer adopts a Python language program to calculate the three-dimensional position of the trash can in the image relative to the acquired visual sensor 3 according to the image depth information, and finally obtains the exact three-dimensional coordinate of the trash can relative to the mechanical arm 4.

Wherein, arm 4 installs on arm seat 6, when calculating the three-dimensional coordinate point of garbage bin, uses the pin joint of both as the origin of coordinates. In the invention, a plurality of visual sensors 3(ISV200-I-01) are adopted in four directions, so that comprehensive search can be carried out on garbage cans existing in the surrounding environment, and errors of position calculation are reduced.

In the vehicle driving process or the garbage collecting process, the vision sensor 3 captures image information of an object in a monitoring area in real time and transmits the captured image information to the operation host 8 in real time, the image information of various garbage cans is stored in the operation host 8, the operation host 8 calls the image information of the garbage cans in the storage area to be compared with the captured image, judgment is carried out, whether the garbage cans are contained or not is identified, and if yes, the driving indication module or the mechanical arm control unit is correspondingly started according to the space three-dimensional coordinate relation of the garbage cans and the mechanical arm.

Firstly, judging whether the garbage can is in a grippable range or not according to the distance between the garbage can and the mechanical arm calculated by the three-dimensional coordinate value, and if so, executing a gripping operation. If not, the driving indication module is operated, the main controller 9 judges the snatchable range of the garbage can according to the three-dimensional coordinate value, calculates the proper parking range of the sanitation vehicle according to the coordinate value, simultaneously displays the result on the display device, adopts images and sounds to prompt the driving direction and the parking position of the driver, and the driver starts the sanitation vehicle to drive in the appointed direction according to the indicated range.

When the vehicle is started, the main controller 9 detects the change of the space position in real time until the vehicle stops in a proper grabbing range; and after the garbage bin is stopped, determining the three-dimensional coordinate value of the garbage bin again until the garbage bin is completely determined to be within the grabbing range of the holding claw, and executing grabbing operation. If the vehicle is in the proper grabbing range, the automatic feeding action can be executed, if the vehicle is not in the proper grabbing range, the prompt is given on the display device, and the sanitation vehicle continues to operate until a proper stopping point is found.

When the grabbing operation is executed, the main controller 9 sends various control instructions to the mechanical arm control unit to control the mechanical arm 4 to automatically complete the whole garbage collection process, and the garbage can is put back to the original place.

Specifically, the main controller 9 receives the ready signal 19, and the robot arm control unit is activated; when the three-dimensional coordinates of the garbage can are in a set range, the main controller 9 calculates the distance of the intelligent mechanical arm 4 which should extend in the horizontal direction according to the current x value, determines a set value of the horizontal distance, outputs a horizontal extending action instruction to drive the telescopic oil cylinder 10 to extend, the mechanical arm 4 extends in the horizontal direction, a displacement sensor 11(KYDM-LP1A4210-GJ0440M4S0300WS) is installed inside the telescopic oil cylinder 10, the displacement sensor 11 feeds back displacement data to the main controller 9 in real time, the main controller 9 adjusts the on-off and valve core opening degrees of the extending electromagnetic proportional valve 17(XSV12-3HH01) and the retracting electromagnetic proportional valve 18(XSV12-3HH01) in a PID closed-loop control mode according to the feedback displacement data, so as to adjust the extending distance and the extending speed of the telescopic oil cylinder 10, when the extending distance is equal to a judgment value obtained according to the position relation, the main controller 9, and memorizes the displacement data of the displacement sensor 11 at this time, and then completes the garbage collection and the operation of putting the garbage can back to the original place.

The main controller 9 outputs a barrel holding command to drive the claw holding oil cylinder to extend, the intelligent mechanical arm 4 carries out a garbage barrel holding action, when the barrel is held in place, the pressure detector 7(PT250R-2104) transmits a barrel holding in place signal to the main controller 9, the main controller 9 stops outputting the barrel holding command and simultaneously outputs a horizontal direction retracting command to drive the telescopic oil cylinder 10 to retract, the intelligent mechanical arm 4 holds the garbage barrel tightly to carry out a horizontal direction retracting action, when the garbage barrel is retracted in place, the built-in displacement sensor 11 of the telescopic oil cylinder 10 feeds back displacement data equal to 0 and transmits the data to the main controller 9, the main controller 9 stops outputting the retracting command and simultaneously outputs an upturning command to drive the barrel overturning oil cylinder 12 to extend, the intelligent mechanical arm 4 holds the garbage barrel tightly to carry out an upturning action, when the upturning in place, the upturning in place detector 14(BD15-S3-M30) transmits an, the main controller 9 stops outputting an upturning instruction, starts timing, executes garbage dumping action, when the timing time reaches a set value, the main controller 9 outputs a downturning instruction, drives the barrel overturning oil cylinder 12 to retract, the intelligent mechanical arm 4 tightly holds the garbage barrel to downturn, when the downturning is in place, the downturning position detector 13(BD15-S3-M30) transmits a downturning position signal to the main controller 9, the main controller 9 stops outputting the downturning instruction and simultaneously outputs a horizontal extending instruction, the intelligent mechanical arm 4 tightly holds the garbage barrel to horizontally extend, the displacement sensor 11 of the telescopic oil cylinder 10 feeds back displacement data in real time, when the feedback displacement data is equal to the memorized displacement data, the main controller 9 stops outputting an extending instruction, simultaneously outputs a barrel loosening instruction, drives the barrel holding oil cylinder 16 to retract, the intelligent mechanical arm 4 executes a claw holding loosening action, when the claw holding is in place, the barrel loosening in-place detector (15-S3-M30) transmits a barrel loosening in-place signal to the main controller, the main controller 9 stops outputting a barrel loosening instruction, the garbage can is put back to the original position, the telescopic oil cylinder retracts, and the intelligent mechanical arm 4 finishes full-automatic collection of garbage.

In the present system, the main controller 9 communicates with the display device using a CAN bus.

The foregoing shows and describes the general principles and broad features of the present invention and advantages thereof. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.