阅读说明：本技术 基于视觉检测的机械手辅助上下料和避障检测系统及方法 (Manipulator-assisted feeding and discharging and obstacle avoidance detection system and method based on visual detection ) 是由 付泽民 吴力杰 赵志繁 石博文 乔涛涛 熊鹏鹏 余奇 王志进 于 2019-09-27 设计创作，主要内容包括：本发明公开了基于视觉检测的机械手辅助上下料和避障检测系统及方法,包括送料系统,视觉检测装置和控制终端；送料系统为上料和下料一组机械手,上料机械手负责将物料送至压力机,下料机械手负责将折弯的板料从压力机下取出并码放在指定位置；视觉检测装置包括升降台、视觉传感器、数据处理单元,该装置主要负责对工作台的环境进行视觉扫描,经数据处理单元处理将信号传输给控制终端；所述控制终端包括控制台和显示器,通过无线收发模块与视觉检测装置相连,可以将视觉传感器检测到的图像直观显示出来,同时将控制指令传输给上料和下料机械手,帮助其及时识别障碍物以完成上料和下料作业,具备远程终端操控功能。(The invention discloses a manipulator-assisted feeding and discharging and obstacle avoidance detection system and method based on visual detection, and the system comprises a feeding system, a visual detection device and a control terminal, wherein the feeding system comprises groups of feeding and discharging manipulators, the feeding manipulator is responsible for feeding materials to a press, the discharging manipulator is responsible for taking out bent plates from the press and stacking the bent plates at an appointed position, the visual detection device comprises a lifting table, a visual sensor and a data processing unit, the device is mainly responsible for visually scanning the environment of the workbench and transmitting signals to the control terminal through processing of the data processing unit, the control terminal comprises a control table and a display, the control terminal is connected with the visual detection device through a wireless transceiving module, images detected by the visual sensor can be visually displayed, control instructions are transmitted to the feeding manipulator and the discharging manipulator, obstacles are helped to be timely recognized to complete feeding and discharging operations, and the system has a remote terminal control function.)

1. Supplementary unloading of unloading and keeping away barrier detecting system of manipulator based on visual detection, its characterized in that includes feeding system, visual detection device and control terminal, wherein:

feeding system includes material loading manipulator switch board, material loading manipulator, unloading manipulator switch board, unloading manipulator and wireless communication device, wherein:

the feeding manipulator control cabinet is arranged in a working area and used for controlling the feeding manipulator to convey a plate to be processed;

the feeding manipulator is arranged above the feeding manipulator control cabinet, is fixedly connected with the feeding manipulator control cabinet and is used for conveying the plate to be processed in the working area to the press machine;

the blanking manipulator control cabinet is arranged in the working area and used for controlling the blanking manipulator to stack the processed plates;

the blanking manipulator is arranged above the blanking manipulator control cabinet, is fixedly connected with the blanking manipulator control cabinet and is used for taking out the bent processed plate from the lower part of the press and stacking the bent processed plate at an appointed position in a working area;

distance sensors are mounted at the tail ends of the feeding mechanical arm and the discharging mechanical arm and are used for detecting the distance between the feeding mechanical arm and/or the discharging mechanical arm and an obstacle in a working area;

the th wireless communication device is respectively arranged in the feeding manipulator control cabinet and the discharging manipulator control cabinet and is used for receiving a th control command, a second control command and/or a third control command transmitted by the control terminal;

the visual detection device comprises a visual sensor, a lifting platform, a data processing unit and a second wireless communication device, wherein:

the lifting platform is arranged in the working area and used for driving the vision sensor to ascend or descend;

the visual sensor is arranged at the top of the lifting platform, is positioned at the side opposite to the feeding mechanical arm and the discharging mechanical arm, and is used for visually scanning the environment in the working area to obtain an original environment image/video;

the data processing unit is arranged in the lifting platform, is electrically connected with the vision sensor and the second wireless communication device, and is used for processing the original environment image/video detected by the vision sensor to obtain environment image/video information;

the second wireless communication device is arranged in the lifting platform, is electrically connected with the data processing unit, is also wirelessly connected with the th wireless communication device and the control terminal, and is used for transmitting the environmental image/video information obtained after the processing of the data processing unit to the control terminal;

the control terminal comprises a console and a third wireless communication device, wherein:

the control console is arranged outside the working area and is used for quickly determining the relative position of the plate to be processed and the feeding hole of the press according to the environment image/video information obtained after the processing of the data processing unit when no moving obstacle exists in the working area so as to form an th control instruction or quickly determining the relative position of the processed plate and the processed plate stacking area so as to form a second control instruction;

the third wireless communication device is arranged in the console, is wirelessly connected with the th wireless communication device and the second wireless communication device, and is used for receiving the environmental image/video information transmitted by the second wireless communication module and transmitting a control command, a second control command and/or a third control command generated by the console to the feeding manipulator and/or the discharging manipulator through the th wireless communication device, so as to complete obstacle avoidance.

2. The vision detection-based manipulator-assisted loading and unloading and obstacle avoidance detection system of claim 1, wherein the control terminal further comprises a display, and the display is electrically connected with the console and used for displaying an original environment image/video detected by the vision sensor in real time.

3. The vision detection-based manipulator-assisted loading and unloading and obstacle avoidance detection system of claim 1, wherein the control terminal further comprises an alarm, the alarm is connected with the console, and the alarm is used for triggering the alarm to give an alarm when a moving obstacle is detected to exist in the working area and the moving speed of the obstacle is too fast to allow the system to re-plan the moving path.

4. The vision detection-based manipulator-assisted loading and unloading and obstacle avoidance detection system as claimed in claim 1, wherein the vision sensor is a binocular vision camera, and is configured to remove interfering impurities in the images/videos through acquisition and post-processing of images/videos in a working area, obtain accurate obstacle position and shape information through filling segmentation and edge extraction of the images/videos by the data processing unit, and further realize map reconstruction and obstacle avoidance and navigation of the manipulator through the data processing unit.

5. The vision inspection-based manipulator-assisted loading and unloading and obstacle avoidance detection system of claim 1, wherein the th wireless communication device, the second wireless communication device and the third wireless communication device are signal transceivers.

6. The vision detection-based manipulator-assisted loading and unloading and obstacle avoidance detection method is characterized in that the vision detection-based manipulator-assisted loading and unloading and obstacle avoidance detection system disclosed by any in claims 1-5 is used for loading and unloading and obstacle avoidance, and the method specifically comprises the following steps:

step 1: presetting a basic moving path for the feeding and discharging manipulator according to the environment of a working area;

step 2: activating the visual inspection device to be in an operating mode;

and step 3: when unknown obstacles do not exist in the moving path, the vision sensor assists the mechanical arm to accurately load and unload the plate; when an unknown obstacle exists in the moving path, the visual sensor collects space and speed information of the obstacle and makes a new path;

and 4, step 4: when the path planning is successful, the vision sensor assists the mechanical arm to accurately load and unload the plate; when the path planning is unsuccessful and collision risk exists between the path planning and an obstacle, the alarm system gives an alarm, gives up the original path and calls a distance sensor at the tail end of the manipulator to simply avoid the obstacle;

and 5: and after the pure obstacle avoidance succeeds, planning a path again, and assisting the mechanical arm to accurately load and unload the plate by using the vision sensor.

7. The vision detection-based manipulator-assisted loading and unloading and obstacle avoidance detection method of claim 6, wherein in the step 2, the vision sensor monitors the whole working area in the whole process when the loading and unloading manipulator adsorbs the plate to move.

8. The vision detection-based manipulator auxiliary feeding and discharging and obstacle avoidance detection method of claim 6, wherein in step 3, when no unknown obstacle exists in the moving path, when the plate to be processed is to be sent to the range of the working area of the press, the vision sensor scans image/video information of the working area of the press, the image/video information is processed by the data processing unit, the relative position of the plate to be processed and a feeding port of the press is rapidly determined, the image/video information obtained after being processed by the data processing unit is transmitted to the console through the second wireless communication device, the console forms a -th control instruction, and the -th control instruction is transmitted to the feeding manipulator through the third wireless communication device, so that the manipulator can complete the feeding process more accurately.

9. The vision detection-based manipulator-assisted loading and unloading and obstacle avoidance detection method of claim 6, wherein in step 3, when an unknown obstacle exists in the moving path, the moving speed of the obstacle is rapidly calculated according to the original environment image/video detected by the vision sensor, the moving track is predicted, and the predicted track and the surrounding partial space range are defined as a dangerous area based on the prediction.

10. The vision inspection-based manipulator-assisted loading and unloading and obstacle avoidance detection system as claimed in claim 6, wherein in step 4, upper speed limits are set for the moving speed of the moving obstacle, and if the moving speed of the obstacle is fast and is not enough for the system to re-plan the moving path, an alarm system is triggered.

Technical Field

The invention relates to the field of intelligent bending operation of manipulators, in particular to a manipulator auxiliary feeding and discharging and obstacle avoidance detection system and method based on visual detection.

Background

In the field of metal plate forming, safe feeding problems of plates are topics which can not be avoided, in the early field of plate forming and manufacturing, plates are conveyed to a press machine or a feeding port of the press machine manually to complete feeding of the plates, but for large machines, close contact can cause great threat to workers, personal injury can be caused to operators slightly due to carelessness, with the development of technology, industrial robots begin to replace people to work with high safety risks, early manipulators cannot work coordinately like human beings in a multi-sense mode in the process of replacing feeding of people, therefore, the hidden danger of collision with the plates or the surrounding environment exists, even personal injury accidents are caused, and with the development of modern sensor technology, the flexibility problem of the manipulators can be better solved.

Disclosure of Invention

In order to overcome the defects in the prior art, the invention provides a manipulator-assisted loading and unloading and obstacle avoidance detection system and method based on visual detection, and solves the safety problem caused by manual loading and unloading and obstacle avoidance in the prior art.

In order to achieve the above purpose, the technical solution for solving the technical problem is as follows:

the invention discloses a manipulator auxiliary feeding and discharging and obstacle avoidance detection system based on visual detection, which comprises a feeding system, a visual detection device and a control terminal, wherein the feeding system comprises a feeding mechanism, a feeding mechanism and a control terminal, wherein the feeding mechanism comprises:

feeding system includes material loading manipulator switch board, material loading manipulator, unloading manipulator switch board, unloading manipulator and wireless communication device, wherein:

the feeding manipulator control cabinet is arranged in a working area and used for controlling the feeding manipulator to convey a plate to be processed;

the feeding manipulator is arranged above the feeding manipulator control cabinet, is fixedly connected with the feeding manipulator control cabinet and is used for conveying the plate to be processed in the working area to the press machine;

the blanking manipulator control cabinet is arranged in the working area and used for controlling the blanking manipulator to stack the processed plates;

the blanking manipulator is arranged above the blanking manipulator control cabinet, is fixedly connected with the blanking manipulator control cabinet and is used for taking out the bent processed plate from the lower part of the press and stacking the bent processed plate at an appointed position in a working area;

distance sensors are mounted at the tail ends of the feeding mechanical arm and the discharging mechanical arm and are used for detecting the distance between the feeding mechanical arm and/or the discharging mechanical arm and an obstacle in a working area;

the th wireless communication device is respectively arranged in the feeding manipulator control cabinet and the discharging manipulator control cabinet and is used for receiving a th control command, a second control command and/or a third control command transmitted by the control terminal;

the visual detection device comprises a visual sensor, a lifting platform, a data processing unit and a second wireless communication device, wherein:

the lifting platform is arranged in the working area and used for driving the vision sensor to ascend or descend;

the visual sensor is arranged at the top of the lifting platform, is positioned at the side opposite to the feeding mechanical arm and the discharging mechanical arm, and is used for visually scanning the environment in the working area to obtain an original environment image/video;

the data processing unit is arranged in the lifting platform, is electrically connected with the vision sensor and the second wireless communication device, and is used for processing the original environment image/video detected by the vision sensor to obtain environment image/video information;

the second wireless communication device is arranged in the lifting platform, is electrically connected with the data processing unit, is also wirelessly connected with the th wireless communication device and the control terminal, and is used for transmitting the environmental image/video information obtained after the processing of the data processing unit to the control terminal;

the control terminal comprises a console and a third wireless communication device, wherein:

the control console is arranged outside the working area and is used for quickly determining the relative position of the plate to be processed and the feeding hole of the press according to the environment image/video information obtained after the processing of the data processing unit when no moving obstacle exists in the working area so as to form an th control instruction or quickly determining the relative position of the processed plate and the processed plate stacking area so as to form a second control instruction;

the third wireless communication device is arranged in the console, is wirelessly connected with the th wireless communication device and the second wireless communication device, and is used for receiving the environmental image/video information transmitted by the second wireless communication module and transmitting a control command, a second control command and/or a third control command generated by the console to the feeding manipulator and/or the discharging manipulator through the th wireless communication device, so as to complete obstacle avoidance.

, the control terminal further comprises a display electrically connected to the console for displaying the original environment image/video detected by the vision sensor in real time.

, the control terminal further comprises an alarm connected to the console for triggering the alarm to give an alarm when it is detected that a moving obstacle exists in the working area and the moving speed of the obstacle is too fast to allow the system to re-plan the moving path.

Preferably, the vision sensor is a binocular vision camera, and is used for eliminating interference impurities in the images/videos through acquisition and post-processing of the images/videos of the working area environment, obtaining accurate obstacle position and shape information through filling and segmentation and edge extraction of the images/videos through the data processing unit, and further realizing map reconstruction through the data processing unit and obstacle avoidance and navigation of the manipulator.

Preferably, the th wireless communication device, the second wireless communication device and the third wireless communication device are all signal transceivers.

The invention also discloses a manipulator auxiliary feeding and discharging and obstacle avoidance detection method based on visual detection, which utilizes the manipulator auxiliary feeding and discharging and obstacle avoidance detection system based on visual detection to carry out feeding and discharging and obstacle avoidance and specifically comprises the following steps:

step 1: presetting a basic moving path for the feeding and discharging manipulator according to the environment of a working area;

step 2: activating the visual inspection device to be in an operating mode;

and step 3: when unknown obstacles do not exist in the moving path, the vision sensor assists the mechanical arm to accurately load and unload the plate; when an unknown obstacle exists in the moving path, the visual sensor collects space and speed information of the obstacle and makes a new path;

and 4, step 4: when the path planning is successful, the vision sensor assists the mechanical arm to accurately load and unload the plate; when the path planning is unsuccessful and collision risk exists between the path planning and an obstacle, the alarm system gives an alarm, gives up the original path and calls a distance sensor at the tail end of the manipulator to simply avoid the obstacle;

and 5: and after the pure obstacle avoidance succeeds, planning a path again, and assisting the mechanical arm to accurately load and unload the plate by using the vision sensor.

And step , in step 2, in the process that the feeding and discharging mechanical arm adsorbs the plate to move, the vision sensor monitors the whole working area in the whole process.

Step , in step 3, when there is no unknown obstacle in the moving path, when the plate to be processed is to be sent to the range of the working area of the press machine, the visual sensor can scan the image/video information of the working area of the press machine, the plate to be processed and the feeding port of the press machine are rapidly determined by the processing of the data processing unit, the image/video information obtained by the processing of the data processing unit is sent to the console through the second wireless communication device, the console forms a control command, and the control command is sent to the feeding manipulator through the third wireless communication device, so that the manipulator can more accurately complete the feeding process.

Step , in step 3, when there is unknown obstacle in the moving path, rapidly calculating the moving speed of the obstacle according to the original environment image/video detected by the vision sensor and predicting the moving track, and defining the predicted track and the surrounding partial space range as the dangerous area based on the prediction.

, in step 4, upper speed limits are set for the moving speed of the moving obstacle, and if the moving speed of the obstacle is high and is not enough for the system to re-plan the moving path, an alarm system is triggered.

Due to the adoption of the technical scheme, compared with the prior art, the invention has the following advantages and positive effects:

the manipulator-assisted feeding and discharging and obstacle avoidance detection system based on visual detection integrates visual detection, is used for scanning environmental obstacles in the whole field of a working area and accurately feeding auxiliary manipulators, realizes the connection and control among systems by utilizing wireless communication, is convenient for operators to remotely observe the running state of equipment in real time through a control terminal in a safe area, and keeps the emergency control right on the equipment.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and other drawings can be obtained by those skilled in the art without inventive effort, wherein:

FIG. 1 is a view of a working scene of a manipulator-assisted safe loading system in the manipulator-assisted loading and obstacle avoidance detection system based on visual detection according to the present invention;

FIG. 2 is a schematic structural diagram of an unloading and loading manipulator in the manipulator-assisted unloading and obstacle avoidance detection system based on visual detection according to the present invention;

FIG. 3 is a schematic structural diagram of a visual detection device in the manipulator-assisted feeding and discharging and obstacle avoidance detection system based on visual detection according to the present invention;

fig. 4 is a schematic diagram of a specific work flow of the manipulator-assisted feeding and discharging and obstacle avoidance detection method based on visual detection.

[ description of main symbols ]

1-a press;

2-a feeding manipulator control cabinet;

3-a feeding manipulator;

4-a feeding manipulator control cabinet;

5-a blanking manipulator;

6-a working area;

7-a visual inspection device;

71-a vision sensor;

72-a lifting platform;

8-processing the plate;

9-a plate to be processed;

and 10, controlling the terminal.

Detailed Description

While the embodiments of the present invention will be described and illustrated in detail below with reference to the accompanying drawings, it is to be understood that only some, but not all, of the embodiments of the present invention are illustrated and described herein, and that all other embodiments obtained by those skilled in the art without any inventive step will fall within the scope of the present invention.