阅读说明：本技术 一种基于单双目视觉的机器人自动焊接系统及方法 (Robot automatic welding system and method based on single and binocular vision ) 是由 谢盛 魏昕 梁梓铭 屈海艳 林佳杰 于 2019-08-28 设计创作，主要内容包括：本发明提出一种基于单双目视觉的机器人自动焊接系统,包括前置视觉检测模块、运动控制模块、焊枪,其中前置视觉检测模块包括单目相机、双目相机、防飞溅挡板、图像采集卡,运动控制模块包括上位机、机器人控制器、运动轴执行机构；单目相机设置在焊枪的一侧,防飞溅挡板设置在单目相机与焊枪之间,双目相机设置在单目相机的另一侧,且单目相机和双目相机分别通过图像采集卡与上位机连接；上位机对焊缝图像进行分析处理,分别得到工件及焊缝的坐标以及运动轴执行机构的运动轨迹规划结果,并通过机器人控制器将运动轨迹规划结果相应的控制命令传送到运动轴执行机构。基于上述系统,本发明还提出了一种基于单双目视觉的机器人自动焊接方法。(the invention provides a robot automatic welding system based on single and binocular vision, which comprises a front vision detection module, a motion control module and a welding gun, wherein the front vision detection module comprises a monocular camera, a binocular camera, a splash-proof baffle plate and an image acquisition card; the monocular camera is arranged on one side of the welding gun, the anti-splashing baffle is arranged between the monocular camera and the welding gun, the binocular camera is arranged on the other side of the monocular camera, and the monocular camera and the binocular camera are respectively connected with the upper computer through the image acquisition card; and the upper computer analyzes and processes the welding seam image, respectively obtains the coordinates of the workpiece and the welding seam and the motion trail planning result of the motion axis executing mechanism, and transmits a control command corresponding to the motion trail planning result to the motion axis executing mechanism through the robot controller. Based on the system, the invention further provides a robot automatic welding method based on single and double eye vision.)

1. the utility model provides an automatic welding system of robot based on single binocular vision which characterized in that, includes leading visual detection module, motion control module, welder, wherein:

the front vision detection module comprises a monocular camera, a binocular camera, a splash-proof baffle and an image acquisition card;

The motion control module comprises an upper computer, a robot controller and a motion axis actuating mechanism;

the welding gun is arranged on the moving shaft executing mechanism, the upper computer is connected with the robot controller, and the robot controller is connected with the moving shaft executing mechanism;

The monocular camera is arranged on one side of the welding gun, the anti-splash baffle is arranged between the monocular camera and the welding gun, the binocular camera is arranged on the other side of the monocular camera, and the monocular camera and the binocular camera are respectively connected with the image acquisition card to transmit the acquired welding seam images to the upper computer; and the upper computer analyzes and processes the welding seam image, respectively obtains the coordinates of the workpiece and the welding seam and the motion trail planning result of the motion axis executing mechanism, and transmits a control command corresponding to the motion trail planning result to the motion axis executing mechanism through the robot controller.

2. The robotic automatic welding system of claim 1, wherein: the monocular camera comprises a camera mounting frame, a monocular camera lens, an optical filter and an auxiliary light source, wherein the monocular camera lens is mounted on the camera mounting frame, and the optical filter is arranged in front of the monocular camera lens.

3. The robotic automatic welding system of claim 2, wherein: the auxiliary light source is 445nm blue light.

4. a robot automatic welding method based on single and binocular vision is characterized by comprising the following steps:

s1: initializing and setting parameters of a monocular camera, and calibrating the monocular camera and the binocular camera respectively;

s2: acquiring images by a binocular camera, and performing binocular vision image processing on the acquired images to obtain spatial position information of a workpiece to be welded;

s3: the upper computer determines a welding starting point coordinate according to the spatial position information of the workpiece to be welded, acquires a current welding gun position coordinate through a robot controller, and plans a movement track of the welding gun according to the current welding gun position coordinate and the starting point coordinate information to obtain a track planning result;

S4: the upper computer transmits the track planning result to a robot controller, and the robot controller controls a moving axis executing mechanism to move the welding gun to a welding starting point;

S5: the robot controller acquires position information of a welding gun in real time, when the welding gun is confirmed to move to a welding starting point, the monocular camera starts to acquire welding seam image information, monocular visual image processing is carried out on the welding seam image to obtain spatial position information of the welding seam, and then welding gun offset corresponding to the currently acquired welding seam image is obtained according to the current welding gun position information acquired by the robot controller;

s6: the upper computer performs welding seam tracking according to the welding gun offset, performs filtering and fitting on the welding seam tracking track, calculates position information of tracking points in real time, and performs motion instruction planning;

s7: and the upper computer transmits the motion instruction plan to the robot controller, and the robot controller controls the motion axis executing mechanism according to the received motion instruction plan.

5. The robotic automated welding method of claim 4, wherein: in the step S1, the monocular camera parameters include camera exposure time, aperture, focal length, and FTP job server parameters, where the FTP job server parameters include, but are not limited to, FTP server IP address, port number, user name, and password.

6. The robotic automated welding method of claim 4, wherein: in the step S1, the monocular camera specifically performs monocular calibration as follows:

Printing a checkerboard calibration plate, and shooting pictures of the checkerboard calibration plate at different angles by using a monocular camera to obtain multi-angle calibration pictures; extracting corner information of each calibration picture, and further extracting sub-pixel corner information of each calibration picture according to the extracted corner information; drawing the found inner corner points on the chessboard calibration graph according to the extracted corner point information and the sub-pixel corner point information, and calculating to obtain a calibration result;

The binocular camera carries out monocular calibration and comprises the following specific steps:

performing monocular calibration on a left-eye lens and a right-eye lens in a binocular camera by adopting the monocular calibration method of the monocular camera to obtain an internal reference matrix and a distortion matrix of the left-eye lens and the right-eye lens; printing a chessboard pattern calibration plate, and shooting pictures of the chessboard pattern calibration plate at different angles by using a binocular camera to obtain multi-angle calibration pictures; extracting corner information of each calibration picture, and further extracting sub-pixel corner information of each calibration picture according to the extracted corner information; drawing the found inner angle points on the chessboard calibration graph according to the extracted angle point information and the sub-pixel angle point information to obtain a rotation matrix and a translation matrix of the left-eye lens and the right-eye lens; and cutting the calibration picture according to the rotation matrix and the translation matrix of the left-eye lens and the right-eye lens to enable polar lines of the calibration picture to be aligned.

7. the robotic automated welding method of claim 4, wherein: in the step S2, the upper computer performs binocular vision image processing on the acquired image by programming based on an Opencv vision library, and the specific steps are as follows:

correcting the acquired image according to preset binocular calibration parameters to obtain corrected images of the left eye and the right eye;

Performing median filtering on the corrected left-eye image and right-eye image respectively;

respectively carrying out histogram equalization on the filtered left eye image and the filtered right eye image;

respectively carrying out gray level processing on the equalized left eye image and the equalized right eye image;

Respectively carrying out self-adaptive threshold segmentation on the left eye image and the right eye image after the gray processing;

Performing morphological opening operation and closing operation on the left eye image and the right eye image after threshold segmentation respectively;

respectively extracting edges of the left eye image and the right eye image after the morphological operation;

Respectively carrying out characteristic point stereo matching and template matching on the left eye image and the right eye image after edge extraction to obtain two-dimensional information and distance information of the outline of the workpiece to be welded;

establishing a workpiece coordinate system with a binocular image coordinate system as a parent coordinate system according to the two-dimensional information and the distance information of the outline of the workpiece to be welded;

And obtaining the spatial position information of the workpiece to be welded according to the conversion between the workpiece coordinate system and the world coordinate system.

8. the robotic automated welding method of claim 7, wherein: in the step S3, the upper computer plans the movement track of the welding gun as follows:

the upper computer accurately positions the workpiece through template matching according to the spatial position information of the workpiece to be welded, and presets a workpiece welding starting point position on the positioning template;

determining current coordinates of the welding gun through the robot controller;

using moveit! And the function package performs joint space trajectory planning on the moving axis executing mechanism to obtain a trajectory planning result of the welding gun moving trajectory, so that the welding gun moves to the welding starting point.

9. the robotic automated welding method of claim 4, wherein: in the step S5, the monocular visual image processing includes the following specific steps:

correcting the acquired image according to the calibration parameters of the monocular camera to obtain an image corrected by the monocular camera;

Performing median filtering on the corrected image;

Carrying out histogram equalization on the filtered image;

carrying out gray level processing on the image after the histogram equalization;

performing self-adaptive threshold segmentation on the image subjected to the gray level processing;

Performing morphological opening operation and closing operation on the image after threshold segmentation;

performing edge extraction on the image after morphological operation;

performing linear fitting on the image after the edge extraction to obtain position information of a welding seam;

establishing a weld coordinate system with a monocular image coordinate system as a parent coordinate system according to the position information of the weld;

And obtaining the spatial position information of the welding seam according to the conversion between the welding seam coordinate system and the world coordinate system and the spatial position information of the workpiece to be welded, which is obtained by measuring through a binocular camera.

10. the robotic automated welding method of claim 9, wherein: in the step S6, the specific steps of the upper computer performing the weld tracking according to the offset of the welding gun are as follows:

and the upper computer performs Cartesian space trajectory interpolation according to the real-time position information of the welding seam, divides the welding gun offset corresponding to each welding seam image according to the frequency of image acquisition, and transmits the obtained control data to the robot controller to realize welding seam tracking.

Technical Field

the invention relates to the technical field of intelligent robot manufacturing, in particular to a robot automatic welding system and method based on single and binocular vision.

Background

the advent of welding robots represents a historical advance in welding automation, replacing manual welding and rigid automation, and creating a flexible automated production mode. At present, a welding robot widely applied to industrial production is mainly a teaching reproducibility robot, and the teaching reproducibility robot can effectively improve the production efficiency and ensure the consistency of welding quality under the condition of simpler welding seam track. However, for a welding seam with a complex track, the teaching reproducibility robot has the problems that manual teaching is long in time consumption, the stability requirement of welding quality is difficult to guarantee, and the like, and for small-batch and single-piece welding production, the robot teaching time accounts for a high proportion of all production working hours, and the overall working efficiency of the robot is influenced.

in order to solve the problems, an automatic welding system based on vision is mainly adopted for welding processing at present, wherein the automatic welding system mainly adopts a monocular or binocular camera for welding seam acquisition, but the monocular or binocular camera is easily affected by strong light splash in the welding process, so that the monocular or binocular camera needs to be provided with a corresponding filtering device or a corresponding protecting device, and the monocular or binocular camera cannot be applied to shooting under normal illumination; meanwhile, due to the precision requirement of welding seam identification, the focal length and the mounting position of a camera for identifying the welding seam are not suitable for searching a workpiece to be processed, and the automatic welding operation of the welding robot cannot be really realized.

Disclosure of Invention

The invention provides a robot automatic welding system based on single and binocular vision and a robot automatic welding method based on the single and binocular vision, aiming at overcoming the defect that the automatic welding operation of a welding robot cannot be realized in the prior art.

In order to solve the technical problems, the technical scheme of the invention is as follows:

the utility model provides a robot automatic weld system based on single binocular vision, includes leading visual detection module, motion control module, welder, wherein:

the front vision detection module comprises a monocular camera, a binocular camera, a splash-proof baffle and an image acquisition card;

the motion control module comprises an upper computer, a robot controller and a motion axis actuating mechanism;

The welding gun is arranged on the moving shaft executing mechanism, the upper computer is connected with the robot controller, and the robot controller is connected with the moving shaft executing mechanism;

the monocular camera is arranged on one side of the welding gun, the anti-splash baffle is arranged between the monocular camera and the welding gun, the binocular camera is arranged on the other side of the monocular camera, and the monocular camera and the binocular camera are respectively connected with the image acquisition card to transmit the acquired welding seam images to the upper computer; and the upper computer analyzes and processes the welding seam image, respectively obtains the coordinates of the workpiece and the welding seam and the motion trail planning result of the motion axis executing mechanism, and transmits a control command corresponding to the motion trail planning result to the motion axis executing mechanism through the robot controller.

in the technical scheme, the front vision detection module is used for acquiring a workpiece image to be identified and a welding seam image, and transmitting the workpiece image and the welding seam image to the motion control module through the Ethernet for image analysis and processing, wherein the binocular camera is used for acquiring three-dimensional information so as to identify and obtain a space coordinate of the workpiece; the monocular camera is used for acquiring two-dimensional plane information so as to identify and obtain welding seam position information; the anti-splashing baffle is arranged between the monocular camera and the welding gun and used for shielding splashing generated in the welding process. The motion control module is used for analyzing and processing the coordinates of the workpiece and the position information of the welding seam, planning the motion trail of the welding machine and controlling the movement of the welding machine by controlling the motion axis executing mechanism, wherein the upper computer receives the image information transmitted by the front visual detection module, analyzes the image information to obtain the coordinates of the workpiece and the position information of the welding seam, analyzes the coordinates of the workpiece and the position information of the welding seam, plans the motion trail of the welding gun, sends the obtained motion trail planning result to the robot controller, and the robot controller analyzes the received motion trail planning result to obtain a corresponding control command, realizes the control of the motion axis executing mechanism, controls the tracking, correcting and compensating motion of the welding machine on the position of the welding seam, and accordingly realizes the autonomous welding operation of the welding robot.

preferably, the monocular camera comprises a camera mounting frame, a monocular camera lens, an optical filter and an auxiliary light source, wherein the monocular camera lens is mounted on the camera mounting frame, and the optical filter is arranged in front of the monocular camera lens.

preferably, the secondary light source is 445nm blue light.

the invention also provides a robot automatic welding method based on single and binocular vision, which is applied to the robot automatic welding system based on the single and binocular vision and comprises the following steps:

S1: initializing and setting parameters of a monocular camera, and calibrating the monocular camera and the binocular camera respectively;

s2: acquiring images by a binocular camera, and performing binocular vision image processing on the acquired images to obtain spatial position information of a workpiece to be welded;

S3: the upper computer determines a welding starting point coordinate according to the spatial position information of the workpiece to be welded, acquires a current welding gun position coordinate through a robot controller, and plans a movement track of the welding gun according to the current welding gun position coordinate and the starting point coordinate information to obtain a track planning result;

S4: the upper computer transmits the track planning result to a robot controller, and the robot controller controls a moving axis executing mechanism to move the welding gun to a welding starting point;

s5: the robot controller acquires position information of a welding gun in real time, when the welding gun is confirmed to move to a welding starting point, the monocular camera starts to acquire welding seam image information, monocular visual image processing is carried out on the welding seam image to obtain spatial position information of the welding seam, and then welding gun offset corresponding to the currently acquired welding seam image is obtained according to the current welding gun position information acquired by the robot controller;

S6: the upper computer performs welding seam tracking according to the welding gun offset, performs filtering and fitting on the welding seam tracking track, calculates position information of tracking points in real time, and performs motion instruction planning;

S7: and the upper computer transmits the motion instruction plan to the robot controller, and the robot controller controls the motion axis executing mechanism according to the received motion instruction plan.

preferably, in step S1, the monocular camera parameters include camera exposure time, aperture, focal length, and FTP job server parameters, wherein the FTP job server parameters include, but are not limited to, FTP server IP address, port number, user name, password.

preferably, in the step S1, the specific steps of the monocular camera performing monocular calibration are as follows:

printing a checkerboard calibration plate, and shooting pictures of the checkerboard calibration plate at different angles by using a monocular camera to obtain multi-angle calibration pictures; extracting corner information of each calibration picture, and further extracting sub-pixel corner information of each calibration picture according to the extracted corner information; drawing the found inner corner points on the chessboard calibration graph according to the extracted corner point information and the sub-pixel corner point information, and calculating a calibration result;

The binocular camera carries out monocular calibration and comprises the following specific steps:

performing monocular calibration on a left-eye lens and a right-eye lens in a binocular camera by adopting the monocular calibration method of the monocular camera to obtain an internal reference matrix and a distortion matrix of the left-eye lens and the right-eye lens; printing a chessboard pattern calibration plate, and shooting pictures of the chessboard pattern calibration plate at different angles by using a binocular camera to obtain multi-angle calibration pictures; extracting corner information of each calibration picture, and further extracting sub-pixel corner information of each calibration picture according to the extracted corner information; drawing the found inner angle points on the chessboard calibration graph according to the extracted angle point information and the sub-pixel angle point information to obtain a rotation matrix and a translation matrix of the left-eye lens and the right-eye lens; and cutting the calibration picture according to the rotation matrix and the translation matrix of the left-eye lens and the right-eye lens so as to align polar lines.

Preferably, in the step S2, the upper computer performs binocular vision image processing on the acquired image by programming based on an Opencv vision library, and the specific steps are as follows:

Correcting the acquired image according to preset binocular calibration parameters to obtain corrected images of the left eye and the right eye;

Performing median filtering on the corrected left-eye image and right-eye image respectively;

Respectively carrying out histogram equalization on the filtered left eye image and the filtered right eye image;

respectively carrying out gray level processing on the equalized left eye image and the equalized right eye image;

respectively carrying out self-adaptive threshold segmentation on the left eye image and the right eye image after the gray processing;

performing morphological opening operation and closing operation on the left eye image and the right eye image after threshold segmentation respectively;

respectively extracting edges of the left eye image and the right eye image after the morphological operation;

respectively carrying out characteristic point stereo matching and template matching on the left eye image and the right eye image after edge extraction to obtain two-dimensional information and distance information of the outline of the workpiece to be welded;

Establishing a workpiece coordinate system with a binocular image coordinate system as a parent coordinate system according to the two-dimensional information and the distance information of the outline of the workpiece to be welded;

And obtaining the spatial position information of the workpiece to be welded according to the conversion between the workpiece coordinate system and the world coordinate system.

preferably, in the step S3, the upper computer plans the movement track of the welding gun as follows:

The upper computer accurately positions the workpiece through template matching according to the spatial position information of the workpiece to be welded, and presets a workpiece welding starting point position on the positioning template;

determining current coordinates of the welding gun through the robot controller;

using moveit! And the function package performs joint space trajectory planning on the moving shaft executing mechanism to obtain a trajectory planning result of the movement trajectory of the welding gun, so that the welding gun is aligned to the welding starting point.

Preferably, in step S5, the monocular visual image processing includes the following specific steps:

Correcting the acquired image according to the calibration parameters of the monocular camera to obtain an image corrected by the monocular camera;

performing median filtering on the corrected image;

carrying out histogram equalization on the filtered image;

carrying out gray level processing on the image after the histogram equalization;

performing self-adaptive threshold segmentation on the image subjected to the gray level processing;

Performing morphological opening operation and closing operation on the image after threshold segmentation;

Performing edge extraction on the image after morphological operation;

Performing linear fitting on the image after the edge extraction to obtain position information of a welding seam;

Establishing a weld coordinate system with a monocular image coordinate system as a parent coordinate system according to the position information of the weld;

and obtaining the spatial position information of the welding seam according to the conversion between the welding seam coordinate system and the world coordinate system and the spatial position information of the workpiece to be welded, which is obtained by measuring through a binocular camera.

preferably, in the step S6, the specific steps of the upper computer performing the weld tracking according to the offset of the welding gun are as follows: and the upper computer performs Cartesian space trajectory interpolation according to the real-time position information of the welding seam, divides the welding gun offset corresponding to each welding seam image according to the frequency of image acquisition, and transmits the obtained control data to the robot controller to realize welding seam tracking.

Compared with the prior art, the technical scheme of the invention has the beneficial effects that: the position information of a workpiece to be welded and a welding seam is collected through a binocular camera and a monocular camera, and then the movement of the welding robot is controlled through a motion control module in combination with image information, so that the autonomous welding operation of the welding robot is realized, and the production efficiency is effectively improved; the motion control module plans the motion track of the welding gun according to the image acquired in real time, so that the welding precision is effectively improved, and the influence of artificial subjective factors on the stability of the welding quality is avoided.

drawings

fig. 1 is a schematic structural diagram of a robot automatic welding system based on single and binocular vision according to embodiment 1.

fig. 2 is a schematic structural diagram of the front visual inspection module in embodiment 1.

Fig. 3 is a flowchart of the robot automatic welding method based on the monocular and binocular vision of embodiment 2.

FIG. 4 is a schematic drawing of a 23X 13 checkerboard calibration plate used in example 2.

Fig. 5 is a flowchart of binocular vision image processing and monocular vision image processing of embodiment 2.

Detailed Description

the drawings are for illustrative purposes only and are not to be construed as limiting the patent;

for the purpose of better illustrating the embodiments, certain features of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product;

it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.

The technical solution of the present invention is further described below with reference to the accompanying drawings and examples.