阅读说明：本技术 一种基于图像配准的工件碰撞检测方法及喷涂系统 (Workpiece collision detection method based on image registration and spraying system ) 是由 王也 陈文彬 林国正 谢宝宁 梁远标 丘焯平 于 2021-07-29 设计创作，主要内容包括：本发明提供了一种基于图像配准的工件碰撞检测方法及喷涂系统,其包括如下步骤：S1、选定检测区域；S2、在检测区域内设置实体坐标系；S3、使用摄像装置捕捉并生成实体坐标系的点云；S4、根据捕捉到的点云拟合出参照坐标系；S5、将参照坐标系对实体坐标系进行对比,若参照坐标系相于实体坐标系的误差小于预设值,则参照坐标系拟合成功；还包括喷涂房、链条传送机构和摄像装置,摄像装置安装于述喷涂房前的两侧,链条传送机构位于喷涂房的顶部,工件进入喷涂房之前,检测工件相对于喷涂房的晃动量,当工件的晃动量超出预设值则停止链条传送机构或者发出碰撞预警；该技术方案具有可对工件进行实时的碰撞检测,并可实现不同区域的分级报警的的优点。(The invention provides a workpiece collision detection method based on image registration and a spraying system, which comprises the following steps: s1, selecting a detection area; s2, setting a solid coordinate system in the detection area; s3, capturing and generating point cloud of a physical coordinate system by using a camera device; s4, fitting a reference coordinate system according to the captured point cloud; s5, comparing the reference coordinate system with the physical coordinate system, and if the error of the reference coordinate system relative to the physical coordinate system is smaller than a preset value, successfully fitting the reference coordinate system; the device comprises a spraying room, a chain conveying mechanism and a camera device, wherein the camera device is arranged on two sides in front of the spraying room, the chain conveying mechanism is positioned at the top of the spraying room, the shaking amount of a workpiece relative to the spraying room is detected before the workpiece enters the spraying room, and when the shaking amount of the workpiece exceeds a preset value, the chain conveying mechanism is stopped or collision early warning is sent out; the technical scheme has the advantages of being capable of performing real-time collision detection on the workpiece and realizing graded alarm in different areas.)

1. A workpiece collision detection method based on image registration is characterized by comprising the following steps:

s1, selecting a detection area;

s2, setting a solid coordinate system in the detection area;

s3, capturing and generating a point cloud of the physical coordinate system by using a camera device;

s4, fitting a reference coordinate system according to the captured point cloud;

and S5, comparing the reference coordinate system with the physical coordinate system, if the error of the reference coordinate system relative to the physical coordinate system is smaller than a preset value, indicating that the reference coordinate system is successfully fitted, otherwise, adjusting the physical coordinate system and/or the camera device.

2. The image registration-based workpiece collision detection method according to claim 1, further comprising the steps of:

s6, when the fitting of the reference coordinate system is successful, fixing the position of the camera device relative to the area to be detected;

s7, converting the captured reference coordinate system into a world coordinate system;

and S8, setting parameters of the world coordinate system to determine triggering conditions of collision early warning.

3. The image registration-based workpiece collision detection method according to claim 2, further comprising the steps of:

and S9, removing the physical coordinate system arranged in the area to be detected after the reference coordinate system is successfully fitted.

4. The image registration-based workpiece collision detection method according to any one of claims 1-3, wherein the physical coordinate system comprises three spheres, the three spheres form end points of an X axis, a Y axis and a Z axis of the physical coordinate system respectively, and the spheres are capture objects of the point cloud.

5. The image registration-based workpiece collision detection method according to claim 4, wherein the surface of the sphere is ground or brushed.

6. The image registration-based workpiece collision detection method according to any one of claims 1 to 3, wherein the camera device respectively performs point cloud capture on the areas where the three spheres are located, and fits the reference coordinate system by using a least square method.

7. The method of claim 2, wherein step S7 further includes obtaining parameter values of the reference coordinate system and forming a Pc matrix, obtaining parameter values of the world coordinate system and forming a Pw matrix, and the Pc matrix is inverse-operated by right-multiplying the Pw matrix to obtain a rotational offset matrix Tctw;

wherein the Pw matrix is represented as: pw ═ (R + L00R + L/3, 0R + L/3, 00R + L R + L/3, 1111);

the Pc matrix is represented as: pc ═ P_1x P_2x P_3x P_1x+P_2x+P_3x/3、P_1y P_2y P_3yP_1y+P_2y+P_3y/3、P_1z P_2zP_3z P_1z+P_2z+P_3z/3、1 1 1 1)。

8. The workpiece collision detection method based on image registration according to claim 1, wherein there are two image capturing devices, the two image capturing devices are respectively located at two sides of the detection area, and the image capturing devices are depth cameras.

9. The image registration-based workpiece collision detection method according to claim 8, wherein the distance from the image pickup device to the detection area is 0.6-1.3M.

10. A spray system, characterized by: the device comprises a spraying room, a chain conveying mechanism and a camera device, wherein the camera device is arranged at two sides in front of the spraying room, the chain conveying mechanism is positioned at the top of the spraying room and used for conveying workpieces into and out of the spraying room, and the camera device detects the shaking amount of the workpieces relative to the spraying room by the workpiece collision detection method based on image registration according to any one of claims 1-9 before the workpieces enter the spraying room;

and when the shaking amount of the workpiece exceeds a preset value, stopping the chain conveying mechanism or sending out collision early warning.

11. The spray system of claim 10, wherein: be provided with collision detection area and safe zone in the detection zone of camera device field of vision cover, the collision detection area correspond to the border in spraying room, the safe zone corresponds to the removal district of work piece, be non-detection area outside the collision detection area.

Technical Field

The invention belongs to the technical field of spraying, and particularly relates to a workpiece collision detection method based on image registration and a spraying system.

Background

In a painting line, the workpieces to be painted move along with the conveyor chain in a suspended manner. Under the mode of suspending the workpiece, the workpiece is almost inevitable to shake, and particularly the workpiece shakes obviously in the starting and stopping processes of the conveying chain; the shaking range of the workpiece is detected in real time, the warning is carried out on the shaking workpiece beyond the safety range, and even the conveying chain is stopped emergently, so that the detection is one of the key points. Specifically, the workpiece shaking is detected to avoid collision of the workpiece with a large shaking amplitude on the spraying room or the spray gun after entering the spraying room, so that equipment is damaged or the spraying quality of the workpiece is affected.

In the prior art, both grating and laser scanning are common position detection means, and after the position detection means are arranged in the width direction, the height direction and other directions, a workpiece is detected on a working plane of the position detection means; however, it is limited in that the plane of detection is fixed and, for workpieces that have already passed through their work plane, if a wobble occurs, it is not possible to continuously track the alarm.

Disclosure of Invention

In view of the above, an object of the present invention is to provide a workpiece collision detection method and a spraying system based on image registration, which can perform real-time collision detection on a workpiece and can achieve hierarchical alarm of different areas.

In order to solve the technical problems, the technical scheme adopted by the invention is as follows:

a workpiece collision detection method based on image registration comprises the following steps:

s1, selecting a detection area;

s2, setting a solid coordinate system in the detection area;

s3, capturing and generating a point cloud of the physical coordinate system by using a camera device;

s4, fitting a reference coordinate system according to the captured point cloud;

and S5, comparing the reference coordinate system with the physical coordinate system, if the error of the reference coordinate system relative to the physical coordinate system is smaller than a preset value, indicating that the reference coordinate system is successfully fitted, otherwise, adjusting the physical coordinate system and/or the camera device.

As a further improvement to the workpiece collision detection method based on image registration, the method further comprises the following steps:

s6, when the fitting of the reference coordinate system is successful, fixing the position of the camera device relative to the area to be detected;

s7, converting the captured reference coordinate system into a world coordinate system;

and S8, setting parameters of the world coordinate system to determine triggering conditions of collision early warning.

As a further improvement to the workpiece collision detection method based on image registration, the method further comprises the following steps:

and S9, removing the physical coordinate system arranged in the area to be detected after the reference coordinate system is successfully fitted.

As a further improvement of the workpiece collision detection method based on image registration, the physical coordinate system includes three spheres, the three spheres respectively form end points of an X axis, a Y axis and a Z axis of the physical coordinate system, and the spheres are capture objects of the point cloud.

As a further improvement to the image registration-based workpiece collision detection method, the surface of the sphere is ground or brushed.

As a further improvement of the workpiece collision detection method based on image registration, the camera device respectively captures point clouds of regions where the three spheres are located, and fits the reference coordinate system by using a least square method.

As a further improvement of the image registration-based workpiece collision detection method, in step S7, the method further includes obtaining parameter values of the reference coordinate system and forming a Pc matrix, obtaining parameter values of the world coordinate system and forming a Pw matrix, and multiplying the Pc matrix inverse operation by the Pw matrix to obtain a rotational offset matrix Tctw;

wherein the Pw matrix is represented as: pw ═ (R + L00R + L/3, 0R + L/3, 00R + L R + L/3, 1111);

the Pc matrix is represented as: pc ═ P_1x P_2x P_3x P_1x+P_2x+P_3x/3、P_1y P_2y P_3y P_1y+P_2y+P_3y/3、P_1zP_2z P_3z P_1z+P_2z+P_3z/3、1 1 1 1)。

As a further improvement of the workpiece collision detection method based on image registration, two image capturing devices are provided, the two image capturing devices are respectively located at two sides of the detection area, and the image capturing devices are depth cameras.

As a further improvement of the workpiece collision detection method based on image registration, the distance between the camera device and the detection area is 0.6-1.3M.

A spraying system comprises a spraying room, a chain conveying mechanism and a camera device, wherein the camera device is installed at two sides in front of the spraying room, the chain conveying mechanism is located at the top of the spraying room, the chain conveying mechanism is used for conveying workpieces into and out of the spraying room, and the camera device detects the shaking amount of the workpieces relative to the spraying room through the workpiece collision detection method based on image registration;

and when the shaking amount of the workpiece exceeds a preset value, stopping the chain conveying mechanism or sending out collision early warning.

As a further improvement of the spraying system, a collision detection area and a safety area are arranged in a detection area covered by the visual field of the camera device, the collision detection area corresponds to the boundary of the spraying room, the safety area corresponds to the moving area of the workpiece, and a non-detection area is arranged outside the collision detection area.

Compared with the prior art, the invention has the following beneficial effects: the invention utilizes the characteristic of wide working range of the structured light depth camera to carry out real-time collision detection on the whole range from the side, and utilizes the algorithm to realize the graded alarm of different areas, so that the collision detection or the protection of the spraying equipment has better pertinence; furthermore, the potential of a camera device (a depth camera) in the aspect of collision detection is excavated, (the quality of point cloud obtained by the surface of a metal wire drawing ball in a visual field range of a consumer-grade structured light camera is found to meet the requirement of a spherical fitting algorithm), a complete set of algorithm is developed, the positioning problem of the depth camera is successfully solved, a matrix construction calculation method of space coordinate conversion is solved in detail, and engineering application is realized on the ground; compared with the common grating and line laser scanning which can only work on the working plane arranged on the laser scanning device, the invention can realize large-scale real-time detection without being limited to a specific working plane.

Drawings

The foregoing and other objects, features and advantages of the invention will be apparent from the following more particular description of preferred embodiments of the invention, as illustrated in the accompanying drawings. Like reference numerals refer to like parts throughout the drawings, and the drawings are not intended to be drawn to scale in actual dimensions, emphasis instead being placed upon illustrating the principles of the invention.

FIG. 1 is a point cloud obtained by a camera device from a physical coordinate system according to the present invention;

FIG. 2 is a schematic view of collision detection in the present invention;

Detailed Description

The present invention will be described in further detail with reference to the drawings and specific examples so that those skilled in the art can better understand the present invention and can implement the present invention, but the examples are not intended to limit the present invention, and in the present examples, it should be understood that the terms "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of describing the present invention, but do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present invention.

It will be understood that when an element is referred to as being "connected" to another element, it can be directly connected to the other element and be integral therewith, or intervening elements may also be present. The terms "mounted," "one end," "the other end," and the like as used herein are for illustrative purposes only.

As shown in fig. 1-2, the present embodiment provides a workpiece collision detection method based on image registration, which includes the following steps:

s1, selecting a detection area; specifically, after a workpiece enters a spraying room or before the workpiece is sprayed correspondingly, the shaking amplitude of the workpiece is ensured to be kept within a safe range, so that the workpiece is prevented from colliding the spraying room or spraying equipment due to shaking; in order to be able to detect the workpiece in an effective manner, a detection region is therefore provided, which in the present exemplary embodiment is the boundary formed by the inner contour in the painting booth or a corresponding region through which the workpiece can pass.

S2, setting a solid coordinate system in the detection area; in this embodiment, three metal surface drawing balls of known radius R are welded to be perpendicular to each other by connecting rods of known length L. The positioning device is mounted on a tripod, placed in the field of view of the depth camera, and leveled using a level. Although the algorithm in the invention is not limited to the position of the ball, as long as the imaging quality meets the requirement of spherical fitting, the ball can be placed at will to establish any coordinate system suitable for field use, but the best practice of the invention is to make the X axis parallel to the direction of the conveying chain, the Y axis and the Z axis respectively in the horizontal direction and the vertical direction (the Y axis can be towards the inner side or the outer side of the powder room without essential difference), and the connection points of the three connecting rods are placed at the position flush with the vertical edge of the powder room. Therefore, the edge of the powder room can be used as a reference object for setting alarm parameters, and the y coordinate is 0, so that the subsequent operation is facilitated.

S3, capturing and generating point cloud of a solid coordinate system by using a camera device, wherein the figure circled by the red circle in the figure 1 is the sphere point cloud captured and generated by the depth camera; the point data set of the product appearance surface obtained by using a measuring instrument in the engineering is also called point cloud; that is, a point cloud refers to a collection of a vast number of points of a target surface characteristic. The number of points obtained by using a three-dimensional coordinate measuring machine is small, and the distance between the points is large, so that the method is called sparse point cloud; the point clouds obtained by using a three-dimensional laser scanning or photographic scanner have a large and dense point number, which is called dense point clouds.

S4, fitting a reference coordinate system according to the captured point cloud;

and S5, comparing the reference coordinate system with the physical coordinate system, if the error of the reference coordinate system relative to the physical coordinate system is smaller than a preset value, indicating that the reference coordinate system is successfully fitted, otherwise, adjusting the physical coordinate system and/or the camera device. Further, the image capturing apparatus in this embodiment is a depth camera, and further, a point cloud is captured in regions where three balls (physical coordinate systems) are located by using the depth camera. The method for capturing is a sphere center radius method: and (3) intercepting the point cloud by taking one point of the spherical surface as the center of the sphere and taking R as the radius so as to ensure that only the point of the spherical surface exists in the obtained point cloud without other interference. The captured spherical point cloud is fitted to the center coordinates (camera coordinate system) of the sphere and the radius R0 of the sphere by the least square method. In this example, a sphere having a radius of 75mm is used, and if the radius error of the fitting sphere is within ± 1mm, the fitting is considered to be successful. If the radius of the spherical surface fitting meets the requirement, the distances among the three spherical centers are calculated, and if the distances all meet √ 2L +/-3 mm, the fitting is considered to be successful. If the fitting radius does not meet the error requirement, the fitting radius is mostly caused by two reasons, one is that the optimal shooting distance is exceeded between the ball and the camera, the other is that strong light interference exists on the spot, the point cloud capturing is incomplete, the trial is required to be carried out from two aspects of shading treatment and moving the small ball, and the position of the camera device is usually reasonably adjusted until the fitting is successful. Experimental results show that the optimal fitting distance of the consumer-grade monocular structured light is 0.6m-1.3m, such as 0.7m, 0.8m, 0.9m, 1.0m, 1.1m and 1.2m, and certainly, if budget is met, a camera with better parameters can be selected.

Further, in a preferred embodiment, the method further comprises the following steps:

s6, when the fitting of the reference coordinate system is successful, fixing the position of the camera device relative to the area to be detected; specifically, the coating rooms with different specifications or detection areas with different sizes, different installation environments, and different specifications of the camera devices all affect the fitting, so that the specific position of the camera device needs to be finally determined and fixed after the fitting is successful, that is, the position of the camera device is located so as to obtain a rotation matrix of the depth camera relative to the coating rooms.

S7, converting the captured reference coordinate system into a world coordinate system; specifically, the point coordinates obtained by the depth camera are converted into a world coordinate system, so that the alarm parameters and the detection parameters can be easily set and calculated. In order to obtain the 4 × 4 rotational offset matrix Tctw (transmission camera to world), two 4 × 4 matrices pw (points of world), pc (points of camera) are constructed using the coordinates obtained in the previous step to find Tctw. In the following formula, each row of Pw is the coordinate of the center of a small sphere of the X axis, the Y axis and the Z axis and the center of an equilateral triangle formed by three points in the world coordinate system, each row of Pc is the coordinate of the centers of three small spheres and the center of an equilateral triangle formed by the three small spheres in the camera coordinate system, the last line of the two matrices is filled with 1, the final line is used as the augmentation of the matrix, the inversion operation is conveniently carried out by using a computer, and the rotation offset matrix Tctw is obtained at one time. Wherein the content of the first and second substances,

since Pw is related to Tctw, Tctw is Pw-1. And multiplying Tctw by the coordinates obtained by all the camera coordinate systems to obtain all the coordinates in the world coordinate system. And finishing the calculation of the space coordinate system transformation matrix. Specifically, a Pc matrix and a Pw matrix, which are respectively formed by three values of the sphere center in the camera coordinate system (reference coordinate system) and values thereof in the world coordinate system, are subjected to matrix multiplication to obtain a rotation matrix, and the Pc matrix of course needs inverse operation. Further, the value of Pw is known (radius of the sphere + length of the rod), and the value of Pc is generated by coordinates of three spherical centers fitted by a spherical point cloud. After the rotation matrix exists, the point coordinates measured by the camera can be corrected, so that the point coordinates can be compared with an actual spraying room, and further collision detection of the workpiece is realized.

And S8, setting parameters of a world coordinate system to determine triggering conditions of collision early warning.

In a preferred embodiment, the method further comprises the following steps:

s9, after the reference coordinate system is successfully fitted, the reference coordinate system is successfully established for the collision detection system, and at the moment, the entity coordinate system arranged in the area to be detected can be removed.

In a preferred embodiment, the physical coordinate system comprises three spheres forming the end points of the X, Y and Z axes of the physical coordinate system, respectively, and the spheres are the captured object of the point cloud. The surface of the sphere is polished or drawn, so that the camera device can acquire and generate point cloud.

As shown in fig. 2, the embodiment further provides a spraying system, which includes a spraying room, a chain conveying mechanism and a camera device, wherein the camera device is installed at two sides in front of the spraying room, the chain conveying mechanism is located at the top of the spraying room, the chain conveying mechanism is used for conveying workpieces into and out of the spraying room, and before the workpieces enter the spraying room, the camera device detects the shaking amount of the workpieces relative to the spraying room by the above workpiece collision detection method based on image registration;

and when the shaking amount of the workpiece exceeds a preset value, stopping the chain conveying mechanism or giving out collision early warning.

As shown in fig. 2, in the preferred embodiment, a collision detection area and a safety area are arranged in the detection area covered by the visual field of the camera device, the collision detection area corresponds to the boundary of the spraying room, the safety area corresponds to the moving area of the workpiece, and the non-detection area is arranged outside the collision detection area. In this embodiment, the alarm range is set as follows: and after the two cameras are respectively arranged and fixed according to the mode, setting the world coordinate range of collision detection according to the acceptable safety margin of the field collision detection. In order to prevent people or other articles beside the camera from entering the detection range of the camera to trigger collision early warning by mistake, three areas, namely a green safety area, a red collision detection area and a gray non-detection area, are set in the coverage range of the camera. The red range is the collision detection zone (spray booth boundary); the area inside the red inner frame is a green safety area (workpiece area); the area outside the red frame is a gray non-detection area (area outside the border of the spray booth). In order to prevent the chain and the hanging tool from passing through to generate false alarm, a rectangular opening area is designed at the top end of the red collision detection area, the opening range is determined by 4 parameters of UYL, UYH, UZL and UZH, and the X range is the same as the red collision detection area.

Compared with the prior art, the invention has the following beneficial effects: the invention utilizes the characteristic of wide working range of the structured light depth camera to carry out real-time collision detection on the whole range from the side, and utilizes the algorithm to realize the graded alarm of different areas, so that the collision detection or the protection of the spraying equipment has better pertinence; furthermore, the potential of a camera device (a depth camera) in the aspect of collision detection is excavated, (the quality of point cloud obtained by the surface of a metal wire drawing ball in a visual field range of a consumer-grade structured light camera is found to meet the requirement of a spherical fitting algorithm), a complete set of algorithm is developed, the positioning problem of the depth camera is successfully solved, a matrix construction calculation method of space coordinate conversion is solved in detail, and engineering application is realized on the ground; compared with the common grating and line laser scanning which can only work on the working plane arranged on the laser scanning device, the invention can realize large-scale real-time detection without being limited to a specific working plane.

In this specification, unless explicitly stated or limited otherwise, a first feature may be "on" or "under" a second feature such that the first and second features are in direct contact, or the first and second features are in indirect contact via an intermediate. Also, a first feature "on," "over," and "above" a second feature may be directly or diagonally above the second feature, or may simply indicate that the first feature is at a higher level than the second feature. A first feature being "under," "below," and "beneath" a second feature may be directly under or obliquely under the first feature, or may simply mean that the first feature is at a lesser elevation than the second feature.

In the description herein, reference to the description of the terms "preferred embodiment," "yet another embodiment," "other embodiments," or "specific examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.

Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.