阅读说明：本技术 一种基于机器视觉的工件装配过程的轴孔对中引导方法 (Machine vision-based shaft hole centering guide method for workpiece assembly process ) 是由 刘志峰 刘康 许静静 杨聪彬 王建华 李龙飞 陈建州 于 2021-08-29 设计创作，主要内容包括：本发明公开了一种基于机器视觉的工件装配过程的轴孔对中引导方法,通过工业相机对待检测工件进行拍摄,获取工件在检测区域中的图像。通过待检测工件的3d对象模型来创建用于匹配的3d形状模型。利用虚拟相机在指定的姿态范围内对3d对象模型进行拍摄,通过不同视角下的图像生成3d形状模型。采用贪婪算法在指定的图像金字塔范围内进行检索,找到最合适匹配对象,并得到匹配对象的姿态,通过对姿态的分析,调整机器人的末端进行正确的轴孔对中引导操作。(The invention discloses a machine vision-based shaft hole centering guide method for a workpiece assembly process. And creating a 3d shape model for matching through the 3d object model of the workpiece to be detected. And shooting the 3d object model in a specified posture range by using a virtual camera, and generating a 3d shape model through images at different view angles. Searching in the range of the appointed image pyramid by a greedy algorithm, finding the most appropriate matching object, obtaining the posture of the matching object, and adjusting the tail end of the robot to perform correct shaft hole centering guide operation through posture analysis.)

1. A shaft hole centering guide method for a workpiece assembly process based on machine vision is characterized by comprising the following steps: the method comprises the following steps:

s1: fixing the calibrated camera to enable the workpiece to be assembled to appear in the visual field of the camera, and installing a light source to illuminate the area where the workpiece is located;

s2: adjusting a camera to a proper shooting height, finely adjusting the camera and a light source to obtain a clear image of a workpiece, shooting the clear image of the workpiece to be assembled by the camera, processing the acquired image by using a basic image preprocessing algorithm, and obtaining depth information of the target workpiece in the image according to a binocular camera imaging model;

s3: acquiring a 3d data model of a workpiece to be assembled, and storing the 3d data model in a dxf data type; reading the model, shooting at different positions around the 3d object model by adopting a virtual camera technology, acquiring plane views of the object at different viewing angles, and obtaining a shape model for matching by fusing different views; in order to better extract key features of the image, a Gaussian pyramid is used for carrying out zooming processing on the image, a target workpiece in the image is found through local decision based on a greedy decision algorithm, and the pose [ x, y, z, alpha, beta, gamma ] of the workpiece relative to an initial coordinate system of the workpiece is returned; and determining the conversion required by the target posture reached by the tail end of the robot through the returned posture information and the current posture information output by the tail end of the robot.

2. The machine vision-based shaft hole centering guide method for the workpiece assembling process, according to claim 1, is characterized in that: in S1, the camera is calibrated by using a "zhangzhengyou calibration method" to obtain key parameters of the camera model.

3. The machine vision-based shaft hole centering guide method for the workpiece assembling process, according to claim 1, is characterized in that: in the step S2, the image preprocessing algorithm including, but not limited to, operations such as noise filtering, image edge enhancement, and gray level normalization is used to improve the quality of the image features, so as to reduce the phenomenon that the image key features are identified incorrectly and the target object is difficult to identify due to bright spots locally appearing in the image due to reflection and highlight.

4. The machine vision-based shaft hole centering guide method for the workpiece assembling process, according to claim 1, is characterized in that: in S2, the depth information in the image is acquired by processing the image captured by the binocular camera through the acquired left and right images:

d＝f*b/d

where f is the camera focal length, b is the left and right camera baseline, and d is the parallax.

5. The machine vision-based shaft hole guiding and centering method for the workpiece assembling process, as recited in claim 1, wherein: in the step S3, the 3d object model is photographed by using the virtual camera in the designated posture range, so as to obtain planar views of the 3d object model at different viewing angles, and the 3d shape model is obtained and stored through the views at different viewing angles.

6. The machine vision-based shaft hole centering guide method for the workpiece assembling process, according to claim 1, is characterized in that: in S3, in order to better extract different scale features of the captured target workpiece image, and at the same time, in order to improve the speed and calculation speed for retrieving and matching the target workpiece in the image, it is generally necessary to perform scaling processing on the image by using a gaussian pyramid, and the image information included in the image pyramid is decreased with the increase of the pyramid; the calculation process of the gaussian pyramid can be expressed as follows:

G_i＝Down(G_i-1)

wherein G is_iRepresenting the image obtained by the i-th sampling, G_i-1The image obtained by sampling at the (i-1) th time is shown, and the Down shows the Down sampling process of the image.

7. The method for centering the guide shaft hole in the workpiece assembling process based on the machine vision is characterized in that: in the step S3, because the task of searching the target in the image for matching is performed, the overall optimal effect is achieved by selecting a series of local optima; and searching and matching the model in the image according to the greedy algorithm and the Gaussian pyramid layering method and according to the 3d shape model data of the target workpiece, and searching an optimal model matching result.

8. The method for centering the guide shaft hole in the workpiece assembling process based on the machine vision is characterized in that: and S3, after the target workpiece is identified and matched, acquiring attitude information of the target, comparing the attitude information with the tail end attitude information of the robot, adjusting the tail end attitude of the robot, and realizing the centering operation of the center of the workpiece clamped by the tail end of the robot and the central shaft of the workpiece to be assembled in the assembling process.

9. The method for centering the guide shaft hole in the workpiece assembling process based on the machine vision is characterized in that: in S3, the pose information returned and the pose information currently output by the robot end determine the transformation data required by the target pose reached by the robot end through the sufficient requirement that the two vectors are parallel to each other.

Technical Field

The invention relates to a detection method combining machine vision and image recognition, which is applied to workpiece matching recognition in a three-dimensional space, is suitable for shaft hole centering guide operation of various workpieces in an automatic production process, and particularly relates to a shaft hole centering method based on machine vision in a workpiece assembly process.

Background

Under the background that the application of intellectualization and automation in the industrial production process is gradually improved, more and more factories begin to develop industrial robot production lines, more production processes begin to be completed by robots, and the automatic assembly process is included. In the process of workpiece assembly, the shaft hole centering link is the basis of numerous assembly operations, the assembly operation can be smoothly completed only by correctly realizing the shaft hole centering, the assembly precision can be ensured, and meanwhile, the damage of the assembly workpiece is avoided, so that the waste of production resources and the increase of production cost are caused. However, in the current automated assembly process, the robot can only face a fixed scene under the condition of assembly according to a given working path, and once the position or the posture of a workpiece is changed, the assembly process cannot be effectively completed. This results in that the task of the pre-selection setting is not well completed once the working scene is changed when the industrial robot is currently applied.

Therefore, in order to improve the adaptability of the industrial robot to the change of the position or the posture of the workpiece in the scene in the assembling and applying process, a high-efficiency, accurate, practical and low-cost method for guiding the target workpiece matching identification and the shaft hole centering in the assembling process is urgently needed, so that the perception of the robot to the scene change and the application rate of the robot in the production field are improved, and the application potential of the industrial robot is maximized as far as possible.

Disclosure of Invention

The invention aims to: aiming at the shaft hole centering process in the existing automatic assembly production, in order to improve the adaptability of a robot to the position or posture change of a workpiece in the environment in the assembly operation process, a centering guide method based on machine vision in the assembly process is provided.

In order to solve the technical problem described above, the present invention provides a method for guiding a tip to align with a workpiece in a vision-based robot application process, which is implemented by the following procedures:

1. obtaining a model diagram of a matched workpiece to be identified as a 3d object model;

2. calibrating a binocular industrial camera by using a calibration plate calibration method, acquiring key parameters and distortion coefficients of a camera model, starting the camera to photograph a scene containing a target workpiece, and acquiring a scene image;

3. processing an image shot by a binocular camera to obtain depth information in the image;

4. due to the influence of ambient light and the used light source, some parts of the acquired image are often too dark or too bright, which brings trouble to the subsequent processing of the image, so a series of image processing operations must be performed on the image.

5. The 3d object model is photographed in a specified posture range by using a virtual camera to obtain plane views of the 3d object model under different visual angles, and the 3d shape model is obtained and stored through the views under different visual angles;

6. in order to increase the speed of searching for a matching model in an image, generally, the image is subjected to scaling processing, a gaussian pyramid is adopted to process the image, and image information contained in the image pyramid is decreased with the increase of the pyramid;

7. and searching and matching the model in the image according to the 3d shape model data according to a greedy algorithm and a Gaussian pyramid layering method, and searching an optimal model matching result. The task of searching for a target in an image for matching can achieve the overall optimal effect by selecting a series of local optima.

8. After the target workpiece is identified and matched, acquiring attitude information of the target, comparing and calculating the attitude information with real-time attitude information of the tail end of the robot at the moment, adjusting the attitude of the tail end of the robot, and realizing centering operation of the center of the workpiece clamped by the tail end of the robot and the central shaft of the workpiece to be assembled in the assembling process;

drawings

FIG. 1 is a flow chart of an implementation of the present invention.

Fig. 2 is a scale space schematic diagram of an image pyramid.

FIG. 3 is a schematic diagram of greedy algorithm based model matching.

Fig. 4 is a schematic illustration of the present invention in guiding centering.

Description of the invention

The shaft hole centering method based on the machine vision in the workpiece assembling process has the following advantages:

1. in the industrial assembly process, shaft hole centering operation is a precondition for a plurality of operations, and the method can greatly improve the efficiency of the automatic assembly process

2. The image processing process is simple and efficient, the calculation speed is high, the overall calculation efficiency is high, and the algorithm complexity is low.

3. The method has strong portability, can be used for detecting and identifying objects, can also be used for matching targets, and is used for realizing the guide process of shaft hole centering operation mentioned in the text;

additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.

Detailed Description

The following are specific embodiments of the present invention and are further described with reference to the accompanying drawings, but the present invention is not limited to these embodiments.

It should also be understood that the specific embodiments described herein are for the purpose of illustration only and are not intended to limit the invention.

The target image processed by the invention originates from an industrial camera of an industrial robot, which is used for detecting workpieces on a production line.

As shown in fig. 1, the step-by-step operation of the workpiece inspection method based on vision provided by the present invention is as follows:

step 1, taking a model diagram of a bolt (M12) to be identified and matched (assembled) as a 3d object model, and when the bolt model is created, enabling a bolt central axis to coincide with a z1 axis of a model coordinate system;

based on the Zhang calibration method principle, using a circular calibration plate special for Halcon, and adopting a calibration algorithm to calibrate the binocular industrial camera to obtain key parameters (the focal length, distortion coefficient, the number of deselected row pixels and column pixels and the original size of the image of the left camera and the right camera) of a camera model;

step 3, the moving camera shoots scenes connected by the bolts to obtain scene images of the bolts;

and 4, due to the influence of the ambient light and the used light source, the situation that some parts in the acquired image are often too dark or too bright is often generated, which brings some troubles to the subsequent processing of the image, so a series of image processing operations must be performed on the image. The method mainly comprises filtering noise reduction, image edge enhancement, gray level equalization and the like, and the imaging quality of an image can be obviously improved through image processing, so that the efficiency and the accuracy in the subsequent processing process are improved;

and 5, processing the images shot by the binocular camera through the collected left and right images to obtain depth information in the images:

d＝f*b/d

wherein f is the focal length of the camera, b is the base line of the left camera and the right camera, and d is the parallax;

step 6, photographing the 3d object model of the bolt in the designated posture range by using a virtual camera to obtain plane views of the bolt model under different visual angles, and generating a 3d shape model of the bolt for serving as a model of a search object in an image through the views under different visual angles;

and 7, in order to better extract different scale features of the shot bolt image and improve the speed of searching the matching target bolt in the image and the calculation speed, generally, the image needs to be zoomed by adopting a Gaussian pyramid, and the image information contained in the image pyramid is decreased with the increase of the pyramid. The calculation process of the gaussian pyramid can be expressed as follows:

G_i＝Down(G_i-1)

wherein G is_iRepresenting the image obtained by the i-th sampling, G_i-1Indicates the i-1 th samplingSampling the obtained image, wherein Down represents the Down sampling process of the image;

and 8, due to the task of searching for the target in the image for matching, the overall optimal effect can be achieved by selecting a series of local optima. Therefore, model searching and matching can be carried out in the image according to the 3d shape model data of the bolt according to a greedy algorithm and a Gaussian pyramid layering method, and an optimal model matching result is searched;

step 9, finding a target bolt in the image, and after completing the identification and matching of the target workpiece, obtaining the attitude information [ x, y, z, alpha, beta, gamma ] of the target at the moment: the data is attitude information of a current bolt attitude relative to a bolt 3d model coordinate system, because an initial coordinate system of a bolt model is the same as a robot base coordinate system, the data can be compared and calculated with robot tail end attitude information at the moment, the tail end attitude of the robot is adjusted, a z1 axis at the tail end of a robot tool is enabled to be parallel to a z2 axis of a target bolt, the tail end of the robot is moved through a positioning algorithm, and a z1 axis is enabled to coincide with a z2 axis, so that the centering operation of the center of a workpiece clamped by the tail end of the robot and the central axis of the workpiece to be assembled in the assembling process is realized, and the specific realization process is as follows:

9.1 taking the Z-axis vector of the bolt coordinate system B-XYZAfter the model is matched with the solid bolt, the posture of the bolt solid B' -XYZ relative to B-XYZ is [ alpha, beta, gamma ]]From this can be obtained

Wherein:

9.2 get the vector on the central axis of the tail end of the robotWhereinR₂The pose information of the end of the robot can be known, and the calculation mode and the R are obtained₁The same is true. The center shaft of the tool at the tail end of the robot is guided to be centered with the center hole of the bolt by adjusting the posture of the tail end of the robot, and the method comprises the following steps:

9.3 by inverse solving the matrix R, the relevant angle information (α ', β ', γ ') can be obtained, i.e. the angle and direction of the posture of the guiding end central axis and the bolt central axis to be adjusted are realized.