阅读说明：本技术 机器人夹持下的导管测量方法 (Method for measuring catheter clamped by robot ) 是由 陈月玲 夏仁波 赵吉宾 赵亮 付生鹏 刘明洋 于彦凤 于 2019-10-14 设计创作，主要内容包括：本发明涉及一种机器人夹持下的导管测量方法。其步骤为：同步触发多目视觉测量箱内的多个工业相机采集不同视角下的多幅原始图像,借助于过度分割边缘、组合平行边缘对以及剔除错误边缘等方法去除测量过程中夹爪的干扰,然后对边缘图像进行骨架提取得到弯管图像的中轴线,借助于图像分割将中轴线分割成直线段和椭圆段区域,最后利用极线约束对控制点进行三维重建,扫掠得到弯管的三维模型。本发明方法能实现在机器人夹持下的导管测量,能够自动去除夹爪的干扰,有效的重建管路的三维模型,得到的精确的控制点,鲁棒性强、精度高、效率高。(The invention relates to a method for measuring a catheter clamped by a robot. The method comprises the following steps: the method comprises the steps of synchronously triggering a plurality of industrial cameras in a multi-view vision measurement box to collect a plurality of original images at different viewing angles, removing interference of clamping jaws in the measurement process by means of methods of excessively segmenting edges, combining parallel edge pairs, removing wrong edges and the like, then carrying out skeleton extraction on edge images to obtain a central axis of a bent pipe image, segmenting the central axis into straight line sections and elliptical section areas by means of image segmentation, finally carrying out three-dimensional reconstruction on control points by means of polar constraint, and scanning to obtain a three-dimensional model of the bent pipe. The method can realize the measurement of the catheter clamped by the robot, can automatically remove the interference of the clamping jaw, effectively rebuilds the three-dimensional model of the pipeline, and obtains accurate control points with strong robustness, high precision and high efficiency.)

1. The method for measuring the catheter clamped by the robot is characterized by comprising the following steps:

step 1: synchronously triggering a plurality of industrial cameras in the multi-view vision measuring box;

step 2: the method comprises the following steps that an industrial camera collects a plurality of original images at different viewing angles;

and step 3: processing each original image as follows to obtain a central axis of the divided bent pipe;

i. extracting an edge image and preprocessing:

removing the clamping component on the image to obtain a bent pipe outline area only containing the bent pipe:

extracting a central axis of the elbow profile region:

iii, dividing the central axis of the bent pipe into a straight section and an elliptical section;

and 4, step 4: selecting control points, obtaining internal and external parameters of the industrial camera through global calibration, and matching the control points by means of polar line constraint to obtain spatial three-dimensional control points of the bent pipe;

and 5: and connecting adjacent space three-dimensional control points of the bent pipe to form a bent pipe space three-dimensional central axis, and sweeping by taking the bent pipe central axis as a path to obtain a three-dimensional catheter model.

2. The method as claimed in claim 1, wherein the multi-view vision measuring box comprises a plurality of industrial cameras at different positions, a backlight source, a light homogenizing plate and a box body, wherein the industrial cameras are uniformly arranged above the inner wall of the box body, and the backlight source and the light homogenizing plate are arranged at the bottom of the box body.

3. The method of claim 2, wherein the industrial camera is positioned such that its measurement range covers the entire field of view of the backlight source.

4. The method of claim 1, wherein the extracting and pre-processing an edge image comprises:

performing edge sub-pixel extraction on the image by using a Canny operator to obtain an edge image containing a clamping part and a bent pipe;

removing edge pixels with the contour length smaller than a threshold value;

and eliminating edge interference pixels in the image by using the symmetrical form of the catheter along the central axis.

5. The method of claim 1, wherein removing the clamp member from the image to obtain an elbow contour region including only elbows comprises:

combining parallel edge pixels in the image, and then eliminating the edge contour of which the distance between the parallel edges is greater than a threshold value to obtain a bent pipe contour region only containing a bent pipe.

6. The method of claim 1, wherein extracting a central axis of an elbow profile region comprises:

filling the obtained bent pipe outline area, and removing a noise area in the foreground image by adopting morphological open operation;

and carrying out skeleton extraction on the edge area of the bent pipe outline to obtain the central axis of the bent pipe outline area.

7. The method of claim 1, wherein the skeleton extraction comprises: and taking pixel points equidistant from two points on the boundaries at the two sides as pixel points of the central axis, and obtaining the skeleton for image refinement.

8. The method of claim 1, wherein the selected control points are: and using the intersection point of two adjacent straight line segments as a control point to be matched in the two images.

Technical Field

The invention relates to the technical field of computer vision, in particular to a method for measuring a catheter clamped by a robot.

Background

Today, the automation level of industrial robots is remarkably improved and the industrial robots gradually become the future trend of automatic production. However, the automatic production of the catheter is realized, and the automatic measurement of the catheter cannot be separated. The automatic measurement of the guide pipe needs to be completed under the assistance of an industrial robot, and then the requirements of a series of intelligent guide pipe manufacturing such as automatic guide pipe cutting, welding and assembling can be met.

At present, the measurement modes of the formed bent pipe include a contact measurement mode and a non-contact measurement mode, wherein the contact measurement mode, such as a mode method, a three-coordinate measuring machine, an articulated coordinate measuring machine and the like, has the defects that a comparison device is difficult to manufacture, is easy to damage or deform, has low efficiency, excessively depends on subjective judgment of an operator, cannot reconstruct a three-dimensional model, has no positioning reference and the like. Non-contact measurement methods, such as raster scanning, multi-view vision measurement scanning, etc., which are highly accurate, cannot be applied to an automated production line and cannot perform reference positioning of a catheter.

Therefore, the situation that the traditional manual piping is low in efficiency and reliability needs to be changed, and the digitalization and lean level of automatic assembly is effectively improved.

How to effectively shorten the development time, reduce the development cost and improve the product quality is the main purpose of fast processing and manufacturing the bent pipe.

Disclosure of Invention

Aiming at the defects in the prior art, the technical problem to be solved by the invention is to provide the method for measuring the catheter clamped by the robot, the method effectively utilizes a multi-view vision processing algorithm, eliminates the interference of a clamping jaw of the robot, realizes the automatic production of the catheter, effectively shortens the development time, reduces the development cost and improves the product quality.

The technical scheme adopted by the invention for realizing the purpose is as follows: the method for measuring the catheter clamped by the robot comprises the following steps:

step 1: synchronously triggering a plurality of industrial cameras in the multi-view vision measuring box;

step 2: the method comprises the following steps that an industrial camera collects a plurality of original images at different viewing angles;

and step 3: processing each original image as follows to obtain a central axis of the divided bent pipe;

i. extracting an edge image and preprocessing:

removing the clamping component on the image to obtain a bent pipe outline area only containing the bent pipe:

extracting a central axis of the elbow profile region:

iii, dividing the central axis of the bent pipe into a straight section and an elliptical section;

and 4, step 4: selecting control points, obtaining internal and external parameters of the industrial camera through global calibration, and matching the control points by means of polar line constraint to obtain spatial three-dimensional control points of the bent pipe;

and 5: and connecting adjacent space three-dimensional control points of the bent pipe to form a bent pipe space three-dimensional central axis, and sweeping by taking the bent pipe central axis as a path to obtain a three-dimensional catheter model.

The multi-view vision measuring box is composed of a plurality of industrial cameras, a backlight source, a light homogenizing plate and a box body, wherein the industrial cameras are located at different positions, the industrial cameras are evenly arranged above the inner wall of the box body, and the backlight source and the light homogenizing plate are arranged at the bottom in the box body.

The industrial camera is arranged in a position such that its measuring range covers the entire field of view of the backlight light source.

The extracting and preprocessing of the edge image comprises:

performing edge sub-pixel extraction on the image by using a Canny operator to obtain an edge image containing a clamping part and a bent pipe;

removing edge pixels with the contour length smaller than a threshold value;

and eliminating edge interference pixels in the image by using the symmetrical form of the catheter along the central axis.

The removing the clamping component on the image to obtain the elbow contour region only containing the elbow comprises the following steps:

combining parallel edge pixels in the image, and then eliminating the edge contour of which the distance between the parallel edges is greater than a threshold value to obtain a bent pipe contour region only containing a bent pipe.

The central axis of the extracted elbow profile region includes:

filling the obtained bent pipe outline area, and removing a noise area in the foreground image by adopting morphological open operation;

and carrying out skeleton extraction on the edge area of the bent pipe outline to obtain the central axis of the bent pipe outline area.

The skeleton extraction comprises: and taking pixel points equidistant from two points on the boundaries at the two sides as pixel points of the central axis, and obtaining the skeleton for image refinement.

The selected control points are as follows: and using the intersection point of two adjacent straight line segments as a control point to be matched in the two images.

The invention has the following beneficial effects and advantages:

1. the method for measuring the catheter clamped by the robot can effectively reconstruct the three-dimensional model of the catheter.

2. The method for measuring the catheter clamped by the robot can eliminate the interference of the clamping jaw and effectively obtain the catheter control point.

3. The method for measuring the catheter clamped by the robot can be effectively applied to an automatic production line of pipelines, and the intelligent manufacturing process of the catheter is promoted.

Drawings

FIG. 1 is a flow chart of the method of the present invention;

FIG. 2(a) is a first original image clamped by a robot in the method of the present invention;

FIG. 2(b) is a central axis image of the original image held by the robot according to the method of the present invention after processing;

FIG. 3(a) is a second original image of a robot under clamping in the method of the present invention;

FIG. 3(b) is a central axis image of the original image II after processing, which is clamped by the robot in the method of the present invention;

FIG. 4(a) is a third original image of a robot under clamping in the method of the present invention;

FIG. 4(b) is a central axis image after three processes of an original image clamped by a robot in the method of the present invention;

fig. 5 is a three-dimensional reconstruction of a catheter used in the method of the invention.

Detailed Description

The present invention will be described in further detail with reference to the accompanying drawings and examples.

Referring to the attached drawing 1, the method for three-dimensional reconstruction of a pipeline by combining topological relation and epipolar constraint comprises the following specific steps:

step 1: the bent elbow is automatically clamped by the robot and sent into the multi-view vision measuring box.

The multi-view vision measuring box is composed of eight industrial cameras, a backlight source, a light homogenizing plate and a closed box body, wherein the industrial cameras are located at the top of the interior of the box body, and the backlight source and the light homogenizing plate are located at the bottom of the interior of the box body. The layout of the eight industrial cameras ensures that the measuring range can cover the whole view field of the backlight light source, and no dead angle is ensured.

Step 2: synchronously triggering a plurality of industrial cameras to collect original images at different viewing angles;

and step 3: processing each bent pipe image according to the following steps 3.1 to 3.7 to obtain a central axis of the divided bent pipe;

i. edge image extraction and preprocessing (steps 3.1-3.3):

step 3.1: performing edge sub-pixel extraction on the image by using a Canny operator to obtain an edge image containing a clamping part and a bent pipe;

step 3.2: filtering an interference edge of the bent pipe on the edge image, namely removing an edge with the contour length smaller than threshold, wherein in an experiment, the threshold is 80;

step 3.3: because the edge region of the catheter is close to symmetry, the contour without the symmetric region is removed before segmentation (namely, the interference pixels in the image are further removed by utilizing the symmetric form of the catheter along the central axis);

removing the clamping component on the image to obtain a bent pipe outline area only containing the bent pipe:

step 3.4: since the gripper components appear as two profiled rectangular blocks on the image, i.e. their edges appear as parallel edges in the image, combining parallel edge pixels in the image and then removing the edge profile where the distance of the parallel edges is greater than the threshold value threshold1 (the pixel threshold value of the profiled rectangular block of the gripper mechanism is threshold 1-100), a curved tube profile region containing only curved tubes is obtained;

extracting a central axis of the elbow profile region (steps 3.5-3.6):

step 3.5: and filling the obtained bent pipe outline area, and removing a noise area in the foreground image by using morphological open operation. The noise refers to the fact that the image is often influenced by interference of the imaging device and external environment noise and the like in the digitization and transmission processes.

Step 3.6: performing skeleton extraction on the edge area of the bent pipe outline to obtain the central axis of the bent pipe outline area; the skeleton extraction comprises: and taking pixel points equidistant from two points on the boundaries at the two sides as pixel points of the central axis, and obtaining the skeleton for image refinement.

iii, dividing the central axis of the bent pipe into a straight section and an elliptical section;

step 3.7: dividing the central axis of the bent pipe into a straight line section and an elliptical section; the method comprises the following steps: the central axis of the bent pipe is approximately represented by an excessively divided polygonal curve, and each side of the polygonal curve is traversed; if the error of the fitting straight line segment is smaller than the error of the fitting elliptical segment, the line segments adjacent to each other are merged into a straight line segment; if the error of the fitted ellipse is smaller than the error of the fitted straight line, the line segments adjacent to each other are merged into an elliptical arc; all edges of the polygon curve are iteratively processed until there are no straight line segments that can be merged.

And 4, step 4: and using the intersection point of two adjacent straight line segments as a control point to be matched in the two images, obtaining internal and external parameters of the industrial camera through global calibration, and matching the control point by means of epipolar constraint to obtain a spatial three-dimensional control point of the catheter. Fig. 2(a), 3(a) and 4(a) show three original images held by a robot in the method of the present invention, and fig. 2((b)), 3(b) and 4(b) show three images after the original images are processed in the method of the present invention.

The internal and external parameters comprise:

the intrinsic parameters represent the transformation from the industrial camera coordinate system to the image plane coordinate system, which is expressed in pixel units; the distortion degree of each lens is different, including radial distortion and eccentric distortion; calibrating the internal parameters of each industrial camera by using a two-dimensional flexible target;

the extrinsic parameters represent a transformation matrix from the industrial camera coordinate system to the world coordinate system, which is achieved by a rotation matrix R and a translation matrix T.

And 5: and connecting adjacent space three-dimensional control points of the bent pipe to form a bent pipe space three-dimensional central axis, and sweeping by taking the bent pipe central axis as a path to obtain a three-dimensional catheter model. Fig. 5 shows a three-dimensional reconstruction of a catheter used in the method of the invention.

The sweep is a modeling method for converting a two-dimensional figure into a three-dimensional figure, and is a three-dimensional figure formed by using a two-dimensional body object as a cross section along a certain path (for example, the central axis direction of a bent pipe).

The invention can measure various pipelines, the measuring precision can reach 0.15mm, and the measuring time efficiency is 2 s. The measuring accuracy is high, and is efficient, has overcome the interference of robot clamping jaw, has stronger stability.