阅读说明：本技术 基于视频图像的液压油缸尺寸测量和位姿估计方法及系统 (Hydraulic oil cylinder size measurement and pose estimation method and system based on video image ) 是由 王敏 陆文杰 王静元 陈柯 于 2020-10-13 设计创作，主要内容包括：本发明公开了基于视频图像的液压油缸尺寸测量和位姿估计方法及系统,属于工业自动化领域。本发明提出的方法及系统包括：(1)使用包含发光面和摄像机的成像装置对液压油缸进行连续视频拍摄；(2)使用带自旋装置的运输线使液压油缸通过成像装置时进行自转；(3)使用运行于工控机中的视频图像识别算法对包含液压油缸的高对比度视频进行检测,识别并测量液压缸筒、缸盖、活塞杆、液压锁等各部件；(4)基于视频图像中液压油缸位置的连续变化,对液压油缸的移动速度进行评估；(5)基于液压缸筒侧面的液压锁尺寸,对液压油缸的位姿进行评估。本发明获得液压油缸的尺寸、位姿和速度信息后,将进一步用于液压油缸的清洗、喷涂、抓取等操作。(The invention discloses a method and a system for measuring the size and estimating the pose of a hydraulic oil cylinder based on a video image, belonging to the field of industrial automation. The method and the system provided by the invention comprise the following steps: (1) using an imaging device comprising a light emitting surface and a camera to carry out continuous video shooting on the hydraulic oil cylinder; (2) a transport line with a self-rotating device is used for enabling the hydraulic oil cylinder to rotate when passing through the imaging device; (3) detecting a high-contrast video containing a hydraulic oil cylinder by using a video image recognition algorithm running in an industrial personal computer, and recognizing and measuring each part such as a hydraulic cylinder barrel, a cylinder cover, a piston rod, a hydraulic lock and the like; (4) evaluating the moving speed of the hydraulic oil cylinder based on the continuous change of the position of the hydraulic oil cylinder in the video image; (5) and evaluating the pose of the hydraulic oil cylinder based on the size of a hydraulic lock on the side surface of the hydraulic cylinder barrel. After the size, pose and speed information of the hydraulic oil cylinder is obtained, the hydraulic oil cylinder is further used for operations of cleaning, spraying, grabbing and the like of the hydraulic oil cylinder.)

1. A hydraulic oil cylinder size measurement and pose estimation method based on a video image is characterized by comprising the following steps:

s1, conveying by a conveying line: the hydraulic oil cylinder is conveyed between the two working procedure positions by using a conveying line, and the hydraulic oil cylinder rotates when passing through the imaging device;

s2, continuous imaging: using an imaging device to continuously shoot the hydraulic oil cylinder and transmitting the obtained video image to an industrial personal computer for processing

S3, image measurement: detecting the image sent by the imaging device by using a video image recognition algorithm in an industrial personal computer, and recognizing and measuring each part of the hydraulic cylinder barrel, the cylinder cover, the piston rod and the hydraulic lock

S4, speed evaluation: measuring the position of a hydraulic cylinder in a continuous multi-frame video image by using a video image recognition algorithm in an industrial personal computer, and calculating the moving speed of the hydraulic cylinder

S5, pose evaluation: and (3) measuring the visible size of the hydraulic lock image on the side surface of the hydraulic cylinder barrel in real time by using a video image recognition algorithm in an industrial personal computer, calculating the self-rotation angle of the hydraulic cylinder barrel, and evaluating the pose of the hydraulic cylinder barrel.

2. The method for measuring the size and pose of the hydraulic cylinder based on the video image as claimed in claim 1, wherein the video image recognition algorithm in the step S3 comprises the following steps:

s31, extracting the outline of the hydraulic oil cylinder in the video image,

s32, distinguishing and positioning the outline of the hydraulic oil cylinder based on the semantic positions and the geometric shapes of the hydraulic cylinder barrel, the cylinder cover, the piston rod and the hydraulic lock,

and S33, measuring the sizes of the hydraulic oil cylinder and the hydraulic cylinder barrel, the cylinder cover, the piston rod and the hydraulic lock of the hydraulic oil cylinder based on the camera calibration information.

3. The video-image-based hydraulic cylinder dimension measurement and pose estimation method according to claim 1, wherein the evaluation of the moving speed of the hydraulic cylinder in step S4 comprises the steps of:

s41, calculating the pixel displacement of the hydraulic oil cylinder in the continuous image frame of the video,

s42, converting the pixel displacement into the actual moving distance according to the camera calibration information,

and S43, dividing the time difference by the corresponding time difference of the continuous image frames to obtain the moving speed.

4. The video-image-based hydraulic cylinder dimension measurement and pose estimation method according to claim 1, wherein the estimation of the hydraulic cylinder pose in the step S5 is to measure a rotation angle of the hydraulic cylinder on a transport line, comprising the steps of:

s51, detecting that the visible size of the hydraulic lock image on the side of the hydraulic cylinder barrel is maximized,

s52, identifying that the hydraulic lock is at the left side or the right side of the hydraulic cylinder at the moment,

s53, the left side determines that the self-rotation angle of the hydraulic oil cylinder is 0 degree at the moment, the right side determines that the self-rotation angle is 180 degrees,

and S54, calculating the self-rotation angle of the hydraulic oil cylinder at any time.

5. A system for realizing the method for measuring the size and the pose of the hydraulic oil cylinder based on the video image according to any one of claims 1 to 4, wherein the method comprises the following steps: the device comprises a conveying line, an imaging device and an industrial personal computer, wherein the conveying line is a chain type conveying line, clamping devices for fixing hydraulic oil cylinders are hung on the conveying line at equal intervals, the clamping devices are connected with cylinder covers of the hydraulic oil cylinders, gears are arranged at the tail ends of the clamping devices and rotate around the hung central axes, and tooth grooves or powered chains are arranged on the conveying line and drive the gears to enable the hung hydraulic oil cylinders to rotate;

the imaging device is composed of a light-emitting background plate and an industrial camera, the height of the light-emitting background plate is larger than that of the hydraulic oil cylinder in a suspension state, and the width of the light-emitting background plate is larger than the moving distance of the hydraulic oil cylinder on a conveying line in the time required by one rotation circle of the hydraulic oil cylinder.

The industrial personal computer detects images sent by the imaging device by using a video image recognition algorithm, recognizes and measures all parts of the hydraulic cylinder barrel, the cylinder cover, the piston rod and the hydraulic lock, and the visual field of the industrial camera covers the whole luminous background plate.

Technical Field

The invention relates to the technical field of industrial automation, in particular to a method and a system for measuring the size and estimating the pose of a hydraulic oil cylinder based on a video image.

Background

The hydraulic oil cylinder is a hydraulic actuator which converts hydraulic energy into mechanical energy and makes linear reciprocating motion (or swinging motion). It has simple structure and reliable operation. When it is used to implement reciprocating motion, it can omit speed-reducing device, and has no transmission gap, and its motion is stable, so that it can be extensively used in various mechanical hydraulic systems.

The hydraulic cylinder is widely applied and has high demand, but in the production and manufacturing process, various working procedures such as assembly, welding, spraying and the like are needed, but as the cylinder barrel and the piston rod are made of iron, the weight of the cylinder barrel and the piston rod is different from 10 kilograms to hundreds of kilograms, so that a great deal of difficulty and invariance are caused to production workers in the grabbing and carrying process. Therefore, it is necessary to apply related automation equipment or industrial robots to improve production efficiency and reduce the burden on workers, and the most basic conditions for completing the processes of assembling, welding, spraying and the like of the hydraulic oil cylinder are to obtain the information of the size, the pose and the like of the hydraulic oil cylinder.

The current workpiece measurement methods based on image processing are more, but there are few methods for hydraulic rams. General methods such as "an automatic measuring apparatus and measuring method of workpiece angle based on image processing" (201210483489.3) of the invention can test the position and selected angle of the workpiece to be measured on the table, but this method is suitable for small, flat two-dimensional planar workpieces; the invention relates to a workpiece multi-size multi-parameter measuring method based on image processing (201610673414.X), which constructs a measuring platform, utilizes a camera to shoot and evaluate the position of a workpiece, and then uses a scanner to perform sliding scanning, and has the advantages of high measuring precision, low operation efficiency and unsuitability for hydraulic oil cylinders in mass production; the invention relates to an image processing system, an image processing device and a workpiece picking method (201710943152.9), which provides a method for picking up any workpiece position, wherein stacked workpieces are picked up from top to bottom according to the stacking height, and the method is also more suitable for two-dimensional planar workpieces; the invention discloses a workpiece identification and sorting device and method based on multi-image acquisition equipment (201610125496.4), which uses two cameras to respectively acquire images of all workpieces and a single workpiece, identify the models of the workpieces and determine a sorting method, and has certain operability on the production of a hydraulic oil cylinder.

Disclosure of Invention

The invention provides a method and a system for measuring the size and estimating the pose of a hydraulic oil cylinder based on a video image, which are used for acquiring the information such as the size, the pose and the moving speed of the hydraulic oil cylinder and realizing the operations of an industrial robot such as grabbing, assembling, welding, spraying and the like on the hydraulic oil cylinder.

The technical scheme of the invention is realized as follows:

a hydraulic cylinder size measurement and pose estimation method and system based on video images are disclosed, wherein the method comprises the following steps:

s1, conveying by a conveying line: the hydraulic oil cylinder is conveyed between the two working procedure positions by using a conveying line, and the hydraulic oil cylinder rotates when passing through the imaging device;

s2, continuous imaging: using an imaging device to continuously shoot the hydraulic oil cylinder and transmitting the obtained video image to an industrial personal computer for processing

S3, image measurement: detecting the image sent by the imaging device by using a video image recognition algorithm in an industrial personal computer, and recognizing and measuring each part of the hydraulic cylinder barrel, the cylinder cover, the piston rod and the hydraulic lock

S4, speed evaluation: measuring the position of a hydraulic cylinder in a continuous multi-frame video image by using a video image recognition algorithm in an industrial personal computer, and calculating the moving speed of the hydraulic cylinder

S5, pose evaluation: and (3) measuring the visible size of the hydraulic lock image on the side surface of the hydraulic cylinder barrel in real time by using a video image recognition algorithm in an industrial personal computer, calculating the self-rotation angle of the hydraulic cylinder barrel, and evaluating the pose of the hydraulic cylinder barrel.

Further, the video image recognition algorithm in step S3 includes the following steps:

s31, extracting the outline of the hydraulic oil cylinder in the video image,

s32, distinguishing and positioning the outline of the hydraulic oil cylinder based on the semantic positions and the geometric shapes of the hydraulic cylinder barrel, the cylinder cover, the piston rod and the hydraulic lock,

and S33, measuring the sizes of the hydraulic oil cylinder and the hydraulic cylinder barrel, the cylinder cover, the piston rod and the hydraulic lock of the hydraulic oil cylinder based on the camera calibration information.

Further, the evaluation of the moving speed of the hydraulic oil cylinder in the step S4 includes the following steps:

s41, calculating the pixel displacement of the hydraulic oil cylinder in the continuous image frame of the video,

s42, converting the pixel displacement into the actual moving distance according to the camera calibration information,

and S43, dividing the time difference by the corresponding time difference of the continuous image frames to obtain the moving speed.

The assessment of the pose of the hydraulic oil cylinder in the step S5 is to measure the self-rotation angle of the hydraulic oil cylinder on the transport line, and the method comprises the following steps:

s51, detecting that the visible size of the hydraulic lock image on the side of the hydraulic cylinder barrel is maximized,

s52, identifying that the hydraulic lock is at the left side or the right side of the hydraulic cylinder at the moment,

s53, the left side determines that the self-rotation angle of the hydraulic oil cylinder is 0 degree at the moment, the right side determines that the self-rotation angle is 180 degrees,

and S54, calculating the self-rotation angle of the hydraulic oil cylinder at any time.

The system comprises a transport line, an imaging device and an industrial personal computer, wherein the transport line is a chain type transmission line, clamping devices for fixing the hydraulic oil cylinders are hung on the transport line at equal intervals, the clamping devices are connected with cylinder covers of the hydraulic oil cylinders, gears are arranged at the tail ends of the clamping devices and rotate around suspended central axes of the gears, and tooth grooves or powered chains are installed on the transport line and drive the gears to enable the suspended hydraulic oil cylinders to rotate;

the imaging device is composed of a light-emitting background plate and an industrial camera, the height of the light-emitting background plate is larger than that of the hydraulic oil cylinder in a suspension state, and the width of the light-emitting background plate is larger than the moving distance of the hydraulic oil cylinder on a conveying line in the time required by one rotation circle of the hydraulic oil cylinder.

The industrial personal computer detects images sent by the imaging device by using a video image recognition algorithm, recognizes and measures all parts of the hydraulic cylinder barrel, the cylinder cover, the piston rod and the hydraulic lock, and the visual field of the industrial camera covers the whole luminous background plate.

The invention has the beneficial effects that: the method and the system for measuring the size and estimating the pose of the hydraulic oil cylinder based on the video image can dynamically detect, track and identify the hydraulic oil cylinder, give information such as the size, the posture and the running speed of the hydraulic oil cylinder and the position and the size of each part in real time, and are convenient for automatic equipment such as an industrial robot and the like to formulate schemes of grabbing, assembling, welding, spraying and the like. The system provided by the invention can be widely used for production lines of hydraulic oil cylinders of engineering machinery, so that the production efficiency of products is accelerated, the burden of workers is reduced, and the system meets the policy requirements of developing intelligent manufacturing in China.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.

FIG. 1 is a flow chart of a hydraulic cylinder dimension measurement and pose estimation method based on video images;

FIG. 2 is a diagram showing the components of a hydraulic cylinder dimension measurement and pose estimation system based on video images;

FIG. 3 illustrates the change in position of a hydraulic ram in a video image.

In the figure, 101, a transport line 102, a chain 103, a clamping device 104, a gear 201, an imaging device 202, a light-emitting background plate 203, an industrial camera 301, an industrial personal computer 302, a hydraulic cylinder 303, a cylinder cover 304, a piston rod 305, a hydraulic lock 501 and the next operation process are shown.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

FIG. 1 shows a flowchart overview of a hydraulic cylinder dimension measurement and pose estimation method based on a video image, and FIG. 2 shows a composition diagram overview of a hydraulic cylinder dimension measurement and pose estimation system based on a video image.

Step S1 transfer line: the hydraulic ram is transported between two process positions using a transport line 101 and is allowed to spin as it passes through the imaging device.

In an embodiment, the transport line 101 is a chain type transport line, and the transport line 101 is suspended with clamping devices 103 at equal intervals for fixing the hydraulic oil cylinder, the clamping devices can be clamps or hooks, the clamps or hooks can be connected with cylinder covers 303 of the hydraulic oil cylinder, and the tail ends of the clamps or hooks are provided with gears 104 and can rotate around the suspended central axis. The transport line 101 is attached with a toothed or powered chain 102 from the beginning of the imaging device 201 to the next operation 501, which drives a gear 104 to rotate a suspended hydraulic cylinder.

Step S2 continuous imaging: the imaging device 201 is used for continuously shooting the hydraulic oil cylinder, and the obtained video image is transmitted to the industrial personal computer 301 for processing.

In an embodiment, the imaging device 201 is composed of a light-emitting background board 202, an industrial camera 203, and the like. The height of the light-emitting background plate 202 is larger than that of the hydraulic oil cylinder in a suspension state, and the width of the light-emitting background plate is larger than the moving distance of the hydraulic oil cylinder on the transportation line 101 in the time required by the hydraulic oil cylinder to rotate for one circle. The view of the industrial camera 203 covers the whole light-emitting background plate 202, and the hydraulic oil cylinder completes a complete rotation in the view of the industrial camera 203. Under the irradiation of the light-emitting background plate 202, ambient light can be shielded, and the hydraulic oil cylinder can form a high-contrast video image in front of the light-emitting background plate 202.

Step S3 image measurement: the industrial personal computer 301 detects an image transmitted from the imaging device 201 using a video image recognition algorithm, and recognizes and measures each component such as the hydraulic cylinder 302, the cylinder head 303, the piston rod 304, and the hydraulic lock 305.

In the embodiment, the video image recognition algorithm uses a binarization algorithm to distinguish a hydraulic oil cylinder area and a background area of an image, uses an edge detection algorithm to extract a complete outline of the hydraulic oil cylinder, matches the outline according to the relative position and shape characteristics of each part of the hydraulic oil cylinder, and recognizes each part of the hydraulic oil cylinder 302, a cylinder cover 303, a piston rod 304, a hydraulic lock 305 and the like.

Meanwhile, the positions of the length and width end points of the outlines of the hydraulic oil cylinder and each part of the hydraulic oil cylinder 302, the cylinder cover 303, the piston rod 304, the hydraulic lock 305 and the like in the image are obtained, and the positions of the length and width end points of the hydraulic oil cylinder and each part of the hydraulic oil cylinder in a Cartesian coordinate system in an actual physical space are obtained through solving based on the projection transformation obtained through camera calibration, so that the actual size of the hydraulic oil cylinder is obtained.

Step S4 speed assessment: the position of a hydraulic cylinder in a plurality of continuous video frames of images is measured by using a video image recognition algorithm in the industrial personal computer 301, and the moving speed of the hydraulic cylinder is calculated.

In an embodiment, at t₀The image coordinate of the top 306 of the hydraulic cylinder at the moment is (u)₀，v₀)，t₁The image coordinate at the top 307 of the hydraulic cylinder at the moment is (u)₁，v₁) And the projective transformation matrix obtained by calibrating the camera is W, then t₀And t₁The actual positions of the top ends of the hydraulic oil cylinders at the moment are respectively (x)₀，y₀) And (x)₁，y₁)：

[x₀ y₀ z w]′＝W^-1[u₀ v₀ 1]，

[x₁ y₁ z w]′＝W^-1[u₁ v₁ 1]，

Wherein z is a hydraulic oil cylinderThe distance between the suspension plane and the camera, w is a constant term. The default hydraulic oil cylinder only moves transversely (in the X-axis direction), so that the moving speed v of the hydraulic oil cylinder is obtained_x＝(x₁-x₀)/(t₁-t₀)。

Step S5 pose evaluation: the visible size of the image of the hydraulic lock 305 on the side surface of the hydraulic cylinder barrel 302 is measured in real time by using a video image recognition algorithm in the industrial personal computer 301, and the self-rotation angle of the hydraulic cylinder is calculated.

In an embodiment, at t₀At the moment when the visible size of the image of the hydraulic lock 305 is captured to the maximum and the hydraulic lock is located on the left side of the hydraulic cylinder 302, t is determined₀The self-rotation angle of the hydraulic oil cylinder at any moment is 0 degrees, if a tooth groove is adopted to drive a clamp or a hook tail end gear to rotate, the self-rotation angle of the hydraulic oil cylinder at any moment t is as follows:

if a mode that a power chain drives a clamp or a hook tail end gear to rotate is adopted, the self-rotation angle of the hydraulic oil cylinder at any moment t is as follows:

where R is the radius of the gear 104 and v' is the rotational speed of the powered chain 102.

The self-rotation angle of the hydraulic oil cylinder at the moment when the hydraulic oil cylinder reaches the working procedure can be known by the automatic equipment (such as an industrial robot) of the next operation working procedure 501 through pose evaluation, the positions of key components such as a hydraulic lock can be determined when the operations such as grabbing, assembling, welding and spraying are carried out, relevant operation paths are automatically established, and accidents such as collision are prevented.

Through the steps, the size measurement and the pose estimation of the common hydraulic oil cylinder comprising the hydraulic cylinder barrel, the cylinder cover, the piston rod, the hydraulic lock and the like can be realized, the conveying line effectively reduces the carrying strength of workers, and the obtained information of the size, the speed, the rotation angle and the like of the hydraulic oil cylinder can dynamically plan accurate operation paths for the next procedure, such as grabbing, assembling, welding, spraying and the like, so that the automatic production level of the hydraulic oil cylinder is improved.

The invention considers that the hydraulic oil cylinder is basically composed of a cylinder barrel and a cylinder cover, a piston and a piston rod, and a hydraulic lock attached to the cylinder barrel, has a relatively standard cylindrical appearance, and the cylinder cover is generally provided with a circular hole for connecting and fixing, so the hydraulic oil cylinder is suitable for suspension transmission, and the whole outline of the hydraulic oil cylinder can be obtained through an imaging device, so the size, the pose and the moving speed of the hydraulic oil cylinder on a transmission line can be calculated by utilizing an image measurement mode.

In one embodiment of the method and the system for measuring the size and estimating the pose of the hydraulic cylinder based on the video image, the transport line with the self-rotating device is provided with a clamp or a hook which is connected with a cylinder cover of the hydraulic cylinder, so that the hydraulic cylinder is in a vertical suspension state, and tooth spaces or a powered chain is utilized to act on gears on the clamp or the hook, so that the hydraulic cylinder rotates around the central axis of a cylinder barrel of the hydraulic cylinder.

The embodiments of the present invention have been described, but the present invention is not limited to the described embodiments. The design method of the transmission line, the imaging device and the like can be modified according to the application situations of actual grabbing, assembling, welding, spraying and the like, but the implementation process follows the framework scope of the appended claims. The practice of the invention may thus be modified and varied within the spirit and scope of the appended claims, and the description is therefore to be regarded as illustrative rather than limiting.