阅读说明：本技术 一种基于机器视觉的降落伞绳圈位姿检测方法及系统 (Parachute rope loop pose detection method and system based on machine vision ) 是由 史铁林 阳静 熊烽 熊良才 廖圣洁 马海 刘进雷 凡小平 于 2021-01-19 设计创作，主要内容包括：本发明属于人工智能相关技术领域,并公开了一种基于机器视觉的降落伞绳圈位姿检测方法及系统。该方法包括：S1采集降落伞绳圈的图像,对该图像进行处理以此提取绳圈的边缘,根据该边缘确定绳圈的中心线；S2获取绳圈中心线上各点的图像坐标,并将中心线上各个点的图像坐标转化为空间三维坐标,以此确定绳圈的三维曲线,实现绳圈的位置和姿态的检测。本发明还公开了利用上述检测方法的检测系统,包括图像采集模块、图像处理模块和控制中心,图像采集模块采集绳圈的图像,图像处理模块对采集的图像进行处理获得绳圈的中心和平面法矢量；控制中心控制降落伞伞绳穿过绳圈。通过本发明,实现绳圈的自动位姿识别和定位,提升生产效率和自动化程度。(The invention belongs to the technical field of artificial intelligence correlation, and discloses a parachute line loop pose detection method and system based on machine vision. The method comprises the following steps: s1, collecting the image of the parachute loop, processing the image to extract the edge of the loop, and determining the center line of the loop according to the edge; s2, obtaining the image coordinates of each point on the central line of the rope loop, and converting the image coordinates of each point on the central line into space three-dimensional coordinates, so as to determine the three-dimensional curve of the rope loop and realize the detection of the position and the posture of the rope loop. The invention also discloses a detection system using the detection method, which comprises an image acquisition module, an image processing module and a control center, wherein the image acquisition module acquires images of the rope loop, and the image processing module processes the acquired images to obtain the center and the plane normal vector of the rope loop; the control center controls the parachute cord to penetrate through the cord loop. By the aid of the method, automatic pose identification and positioning of the rope loop are achieved, and production efficiency and automation degree are improved.)

1. A parachute line loop pose detection method based on machine vision is characterized by comprising the following steps:

s1, in the packaging process of the parachute, collecting the image of the parachute rope loop, processing the image to extract the edge of the rope loop, and determining the center line of the rope loop according to the edge;

s2, obtaining the image coordinates of each point on the central line of the rope loop, and converting the image coordinates of each point on the central line into a space three-dimensional coordinate, so as to obtain a three-dimensional curve of the rope loop in the space three-dimensional coordinate, and realize the detection of the position and the posture of the rope loop.

2. A machine vision based parachute line loop pose detection method as claimed in claim 1, wherein in step S1, the processing of the image comprises graying, filtering, edge detection and binarization, wherein the graying is used for filtering out background color in the image, the filtering is used for removing residual background noise, and the edge detection and binarization are used for extracting and obtaining edges of the line loop.

3. A parachute loop pose detection method based on machine vision as claimed in claim 1, wherein in step S2, after determining the three-dimensional curve of said loop, the center of the three-dimensional curve and the normal vector of the loop plane are calculated to locate the center of said loop.

4. A parachute line loop pose detection method based on machine vision as claimed in claim 1, wherein in step S2, the image coordinates of each point on the center line are converted into three-dimensional space coordinates by establishing a conversion matrix between the image coordinate system and the three-dimensional space coordinate system, and then converting the image coordinates of each point into three-dimensional coordinates by using the conversion matrix.

5. A detection system for detecting by using the detection method of any one of claims 1 to 4, wherein the system comprises an image acquisition module, an image processing module and a control center, wherein the image acquisition module is used for acquiring an image of a parachute rope loop, and the image processing module is used for processing the acquired image so as to obtain the center of the rope loop and a plane normal vector; the control center is simultaneously connected with the image acquisition module and the image processing module, the image processing module transmits the center of the rope loop and a plane normal vector to the control center, and the control center controls the parachute rope to penetrate through the rope loop.

6. The inspection system of claim 5, wherein the image capture module employs a monocular vision inspection device or a binocular vision inspection device.

7. The inspection system of claim 5, wherein the image capture module is further calibrated for error prior to capturing the image of the parachute eye.

Technical Field

The invention belongs to the technical field of artificial intelligence correlation, and particularly relates to a parachute line loop pose detection method and system based on machine vision.

Background

With the increasing requirements on rapidity and safety of parachute folding and sorting, the adoption of an automatic parachute folding, sorting and packaging process is more and more important. In the automatic production process of folding, arranging and packaging the parachute, the parachute stringing is an important process. The umbrella rope is to orderly pass through the rope loops on the umbrella bag according to a certain sequence. Since the parachute package and the loops thereon are made of soft material and the states thereof take various forms, in order to correctly thread the parachute cord through the respective loops, the positions and postures of the respective loops must be detected.

Therefore, the invention provides a method and a system for automatically detecting the pose of a rope loop on a parachute bag aiming at the requirement that a parachute rope automatically penetrates through the rope loop device, namely a method and a system for automatically detecting the pose of the rope loop on the parachute bag by adopting machine vision.

Disclosure of Invention

Aiming at the defects or the improvement requirements of the prior art, the invention provides a parachute loop pose detection method and a parachute loop pose detection system based on machine vision.

To achieve the above object, according to one aspect of the present invention, there is provided a machine vision-based parachute line loop pose detection method, comprising:

s1, in the packaging process of the parachute, collecting the image of the parachute rope loop, processing the image to extract the edge of the rope loop, and determining the center line of the rope loop according to the edge;

s2, obtaining the image coordinates of each point on the central line of the rope loop, and converting the image coordinates of each point on the central line into space three-dimensional coordinates, so as to obtain the three-dimensional curve of the rope loop in the space three-dimensional coordinates, and realize the detection of the position and the posture of the rope loop.

Further preferably, in step S1, the processing on the image includes graying, filtering, edge detection and binarization, wherein the graying is used for filtering out background color in the image, the filtering is used for removing residual background noise, and the edge detection and binarization are used for extracting edges of the obtained rope loops.

Further preferably, in step S2, after determining the three-dimensional curve of the loop, the center of the three-dimensional curve and the normal vector of the loop plane are calculated, so as to locate the center of the loop.

Further preferably, in step S2, the image coordinates of each point on the center line are converted into the three-dimensional spatial coordinates by establishing a conversion matrix between the image coordinate system and the three-dimensional spatial coordinate system, and then converting the image coordinates of each point into the three-dimensional coordinates by using the conversion matrix.

According to another aspect of the present invention, there is also provided a detection system for detecting by the above detection method, the system includes an image acquisition module, an image processing module and a control center, wherein the image acquisition module is configured to acquire an image of a parachute loop, and the image processing module is configured to process the acquired image, so as to obtain a center of the parachute loop and a plane normal vector; the control center is simultaneously connected with the image acquisition module and the image processing module, the image processing module transmits the center of the rope loop and a plane normal vector to the control center, and the control center controls the parachute rope to penetrate through the rope loop.

Further preferably, a monocular vision detection device or a binocular vision detection device is adopted in the image acquisition module.

Further preferably, before the image acquisition module acquires the image of the parachute line loop, the error of the image acquisition module needs to be calibrated.

Generally, compared with the prior art, the technical scheme of the invention has the following beneficial effects:

1. the method has the advantages of strong stereoscopic impression compared with two-dimensional image identification and greatly reduced calculated amount compared with the common three-dimensional reconstruction method, so that the method not only can be used for three-dimensional reconstruction of the outline center line of the rope loop, but also is very suitable for positioning the center position of the rope loop and recognizing the posture;

2. according to the invention, the automatic detection system is constructed, the structure of the detection system is simple, the monocular or binocular detection device is adopted to collect the images of the rope loops, then the images are processed to obtain the centers of the rope loops, the umbrella ropes are fed into the centers of the rope loops to penetrate through the rope loops according to the determined centers, the automatic rope threading process is realized, the labor cost is reduced, and the rope threading efficiency is improved.

Drawings

FIG. 1 is a flow chart of a machine vision based parachute eye pose detection method constructed in accordance with a preferred embodiment of the present invention;

FIG. 2 is a schematic diagram of a monocular vision based loop pose detection system constructed in accordance with a preferred embodiment of the present invention;

fig. 3 is a schematic structural diagram of a binocular vision-based loop pose detection system constructed according to a preferred embodiment of the invention.

The same reference numbers will be used throughout the drawings to refer to the same or like elements or structures, wherein:

101-loop, 102-light source, 103-camera, 104-motion mechanism, 105-track, 106-host.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

A parachute line loop pose detection method based on machine vision is characterized in that the method comprises the following basic processes: respectively acquiring images of a detected area by the calibrated left camera and the calibrated right camera, inputting the acquired images into a computer, filtering the images acquired by the left camera and the right camera by a computer image processing program, then extracting the edge of a rope loop of the left image and the right image after filtering, and then extracting the outline center lines of the rope loop of the left image and the rope loop of the right image; and finally, calculating the space coordinates of each point of the rope loop contour central line according to the rope loop contour central lines of the left image and the right image in sequence, and constructing an image of a rope loop central curve in a three-dimensional space, thereby determining the position and the posture of each rope loop.

In order to realize the position and pose detection of the rope loop, the invention respectively constructs a rope loop position and pose detection system based on monocular vision and a rope loop position and pose detection system based on binocular vision.

The loop pose detection system based on monocular vision comprises a light source, an industrial camera, a motion device for driving the industrial camera to move, an image acquisition and transmission device (the motion device can be integrated in the industrial camera), hardware such as an image processing computer and image acquisition and processing software. The difference between the composition of the rope loop pose detection system based on binocular vision and the rope loop pose detection system based on monocular vision lies in that two industrial cameras which are fixedly installed are needed, and a motion device which drives the industrial cameras to move is not needed. The system software comprises an image acquisition module, a camera calibration module, an image preprocessing module, a left and right image rope loop edge extraction module, a left and right image rope loop contour center line fitting module, a rope loop contour center line calculation module of a world coordinate system and a rope loop pose output module.

The image acquisition module controls the left camera and the right camera to acquire images and stores the acquired images in the image processing computer; the camera calibration module is used for calibrating a binocular camera of the system when the internal and external parameters of the camera are changed, determining the internal and external parameters and providing a basis for calculating the three-dimensional coordinates of the contour center line of the rope loop at the back; the image preprocessing module realizes preprocessing functions of enhancing, denoising and the like of the left image and the right image acquired by the camera; the left and right image rope loop edge extraction module realizes the extraction of the rope loop edges in the left and right images and determines the position, shape and size of the rope loop on an image plane; the left and right image rope loop contour center line fitting module fits the rope loop contour center lines in respective images according to the rope loop edges; the rope loop contour central line world coordinate calculation module has the function of calculating the coordinates of each point of a three-dimensional curve at the center of the rope loop in a world coordinate system according to the rope loop contour central lines of the left image and the right image; and the rope loop pose output module calculates the center position and the posture of the rope loop according to the three-dimensional coordinates of each point of the contour center line of the rope loop.

The implementation method of each module is explained as follows:

the camera calibration module adopts a Zhangzhengyou classical calibration method, and the calibration board adopts a checkerboard. Firstly, for a binocular system, acquiring images of a plurality of chequers through a left camera and a right camera; for the monocular system, the camera position is changed for multiple times through the motion system, and images of multiple checkerboards are acquired. And then, sequentially reading the collected calibration plate images and extracting the corner point coordinates of the checkerboard. And then, calibrating the camera by using the coordinate values of the angular points, and solving an internal parameter matrix, a distortion parameter and an external parameter matrix of each calibration plate image of the camera. And finally, carrying out re-projection calculation on the angle points through the obtained internal and external parameters of the camera, and calculating the calibration error of the camera.

The image preprocessing module comprises the steps of graying, filtering, edge detection, binarization and the like, extracts components consistent with the colors of the rope loops from the left image and the right image according to the characteristic that the colors of the rope loops of the existing umbrella bags are different from the colors of the umbrella bags, filters the background colors, performs graying processing, processes residual background noise by adopting a filtering method, and finally performs edge detection and binarization processing to obtain the edge images of the rope loops.

The left and right image rope loop contour center line fitting module takes an average value according to the edge coordinates of each section of the rope loop in the left and right images to obtain the center point of the corresponding position, so that the contour center lines of the rope loop in the left and right images are fitted.

And the rope loop contour center line calculation module of the world coordinate system determines the actual coordinates of corresponding points according to the image plane coordinates of the rope loop contour center lines of the left image and the right image in sequence according to the fitted rope loop contour center lines in the left image and the right image and the triangulation principle, thereby determining the center line of the rope loop in the world coordinate system.

And the rope loop pose output module determines the three-dimensional coordinate of the center of the rope loop according to the three-dimensional curve of the center of the rope loop of the world coordinate system and outputs the normal vector of the rope loop.

The method is characterized in that the method fits the rope loop contour central lines in the left and right images according to the characteristic that the diameter of the rope loop is small, then calculates the three-dimensional coordinates of the rope loop contour central lines in the world coordinate system according to the corresponding points of the rope loop contour central lines of the fitted left and right images to obtain the rope loop contour central lines of the world coordinate system, and accordingly determines the coordinates of the center points of the rope loops and the normal vectors of the rope loop surface. Compared with two-dimensional image recognition, the method has the advantage of strong stereoscopic impression, and compared with a common three-dimensional reconstruction method, the method has the advantage of greatly reducing the calculated amount, so that the method not only can be used for three-dimensional reconstruction of the outline center line of the rope loop, but also is very suitable for positioning the center position of the rope loop and recognizing the posture.

Example 1:

FIG. 1 shows the hardware composition of a loop pose detection system based on monocular vision. After the installation is completed, camera calibration is performed. The calibration process is that the light source 102 irradiates the calibration plate, the camera 103 shoots an image of the calibration plate at the left position as a left image, then the camera is driven by the motion mechanism 104 to move to the right position, and the camera 103 shoots an image of the calibration plate at the right position as a right image. Repeating for multiple times to finish camera calibration and obtain the internal and external parameters of the camera.

After the calibration of the camera, the system can be normally applied. The working process is as follows: the light source 102 irradiates the loop 101, the camera 103 takes an image at the left position as a left image, then the camera moves to the right side of the track 105 under the drive of the motion mechanism 104, and the camera 103 takes an image at the right position as a right image. The method comprises the steps of sequentially carrying out image preprocessing, extraction of the rope ring edges of the left image and the right image, fitting of the rope ring contour center lines of the left image and the right image, calculation of the rope ring contour center line of a world coordinate system and output of the rope ring pose to obtain the rope ring contour center line, the rope ring center coordinate and a normal vector of a rope ring plane, and transmitting the results to the host computer 106.

Example 2:

fig. 2 shows the hardware composition of the loop pose detection system based on binocular vision. The remaining embodiment 1 is different in that the left and right cameras 103 are employed without a moving mechanism. The working process differs from that of embodiment 1 in that the acquisition of the left and right images does not require the action of a moving mechanism.

The steps in the actual operation of the invention are as follows:

s1, when the parachute cord automatically passes through the coil, the parachute cord is fixed according to certain requirements through the related cord threading mechanism;

s2 the umbrella bag with coil is fixed on the workbench;

s3, the stringing mechanism drives the umbrella rope fixed according to the requirement to the position near the umbrella bag of the workbench;

s4, acquiring the image of the coil by the image acquisition device, and performing relevant machine vision processing to obtain the position posture of the coil on the workbench;

s5, after the position posture of the coil is determined, the cord threading mechanism controls the umbrella cord to thread into the coil;

and S6, moving the image acquisition device, acquiring the position and posture data of the next coil, and repeating the threading step to realize that all umbrella ropes penetrate into the corresponding coils.

It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.