阅读说明：本技术 一种多双目相机与线激光协同检测方法 (Multi-binocular camera and line laser cooperative detection method ) 是由 董霄剑 曾洪庆 钱超超 于 2019-08-27 设计创作，主要内容包括：一种多双目相机与线激光协同检测方法,包括以下步骤：架设双目相机模组；对各双目相机的左右摄像头进行立体标定；分别获取被测物体的左图像和右图像,进行立体校正,匹配点对,根据左右视图视差,计算得到被测物体的三维信息；进行位姿关系的坐标转换；融合同一时刻的实时深度图像数据和实时姿态数据,分别计算各坐标系的三维位置,以得到完整的三维数据。双目相机模组安装在平台的同一平面上,且呈一字型排列安装。本发明采用多组线激光与双目相机的模组平行安装于一平面进行物体检测,获取三维坐标后进行三维重建、数据融合,采用多模组协同扫描的方式大大加快了测量速度,提高了测量效率,测量精度也比传统单双目相机精度更高。(A multi-binocular camera and line laser cooperative detection method comprises the following steps: erecting a binocular camera module; carrying out three-dimensional calibration on left and right cameras of each binocular camera; respectively acquiring a left image and a right image of a measured object, performing three-dimensional correction, matching point pairs, and calculating to obtain three-dimensional information of the measured object according to left and right view parallax; carrying out coordinate conversion of the pose relation; and fusing the real-time depth image data and the real-time attitude data at the same moment, and respectively calculating the three-dimensional position of each coordinate system to obtain complete three-dimensional data. The binocular camera modules are arranged on the same plane of the platform and are arranged in a straight line. The invention adopts a plurality of groups of line lasers and binocular camera modules which are arranged on a plane in parallel to carry out object detection, three-dimensional reconstruction and data fusion are carried out after three-dimensional coordinates are obtained, the measurement speed is greatly accelerated by adopting a multi-module collaborative scanning mode, the measurement efficiency is improved, and the measurement precision is higher than that of the traditional single-binocular camera.)

1. A multi-binocular camera and line laser cooperative detection method is characterized by comprising the following steps:

s1: the method is characterized in that a platform where a measured object is located is used as a reference surface, a binocular camera module consisting of a binocular camera and a line laser emitter is erected above the reference surface, the binocular camera can shoot laser lines, and the positions of the binocular camera and the line laser emitter are kept relatively fixed.

S2: and carrying out three-dimensional calibration on the left camera and the right camera of each binocular camera.

S3: each binocular camera module respectively acquires a left image and a right image of a measured object through a left camera and a right camera of a binocular camera, and performs stereo correction on the left image and the right image;

matching the corrected left view and the corrected right view to obtain a linear laser imaging matching point pair;

and obtaining left and right view parallax according to the line laser matching point pairs, and calculating to obtain three-dimensional information of the object to be measured according to the left and right view parallax.

S4: and (4) carrying out coordinate conversion of the pose relationship on the binocular cameras, namely converting the pose relationship into the same coordinate system.

S5: and fusing the real-time depth image data and the real-time attitude data of the measured object acquired by each binocular camera module at the same moment, and respectively calculating the three-dimensional positions of the corresponding calibration objects in the coordinate systems of the binocular stereo cameras to obtain complete three-dimensional data.

2. The multi-binocular camera and line laser cooperative detection method according to claim 1, wherein in the step S1, the binocular camera modules are installed on the same plane of the platform of the object to be detected and arranged in a straight line.

3. The multi-binocular camera and line laser cooperative detection method according to claim 1, wherein in the step S1, the number of the binocular camera modules is two or more.

4. The multi-binocular camera and line laser cooperative detection method according to claim 1, wherein in the step S1, the line laser emitter is built in the binocular camera, or the line laser emitter is externally hung on the binocular camera and forms a common structure with the binocular camera, or the line laser emitter and the binocular camera are installed in a split manner.

5. The multi-binocular camera and line laser cooperative detection method according to claim 1, wherein in the step S2, a left camera and a right camera of each binocular camera are calibrated stereoscopically; including the calculation of the internal reference matrix K and the distortion matrix D:

for each image and the corresponding calibration checkerboard angular point, the least square solution of B can be obtained by a matrix method, and the camera internal parameter K is further obtained. Further solving an optimal solution according to the checkerboard angular points of the plurality of images, wherein the optimal solution is a solution which enables the following equation to be minimum:

considering the radial distortion, the above optimization problem translates into a minimization problem of the following formula:

and (3) by using a Levenberg-Marquardt method, and using the solution obtained from the first image as an initial value, thereby obtaining an optimal solution by iteration.

6. The method for detecting the coordination of the multiple binocular cameras and the line laser according to claim 1, wherein in the step S2, the binocular camera modules work cooperatively when starting to work, and the timing synchronization is maintained, that is, the binocular camera modules are set to be simultaneously triggered by an external trigger signal at the same time, or an upper computer triggers the binocular cameras to enter the stream mode, and the binocular stereo camera timers are simultaneously set to be zero at the same time, and one master camera sends a trigger signal and works simultaneously after receiving the trigger signal from the cameras.

7. The method for detecting the coordination of the multi-binocular camera and the line laser according to claim 1, wherein in the step S4, the distance between the coordinate systems of the binocular camera modules in the x-axis direction is calculated according to the distance between the binocular camera modules and the moving speed of the object to be detected, and the three-dimensional coordinates of the object to be detected in the coordinate systems of the binocular camera modules are obtained according to the distance in the x-axis direction.

8. The multi-binocular camera and line laser cooperative detection method according to claim 1, wherein in the step S4, the binocular stereo camera module respectively acquires three-dimensional data of the object to be detected, and stereo-matches the same features of each group of views with a unified calibration coordinate system as an origin to acquire the overall three-dimensional data of the object to be detected.

Technical Field

The invention relates to the technical field of object three-dimensional information detection, in particular to a multi-binocular camera and line laser cooperative detection method.

Background

Binocular vision is an important branch of computers, can simulate human eyes and the process of human stereoscopic vision perception, and is one of the core subjects of computer vision research. In recent years, binocular vision technology has been widely used in the fields of obstacle detection, target object detection, and the like.

The binocular stereo vision uses one or two CCD or CMOS digital cameras to shoot the same surface of the measured object from different angles, and obtains the three-dimensional coordinates of the point by calculating the parallax of the space point in the two images. This measurement method requires determining the corresponding position of the same point in space on two or more images taken at different angles. The information of the surface of an object is collected when the object is scanned, laser rays are projected to the surface of the object by using a laser, the laser rays are collected by a camera after being reflected by the surface of the object, and the laser stripes on the collected image can provide the information of dense object surface points.

In the common binocular vision shooting range, three-dimensional coordinate data of all points in a space are calculated through different visual angle image matching and triangulation principles, the precision of three-dimensional data obtained by a stereo camera completely depends on the precision of image matching, the precision of image matching is greatly influenced by factors such as illumination, a measured space and the like, simultaneously, because line laser is emitted from the top or the side, only object three-dimensional information in a certain visual plane can be obtained through one-time detection, and certain blind areas exist due to the fact that omnibearing shooting cannot be carried out, and the precision of three-dimensional detection is seriously influenced. The laser scanner can only obtain the three-dimensional coordinate data of points on the laser line, the laser scanner or a scanned object needs to be moved, the three-dimensional data of all the points in the scanned space can be obtained through splicing, and the cost of the whole system is high; in summary, no three-dimensional data acquisition product with high precision, low price, large shooting range and spatial resolution exists in the market at present.

Disclosure of Invention

The invention aims to solve the technical problem of providing a multi-binocular camera and line laser cooperative detection device and method aiming at the defects involved in the background technology.

The purpose of the invention and the technical problem to be solved are realized by adopting the following technical scheme:

a multi-binocular camera and line laser cooperative detection method comprises the following steps:

s1: the method is characterized in that a platform where a measured object is located is used as a reference surface, a binocular camera module consisting of a binocular camera and a line laser emitter is erected above the reference surface, the binocular camera can shoot laser lines, and the positions of the binocular camera and the line laser emitter are kept relatively fixed.

S2: and carrying out three-dimensional calibration on the left camera and the right camera of each binocular camera.

S3: each binocular camera module respectively acquires a left image and a right image of a measured object through a left camera and a right camera of a binocular camera, and performs stereo correction on the left image and the right image;

matching the corrected left view and the corrected right view to obtain a linear laser imaging matching point pair;

and obtaining left and right view parallax according to the line laser matching point pairs, and calculating to obtain three-dimensional information of the object to be measured according to the left and right view parallax.

S4: and (4) carrying out coordinate conversion of the pose relationship on the binocular cameras, namely converting the pose relationship into the same coordinate system.

S5: and fusing the real-time depth image data and the real-time attitude data of the measured object acquired by each binocular camera module at the same moment, and respectively calculating the three-dimensional positions of the corresponding calibration objects in the coordinate systems of the binocular stereo cameras to obtain complete three-dimensional data.

Preferably, in the step S1, the binocular camera modules are installed on the same plane of the platform of the object to be measured and are installed in a linear arrangement.

Preferably, in step S1, the number of the binocular camera modules is two or more.

Preferably, in the step S1, the line laser emitter is built in the binocular camera, or the line laser emitter is externally hung on the binocular camera and forms a common structure with the binocular camera, or the line laser emitter and the binocular camera are installed in a split manner.

Preferably, in the step S2, the left camera and the right camera of each binocular camera are calibrated stereoscopically; including the calculation of the internal reference matrix K and the distortion matrix D:

for each image and the corresponding calibration checkerboard angular point, the least square solution of B can be obtained by a matrix method, and the camera internal parameter K is further obtained. Further solving an optimal solution according to the checkerboard angular points of the plurality of images, wherein the optimal solution is a solution which enables the following equation to be minimum:

considering the radial distortion, the above optimization problem translates into a minimization problem of the following formula:

and (3) by using a Levenberg-Marquardt method, and using the solution obtained from the first image as an initial value, thereby obtaining an optimal solution by iteration.

Preferably, in the step S2, the binocular camera modules work cooperatively when beginning to work, and the timing synchronization is maintained, that is, the binocular camera modules are set to be triggered simultaneously by an external trigger signal at the same time, or an upper computer triggers an incoming flow mode of each binocular camera, and the timers of each binocular stereo camera are set to be cleared at the same time, and a master camera sends a trigger signal and works simultaneously after receiving the trigger signal from the slave camera.

Preferably, in the step S4, the distance in the x-axis direction between the coordinate systems to which the binocular camera modules belong is calculated according to the distance between the binocular camera modules and the moving speed of the object to be measured, and the three-dimensional coordinates of the object to be measured in the coordinate systems of the binocular camera modules are obtained from the distance in the x-axis direction.

Preferably, in the step S4, the binocular stereo camera module respectively acquires three-dimensional data of the object to be measured, and stereo-matches the same characteristics of each group of views with the multiple groups of three-dimensional data using the unified calibration coordinate system as an origin to obtain the entire three-dimensional data of the object to be measured.

Advantageous effects

The invention adopts a plurality of groups of line lasers and binocular camera modules which are arranged in parallel on a plane for object detection, effectively utilizes the high brightness and high linearity of the lasers to improve the anti-interference capability of the measuring method, and carries out three-dimensional reconstruction and data fusion after acquiring three-dimensional coordinates.

Drawings

Fig. 1 is a schematic flow chart of a multi-binocular camera and line laser cooperative detection method.

Detailed Description

The technical solution of the present invention is further described in detail below with reference to the accompanying drawings. It is obvious that the described embodiments are only a part of the embodiments of the present invention, not all embodiments, and all other embodiments obtained by those skilled in the art without inventive efforts belong to the protection scope of the present invention.

It should be understood that in the description of the present invention, it should be noted that the terms "upper", "lower", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings or orientations or positional relationships conventionally used in the product of the present invention, which are only used for convenience in describing the present invention and simplifying the description, but do not indicate or imply that the device or element referred to must have a specific orientation, be constructed in a specific orientation, and be operated, and thus should not be construed as limiting the present invention. Furthermore, although the terms first, second, third, etc. may be used herein to describe various elements, components and/or sections, these elements, components and/or sections should not be limited by these terms.

In this embodiment, a method for detecting cooperation of multiple binocular cameras and line laser includes the following steps:

s1: the method is characterized in that a platform where a measured object is located is used as a reference surface, a binocular camera module consisting of a binocular camera and a line laser emitter is erected above the reference surface, the binocular camera can shoot laser lines, and the positions of the binocular camera and the line laser emitter are kept relatively fixed.

S2: and carrying out three-dimensional calibration on the left camera and the right camera of each binocular camera.

S3: each binocular camera module respectively acquires a left image and a right image of a measured object through a left camera and a right camera of a binocular camera, and performs stereo correction on the left image and the right image;

matching the corrected left view and the corrected right view to obtain a linear laser imaging matching point pair;

and obtaining left and right view parallax according to the line laser matching point pairs, and calculating to obtain three-dimensional information of the object to be measured according to the left and right view parallax.

S4: and (4) carrying out coordinate conversion of the pose relationship on the binocular cameras, namely converting the pose relationship into the same coordinate system.

S5: and fusing the real-time depth image data and the real-time attitude data of the measured object acquired by each binocular camera module at the same moment, and respectively calculating the three-dimensional positions of the corresponding calibration objects in the coordinate systems of the binocular stereo cameras to obtain complete three-dimensional data.

The method comprises the steps of adopting a module consisting of a plurality of groups of binocular cameras and line lasers to collect images of a measured object and the line lasers in real time, sending collected image information to an image processor for processing, completing the stereo matching of left and right images, performing three-dimensional reconstruction and data fusion of coordinates of each binocular camera module, and fitting complete three-dimensional pose information of the measured object.

Preferably, in the step S1, the binocular camera modules are installed on the same plane of the platform of the object to be measured and are installed in a linear arrangement.

Preferably, in step S1, the number of the binocular camera modules is two or more.

Preferably, in the step S1, the line laser emitter is built in the binocular camera, or the line laser emitter is externally hung on the binocular camera and forms a common structure with the binocular camera, or the line laser emitter and the binocular camera are installed in a split manner.

In step S2, performing stereo calibration on the left camera and the right camera of each binocular camera; including the calculation of the internal reference matrix K and the distortion matrix D:

for each image and the corresponding calibration checkerboard angular point, the least square solution of B can be obtained by a matrix method, and the camera internal parameter K is further obtained. Further solving an optimal solution according to the checkerboard angular points of the plurality of images, wherein the optimal solution is a solution which enables the following equation to be minimum:

considering the radial distortion, the above optimization problem translates into a minimization problem of the following formula:

and (3) by using a Levenberg-Marquardt method, and using the solution obtained from the first image as an initial value, thereby obtaining an optimal solution by iteration.

Preferably, in step S2, the left camera and the right camera of the binocular camera are subjected to stereo calibration to obtain an internal reference matrix a of the binocular camera, a rotation matrix R between the left camera and the right camera, and a translation vector T:

respectively calibrating a left camera and a right camera of the binocular camera to obtain an internal reference matrix A of the binocular camera, a rotation matrix R1 of the left camera and a rotation matrix Rr of the right camera, and a translation vector T1 of the left camera and a translation vector Tr of the right camera;

and calculating a rotation matrix R and a translation vector T between the left camera and the right camera by the following formula:

preferably, in the step S2, the binocular camera modules work cooperatively when beginning to work, and the timing synchronization is maintained, that is, the binocular camera modules are set to be triggered simultaneously by an external trigger signal at the same time, or an upper computer triggers an incoming flow mode of each binocular camera, and the timers of each binocular stereo camera are set to be cleared at the same time, and a master camera sends a trigger signal and works simultaneously after receiving the trigger signal from the slave camera.

In this embodiment, performing stereo correction on the left steel bar end face contour image and the right steel bar end face contour image includes:

decomposing the rotation matrix R into two rotation matrices R1 and rr, wherein R1 and rr are obtained by assuming that the optical axes of the left camera and the right camera are parallel by rotating each of the left camera and the right camera by half;

aligning the left steel bar end face outline image and the right steel bar end face outline image by the following formula:

where Rrect is a rotation matrix that aligns the rows:

the rotation matrix Rrect starts from the direction of the pole e1, the origin of the profile image of the end face of the left steel bar is taken as the main point, and the direction of the translation vector of the left camera to the right camera is taken as the direction of the main point:

e₁and e₂Is orthogonal to e₁Normalized to unit vector:

wherein Tx is a component of the translation vector T in the horizontal direction in the plane where the binocular camera is located, and Ty is a component of the translation vector T in the vertical direction in the plane where the binocular camera is located;

e3 is orthogonal to e1 and e2, and e3 is calculated by the following formula:

e₃＝e₂×e₁

according to the physical significance of the rotation matrix, the method comprises the following steps:

wherein alpha is the angle of the left camera and the right camera which need to rotate in the plane where the left camera and the right camera are located, and alpha is more than or equal to 0 and less than or equal to 180 degrees; the left camera is rotated by α' about the e3 direction, and the right camera is rotated by α "about the e3 direction.

Preferably, in the step S4, the distance in the x-axis direction between the coordinate systems to which the binocular camera modules belong is calculated according to the distance between the binocular camera modules and the moving speed of the object to be measured, and the three-dimensional coordinates of the object to be measured in the coordinate systems of the binocular camera modules are obtained from the distance in the x-axis direction.

Preferably, in the step S4, the binocular stereo camera module respectively acquires three-dimensional data of the object to be measured, and stereo-matches the same characteristics of each group of views with the multiple groups of three-dimensional data using the unified calibration coordinate system as an origin to obtain the entire three-dimensional data of the object to be measured.

Advantageous effects

The invention adopts a plurality of groups of line lasers and binocular camera modules which are arranged in parallel on a plane for object detection, effectively utilizes the high brightness and high linearity of the lasers to improve the anti-interference capability of the measuring method, and carries out three-dimensional reconstruction and data fusion after acquiring three-dimensional coordinates.

The preferred embodiments of the present disclosure have been disclosed to assist in describing the disclosure, and alternative embodiments have not been set forth in detail to avoid obscuring the invention in the particular embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the specification and its practical application, to thereby enable others skilled in the art to best understand the specification and its practical application. The specification is limited only by the claims and their full scope and equivalents.