阅读说明：本技术 基于空间划分的slam快速回环检测方法 (SLAM rapid loop detection method based on space division ) 是由 罗楠 王义峰 万波 赵辉 梁婕 卢波 于 2021-12-10 设计创作，主要内容包括：本发明公开了基于空间划分的SLAM快速回环检测方法,其包括通过无人机机载传感器获取待测场景的图像序列、IMU测量值和GPS测量值,并通过LK光流法和预积分方法计算得到两组对应的无人机的位置和姿态；根据待测场景的GPS位置坐标,通过八叉树模型对待测场景进行空间划分及编码；对图像序列中的图像进行空间编码比对和筛选,并对筛选的图像进行哈希编码,检测图像间是否存在回环；将回环检测结果即图像间的回环关系与计算的两组无人机位姿进行融合,得到全局一致的无人机位置和姿态。本发明实现了对回环的快速检测,在保持检测准确度的同时提高了检测速度,减少了系统开销,可用于无环境先验信息的场景中进行无人机位姿估计。(The invention discloses a space division-based SLAM rapid loop detection method which comprises the steps of obtaining an image sequence, IMU measurement values and GPS measurement values of a scene to be detected through an unmanned aerial vehicle airborne sensor, and calculating through an LK optical flow method and a pre-integration method to obtain the positions and the postures of two groups of corresponding unmanned aerial vehicles; according to the GPS position coordinates of the scene to be detected, carrying out space division and coding on the scene to be detected through an octree model; carrying out spatial coding comparison and screening on images in the image sequence, carrying out Hash coding on the screened images, and detecting whether a loop exists between the images; and fusing the loop detection result, namely the loop relation between the images, with the calculated positions and postures of the two groups of unmanned aerial vehicles to obtain the globally consistent positions and postures of the unmanned aerial vehicles. The invention realizes the rapid detection of the loop, improves the detection speed while maintaining the detection accuracy, reduces the system overhead, and can be used for unmanned aerial vehicle pose estimation in scenes without environment prior information.)

1. A SLAM quick loop detection method based on space division is characterized by comprising the following steps:

s1, acquiring an image sequence, an IMU (inertial measurement Unit) measurement value and a GPS (global positioning system) measurement value of a scene to be measured through an unmanned aerial vehicle airborne sensor, and calculating a GPS position coordinate;

s2, according to the GPS position coordinates of the scene to be measured, carrying out space division and coding on the scene to be measured through an octree model, and calculating the space code corresponding to each frame of image;

s3, calculating an image sequence of the scene to be detected through an LK optical flow method to obtain the position and the posture of the corresponding unmanned aerial vehicle;

s4, calculating IMU measurement values through a pre-integration method to obtain the corresponding positions and postures of the unmanned aerial vehicles;

s5, selecting a frame of image from the obtained image sequence of the scene to be detected as an image to be detected;

s6, selecting other frame images from the obtained image sequence of the scene to be detected as comparison images, and extracting the spatial codes of the image to be detected and the comparison images according to the calculation result of the step S2;

s7, judging whether the space codes of the image to be detected and the comparison image are consistent or similar, if so, entering the step S8; otherwise, judging that no loop exists between the image to be detected and the comparison image, selecting the next frame of image from the obtained image sequence of the scene to be detected as the image to be detected, and returning to the step S6;

s8, carrying out hash coding on the image to be detected and the comparison image by a perceptual hash method;

s9, judging whether the different encoding bits of the Hash encoding of the image to be detected and the comparison image are smaller than a threshold value, if so, judging that a loop exists between the two frames of images and entering the step S10; otherwise, judging that no loop exists between the image to be detected and the comparison image, selecting the next frame of image from the obtained image sequence of the scene to be detected as the image to be detected, and returning to the step S6;

and S10, fusing the loop detection result, namely the loop relation between the images, with the position and the posture of the unmanned aerial vehicle obtained in the steps S3 and S4 to obtain the globally consistent position and posture of the unmanned aerial vehicle.

2. The spatial partitioning-based SLAM fast loopback detection method as claimed in claim 1, wherein the specific method of step S1 comprises the following sub-steps:

s1-1, acquiring an image sequence, an IMU (inertial measurement Unit) measurement value and a GPS (global positioning system) measurement value of a scene to be measured through an unmanned aerial vehicle airborne sensor, and aligning the IMU measurement value and the GPS measurement value to the image sequence of the scene to be measured according to a timestamp;

s1-2, converting the longitude and latitude information of the aligned GPS measurement value into position information under a northeast coordinate system, and obtaining the global position coordinate of each frame of image, namely the GPS position coordinate.

3. The method of claim 1, wherein the onboard sensors in step S1 include a camera, an IMU sensor and a GPS sensor.

4. The spatial partitioning-based SLAM fast loopback detection method as claimed in claim 1, wherein the specific method of step S2 comprises the following sub-steps:

s2-1, acquiring GPS position coordinates of eight vertexes of the scene to be detected, and determining the boundary of the scene to be detected and the center of the scene to be detected according to the GPS position coordinates of the eight vertexes of the scene to be detected;

s2-2, establishing a three-dimensional space coordinate system by taking the center of the scene to be measured as a coordinate origin and the boundary of the scene to be measured as a coordinate boundary;

s2-3, coding a negative half shaft of the three-dimensional space coordinate system as 0 and coding a positive half shaft of the three-dimensional space coordinate system as 1, completing the first space division of the scene to be detected, and obtaining eight scene subspaces and corresponding space codes;

s2-4, sequentially repeating the step 2-1, the step 2-2 and the step 2-3 to carry out multilayer space division on each scene subspace, generating corresponding space codes, stopping division until the space size is lower than a preset threshold value, and finishing space division and coding of the scene to be detected;

and S2-5, determining the spatial grid to which the acquired image belongs according to the GPS position coordinate corresponding to each frame of image, and selecting the spatial code of the spatial grid as the spatial code corresponding to the image.

5. The SLAM rapid loop detection method based on space division as recited in claim 4, wherein the space of the scene to be detected in step S2-4 is finally divided into grids represented by octree, and each grid is a leaf node in the tree, and each leaf node has a globally unique spatial code.

6. The spatial partitioning-based SLAM fast loopback detection method as claimed in claim 1, wherein the specific method of step S8 comprises the following sub-steps:

s8-1, reducing the sizes of the image to be detected and the contrast image, simplifying the color of the image and finishing the image processing;

s8-2, separating the processed image to be detected and the contrast image into a collection of fractions by a DCT method to obtain a 32 x 32 bit DCT matrix;

s8-3, selecting a matrix of 8 x 8 at the upper left corner of the DCT matrix to obtain low-frequency information of the image;

s8-4, setting DCT8 x 8 matrix to 64-bit hash value of form 0 or 1;

s8-5, combining the 64-bit hash values into 64 information bits to obtain the hash codes of the image to be detected and the comparison image.

7. The SLAM rapid loop detection method based on space division as claimed in claim 6, wherein the specific method of step S8-4 is:

and acquiring the average value of the DCT matrix, traversing the DCT8 x 8 matrix, setting the bit value to be more than or equal to the bit value of the average value of the DCT matrix to be 1, and setting the bit value to be less than the bit value of the average value of the DCT matrix to be 0.

Technical Field

The invention relates to the field of computer vision, in particular to a SLAM quick loop detection method based on space division.

Background

SLAM is known as Simultaneous Localization and Mapping, i.e., Simultaneous Localization and Mapping. The robot equipment is provided with various sensors, and under an unfamiliar environment, the robot equipment estimates the position and the posture of the robot equipment while moving, and reconstructs a model of the surrounding environment. The complete SLAM comprises five modules of sensor information acquisition, visual odometry, loop detection, optimization and mapping. In recent years, the multi-sensor fusion SLAM technology centered on cameras has become the basis of many technologies, such as autonomous robotic navigation, assisted driving, augmented reality, and three-dimensional reconstruction.

Because the SLAM system has accumulated errors when estimating the self pose, the position passing by for multiple times in reality may not coincide in the motion trail constructed by the estimated pose. Loop Detection (Loop Loop Detection) mainly judges whether the robot reaches the same position for multiple times in the moving process, and if the robot passes through a certain point for multiple times, a Loop exists in the moving track of the robot. Because the SLAM system has accumulated errors when estimating the self pose, the position passing by for multiple times in reality may not coincide in the motion trail constructed by the estimated pose. The estimation result can be optimized and corrected by back-end optimization or relocation at this time. At present, two methods are mainly used for detecting loopback, namely a geometric relationship detection method based on an odometer and a geometric relationship detection method based on appearance. Wherein:

odometer-based geometric relationship detection is the detection of loops when the system finds that the camera moves two or more times near a certain location, by the operation of a visual odometer. However, through the above description, it is known that the visual odometer itself causes accumulated errors, and therefore, the method has inherent defects and cannot achieve a good detection effect.

The method and the visual odometer are in two calculation dimensions, and accumulated errors can be eliminated, so that the method becomes a common method for loop detection.

The method is a representative of a geometric relationship detection method based on appearance, and analyzes each image in an image sequence, and highly abstracts characteristic points in the images by using characteristic vectors. In SLAM, a bag-of-words model performs similarity determination by representing each image as a feature word description and comparing the feature words, which are criteria for measuring image similarity. In SLAM, the bag-of-words model is used in two steps: generating a dictionary and carrying out similarity detection by utilizing the dictionary. The method can detect the loop in the map to the maximum extent, but a maximum dictionary file must be loaded before calculation, and the dictionary also needs to be maintained during calculation, so that the problem of huge system overhead exists.

Appearance-based geometric relationship loop detection belongs to the cross research field of computer vision, computer graphics and image processing. How to balance detection speed, precision and system overhead is a major issue to be considered by researchers in the field. For example: (1) the patent application with the application patent number of CN202110028706.9 and the name of 'an outdoor large scene three-dimensional mapping method integrating multiple sensors' is to perform similarity judgment on all images in an image sequence through a bag-of-words model to detect a loop. The method seems to obtain good results, and is very huge in detection speed and memory consumption. Meanwhile, the method directly uses all images in the sequence to judge, so that the algorithm has a great part of energy in processing the area where loop-back is not possible. This not only results in a waste of time, but also results in erroneous detection since these extra areas are likely to cause loop-back mismatch. (2) The patent application with the application patent number of CN110188809A and the name of 'a loop detection method based on image blocks' is to detect loops by taking the image blocks as grids, extracting the characteristic vector of each grid, training word bags and then carrying out image similarity judgment. The method seems to be faster and more accurate than the traditional bag-of-words method, and actually does not fundamentally avoid the expenses of bag-of-words training and bag-of-words maintenance, and the method still needs to carry out feature calculation and judgment on all images in the sequence, and has huge consumption on detection speed and memory. This is not the case in the currently published algorithms for loop detection based on appearance. Therefore, how to reduce the system overhead while improving the detection speed and accuracy and how to make the detection algorithm more "focus" on the area where loop-back may occur is a problem that researchers of the loop-back detection algorithm should urgently solve at present.

Disclosure of Invention

Aiming at the defects in the prior art, the SLAM quick loop detection method based on space division provided by the invention solves the problems of low detection speed, poor precision and high system overhead in the traditional technology.

In order to achieve the purpose of the invention, the invention adopts the technical scheme that:

the method for detecting the rapid loop of the SLAM based on the space division comprises the following steps:

s1, acquiring an image sequence, an IMU (inertial measurement Unit) measurement value and a GPS (global positioning system) measurement value of a scene to be measured through an unmanned aerial vehicle airborne sensor, and calculating a GPS position coordinate;

s2, according to the GPS position coordinates of the scene to be measured, carrying out space division and coding on the scene to be measured through an octree model, and calculating the space code corresponding to each frame of image;

s3, calculating an image sequence of the scene to be detected through an LK optical flow method to obtain the position and the posture of the corresponding unmanned aerial vehicle;

s4, calculating IMU measurement values through a pre-integration method to obtain the corresponding positions and postures of the unmanned aerial vehicles;

s5, selecting a frame of image from the obtained image sequence of the scene to be detected as an image to be detected;

s6, selecting other frame images from the obtained image sequence of the scene to be detected as comparison images, and extracting the spatial codes of the image to be detected and the comparison images according to the calculation result of the step S2;

s7, judging whether the space codes of the image to be detected and the comparison image are consistent or similar, if so, entering the step S8; otherwise, judging that no loop exists between the image to be detected and the comparison image, selecting the next frame of image from the obtained image sequence of the scene to be detected as the image to be detected, and returning to the step S6;

s8, carrying out hash coding on the image to be detected and the comparison image by a perceptual hash method;

s9, judging whether the different encoding bits of the Hash encoding of the image to be detected and the comparison image are smaller than a threshold value, if so, judging that a loop exists between the two frames of images and entering the step S10; otherwise, judging that no loop exists between the image to be detected and the comparison image, selecting the next frame of image from the obtained image sequence of the scene to be detected as the image to be detected, and returning to the step S6;

and S10, fusing the loop detection result, namely the loop relation between the images, with the position and the posture of the unmanned aerial vehicle obtained in the steps S3 and S4 to obtain the globally consistent position and posture of the unmanned aerial vehicle.

Further, the specific method of step S1 includes the following sub-steps:

s1-1, acquiring an image sequence, an IMU (inertial measurement Unit) measurement value and a GPS (global positioning system) measurement value of a scene to be measured through an unmanned aerial vehicle airborne sensor, and aligning the IMU measurement value and the GPS measurement value to the image sequence of the scene to be measured according to a timestamp;

s1-2, converting the longitude and latitude information of the aligned GPS measurement value into position information under a northeast coordinate system, and obtaining the global position coordinate of each frame of image, namely the GPS position coordinate.

Further, the onboard sensors in step S1 include a camera, an IMU sensor, and a GPS sensor.

Further, the specific method of step S2 includes the following sub-steps:

s2-1, acquiring GPS position coordinates of eight vertexes of the scene to be detected, and determining the boundary of the scene to be detected and the center of the scene to be detected according to the GPS position coordinates of the eight vertexes of the scene to be detected;

s2-2, establishing a three-dimensional space coordinate system by taking the center of the scene to be measured as a coordinate origin and the boundary of the scene to be measured as a coordinate boundary;

s2-3, coding a negative half shaft of the three-dimensional space coordinate system as 0 and coding a positive half shaft of the three-dimensional space coordinate system as 1, completing the first space division of the scene to be detected, and obtaining eight scene subspaces and corresponding space codes;

s2-4, sequentially repeating the step 2-1, the step 2-2 and the step 2-3 to carry out multilayer space division on each scene subspace, generating corresponding space codes, stopping division until the space size is lower than a preset threshold value, and finishing space division and coding of the scene to be detected;

and S2-5, determining the spatial grid to which the acquired image belongs according to the GPS position coordinate corresponding to each frame of image, and selecting the spatial code of the spatial grid as the spatial code corresponding to the image.

Further, the space of the scene to be measured in step S2-4 is finally divided into grids represented by octree, and each grid is a leaf node in the tree, and each leaf node has a globally unique spatial code.

Further, the specific method of step S8 includes the following sub-steps:

s8-1, reducing the sizes of the image to be detected and the contrast image, simplifying the color of the image and finishing the image processing;

s8-2, separating the processed image to be detected and the contrast image into a collection of fractions by a DCT method to obtain a 32 x 32 bit DCT matrix;

s8-3, selecting a matrix of 8 x 8 at the upper left corner of the DCT matrix to obtain low-frequency information of the image;

s8-4, setting DCT8 x 8 matrix to 64-bit hash value of form 0 or 1;

s8-5, combining the 64-bit hash values into 64 information bits to obtain the hash codes of the image to be detected and the comparison image.

Further, the specific method of step S8-4 is:

and acquiring the average value of the DCT matrix, traversing the DCT8 x 8 matrix, setting the bit value to be more than or equal to the bit value of the average value of the DCT matrix to be 1, and setting the bit value to be less than the bit value of the average value of the DCT matrix to be 0.

The invention has the beneficial effects that:

1. when SLAM is carried out on a target scene, the scene area is determined and the octree of the area is divided through the GPS information provided by the unmanned aerial vehicle-mounted GPS sensor, the area space code is dynamically generated, and when SLAM loop detection is carried out, images with similar space codes are only needed to be compared, so that the loop detection flow and the calculated amount are simplified, unnecessary loop redundancy is removed, the loop detection speed can be increased, and the detection precision can be improved.

2. The invention provides a rapid loop detection method based on space division for loop detection, which only needs to judge the similarity of the image in the grid adjacent to the current image to determine whether a loop is generated, and a bag-of-words dictionary does not need to be loaded and maintained in the SLAM process, thereby greatly saving the memory overhead and the running time of the loop detection.

Drawings

FIG. 1 is a flow chart of the present invention.

Detailed Description

The following description of the embodiments of the present invention is provided to facilitate the understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and it will be apparent to those skilled in the art that various changes may be made without departing from the spirit and scope of the invention as defined and defined in the appended claims, and all matters produced by the invention using the inventive concept are protected.

As shown in fig. 1, the method for detecting SLAM fast loop based on space division includes the following steps:

s1, acquiring an image sequence, an IMU (inertial measurement Unit) measurement value and a GPS (global positioning system) measurement value of a scene to be measured through an unmanned aerial vehicle airborne sensor, and calculating a GPS position coordinate;

s2, according to the GPS position coordinates of the scene to be measured, carrying out space division and coding on the scene to be measured through an octree model, and calculating the space code corresponding to each frame of image;

s3, calculating an image sequence of the scene to be detected through an LK optical flow method to obtain the position and the posture of the corresponding unmanned aerial vehicle;

s4, calculating IMU measurement values through a pre-integration method to obtain the corresponding positions and postures of the unmanned aerial vehicles;

s5, selecting a frame of image from the obtained image sequence of the scene to be detected as an image to be detected;

s6, selecting other frame images from the obtained image sequence of the scene to be detected as comparison images, and extracting the spatial codes of the image to be detected and the comparison images according to the calculation result of the step S2;

s7, judging whether the space codes of the image to be detected and the comparison image are consistent or similar, if so, entering the step S8; otherwise, judging that no loop exists between the image to be detected and the comparison image, selecting the next frame of image from the obtained image sequence of the scene to be detected as the image to be detected, and returning to the step S6;

s8, carrying out hash coding on the image to be detected and the comparison image by a perceptual hash method;

s9, judging whether the different encoding bits of the Hash encoding of the image to be detected and the comparison image are smaller than a threshold value, if so, judging that a loop exists between the two frames of images and entering the step S10; otherwise, judging that no loop exists between the image to be detected and the comparison image, selecting the next frame of image from the obtained image sequence of the scene to be detected as the image to be detected, and returning to the step S6;

and S10, fusing the loop detection result, namely the loop relation between the images, with the position and the posture of the unmanned aerial vehicle obtained in the steps S3 and S4 to obtain the globally consistent position and posture of the unmanned aerial vehicle.

The specific method of step S1 includes the following substeps:

s1-1, acquiring an image sequence, an IMU (inertial measurement Unit) measurement value and a GPS (global positioning system) measurement value of a scene to be measured through an unmanned aerial vehicle airborne sensor, and aligning the IMU measurement value and the GPS measurement value to the image sequence of the scene to be measured according to a timestamp;

s1-2, converting the longitude and latitude information of the aligned GPS measurement value into position information under a northeast coordinate system, and obtaining the global position coordinate of each frame of image, namely the GPS position coordinate.

The onboard sensors in step S1 include a camera, an IMU sensor, and a GPS sensor.

The specific method of step S2 includes the following substeps:

s2-1, acquiring GPS position coordinates of eight vertexes of the scene to be detected, and determining the boundary of the scene to be detected and the center of the scene to be detected according to the GPS position coordinates of the eight vertexes of the scene to be detected;

s2-2, establishing a three-dimensional space coordinate system by taking the center of the scene to be measured as a coordinate origin and the boundary of the scene to be measured as a coordinate boundary;

s2-3, coding a negative half shaft of the three-dimensional space coordinate system as 0 and coding a positive half shaft of the three-dimensional space coordinate system as 1, completing the first space division of the scene to be detected, and obtaining eight scene subspaces and corresponding space codes;

s2-4, sequentially repeating the step 2-1, the step 2-2 and the step 2-3 to carry out multilayer space division on each scene subspace, generating corresponding space codes, stopping division until the space size is lower than a preset threshold value, and finishing space division and coding of the scene to be detected;

and S2-5, determining the spatial grid to which the acquired image belongs according to the GPS position coordinate corresponding to each frame of image, and selecting the spatial code of the spatial grid as the spatial code corresponding to the image.

The space of the scene to be measured in step S2-4 is finally divided into grids represented by octree, and each grid is a leaf node in the tree, and each leaf node has a globally unique spatial code.

The specific method of step S8 includes the following substeps:

s8-1, reducing the sizes of the image to be detected and the contrast image, simplifying the color of the image and finishing the image processing;

s8-2, separating the processed image to be detected and the contrast image into a collection of fractions by a DCT method to obtain a 32 x 32 bit DCT matrix;

s8-3, selecting a matrix of 8 x 8 at the upper left corner of the DCT matrix to obtain low-frequency information of the image;

s8-4, setting DCT8 x 8 matrix to 64-bit hash value of form 0 or 1;

s8-5, combining the 64-bit hash values into 64 information bits to obtain the hash codes of the image to be detected and the comparison image.

The specific method of step S8-4 is as follows:

and acquiring the average value of the DCT matrix, traversing the DCT8 x 8 matrix, setting the bit value to be more than or equal to the bit value of the average value of the DCT matrix to be 1, and setting the bit value to be less than the bit value of the average value of the DCT matrix to be 0.

In one embodiment of the invention, a loop detection validation is performed using the EuRoC dataset, run on a desktop computer configured with Intel (R) core (TM) i7-8700 CPU, 8GB memory, and compare performance on the same machine with the most popular bag of words DBoW2 algorithm in recent years. In the experiment, 8 sequences with different difficulties and different loop numbers in the EuRoC data set are selected to carry out loop detection experiments, and the loop detection performance of the system is measured by two indexes.

As shown in table 1, the run-time performance of the present invention was compared to DBoW2 on desktop computers, specifically the average time taken for the present invention to check the previous 300 loops with DBoW 2. The table shows that a large amount of calculation time is consumed because all images need to be processed by the DBoW2, and the image similarity comparison is only carried out on the areas where loop loops possibly exist through image pre-screening, so that the loop detection speed is greatly improved, and the expenses of training and maintaining word bags are avoided.

TABLE 1

As shown in table 2, the location performance of the present invention was compared with DBoW2, and the accuracy of SLAM fast loop estimation location using the present invention was evaluated using a EuRoC dataset. Where "DBoW 2" represents the median absolute track error of all frames in a SLAM system using DBoW2, and "the present invention" represents the median absolute track error of all frames in a SLAM fast loop estimation using the present invention. As can be seen from the table, because the image similarity comparison is performed only on the areas where the loop may exist through image pre-screening, the loop mismatching is reduced, and the track error is reduced. Therefore, the median absolute track error of all frames in the SLAM fast loop estimation applying the invention is better than that of all frames in the SLAM system applying DBoW 2.

TABLE 2

When SLAM is carried out on a target scene, the scene area is determined and the octree of the area is divided through the GPS information provided by the unmanned aerial vehicle-mounted GPS sensor, the area space code is dynamically generated, and when SLAM loop detection is carried out, images with similar space codes are only needed to be compared, so that the loop detection flow and the calculated amount are simplified, unnecessary loop redundancy is removed, the loop detection speed can be increased, and the detection precision can be improved. The invention provides a rapid loop detection method based on space division for loop detection, which only needs to judge the similarity of the image in the grid adjacent to the current image to determine whether a loop is generated, and a bag-of-words dictionary does not need to be loaded and maintained in the SLAM process, thereby greatly saving the memory overhead and the running time of the loop detection.