阅读说明：本技术 基于视觉slam的地图构建的方法、导航系统及装置 (Map construction method based on visual SLAM, navigation system and device ) 是由 许登科 肖刚军 赵伟兵 于 2019-09-26 设计创作，主要内容包括：本发明公开一种基于视觉SLAM的地图构建的方法,其中,所述地图构建方法包括以下步骤：获取当前位置视屏图像数据与目标位置视屏图像数据；识别所述当前位置视屏图像数据与所述目标位置视屏图像数据的点云数据,并获取各所述当前视屏图像数据的点云与所述目标视屏图像数据的点云之间的距离；根据各所述点云之间的距离构建点云地图,以便于移动机器人的定位、避障、自主导航以及三维重建和人机交互。本发明还公开了一种基于视觉SLAM及网络地图结合的导航系统和具有存储功能的装置。(The invention discloses a map construction method based on visual SLAM, wherein the map construction method comprises the following steps: acquiring current position view screen image data and target position view screen image data; identifying point cloud data of the current position view screen image data and the target position view screen image data, and acquiring the distance between the point cloud of each current view screen image data and the point cloud of the target view screen image data; and constructing a point cloud map according to the distance between the point clouds so as to facilitate the positioning, obstacle avoidance, autonomous navigation, three-dimensional reconstruction and human-computer interaction of the mobile robot. The invention also discloses a navigation system based on the combination of the visual SLAM and the network map and a device with a storage function.)

1. A map construction method based on visual SLAM is characterized by comprising the following steps:

acquiring current position view screen image data and target position view screen image data;

identifying point cloud data of the current position view screen image data and the target position view screen image data, and acquiring the distance between the point cloud of each current view screen image data and the point cloud of the target view screen image data;

and constructing a point cloud map according to the distance between the point clouds.

2. The method of visual SLAM-based mapping of claim 1, wherein the step of constructing a point cloud map as a function of distance between each of the point clouds is followed by:

acquiring loop detection data;

and adjusting the point cloud map according to the loopback detection data.

3. The visual SLAM-based mapping method of claim 1, wherein the step of obtaining current location view screen image data and target location view screen image data is followed by:

and acquiring a relative motion point cloud according to the current position view screen image data and the target position view screen image data.

4. The method of visual SLAM-based mapping of claim 3, wherein said step of obtaining a point cloud of relative motion from said current position view screen image data and said target position view screen image data comprises:

acquiring point characteristics and line characteristics of the current position and the target position according to the current position video screen image data and the target position video screen image data;

and obtaining a relative motion point cloud according to the obtained point characteristics and line characteristics.

5. The visual SLAM-based mapping method of claim 4, wherein the step of obtaining point and line features for a current location and a target location from the current location view screen image data and the target location view screen image data comprises:

selecting a key frame according to the current position video image data and the target position video image data;

and extracting point features and line features according to the key frames.

6. The method of visual SLAM-based mapping of claim 5, wherein the step of constructing a point cloud map as a function of distance between each of the point clouds comprises:

and constructing a motion track according to the extracted point features and line features.

7. The method of visual SLAM-based mapping of claim 5, wherein said step of selecting key frames from said current location view screen image data and said target location view screen image data is followed by:

and when the number of the feature blocks on the new image frame is less than a certain threshold value, the image frame is taken as a new key frame.

8. The visual SLAM-based mapping method of claim 1, wherein the step of identifying point cloud data of the current location view screen image data and the target location view screen image data, and obtaining a distance between a point cloud of each of the current view screen image data and a point cloud of the target view screen image data comprises:

acquiring inertial movement data;

and calculating the distance between the visual calibration points according to the current position video screen image data, the target position video screen image data, the inertial movement data and the relative motion point cloud.

9. A navigation system based on combination of visual SLAM and a network map is characterized in that the navigation system comprises: the map construction method based on the visual SLAM comprises a data communication processing module, a visual collection module, an identification module, a distance acquisition module and a position estimation module, wherein the data communication processing module is respectively connected with the visual collection module, the identification module, the distance acquisition module and the position estimation module, and the visual collection module, the identification module, the distance acquisition module and the position estimation module can realize the steps in the map construction method based on the visual SLAM according to any one of claims 1-8 when in work.

10. An apparatus having a storage function, on which program data is stored, wherein the program data, when executed by a processor, implements the steps in the method for visual SLAM-based mapping according to any of claims 1-8.

Technical Field

The invention relates to the field of intelligent robots, in particular to a map construction method based on visual SLAM, a navigation system and a device.

Background

Simultaneous Localization and Mapping (SLAM) is an important way to realize intelligent perception of a robot, and mainly solves the problem that in an unknown environment, a mobile robot acquires environment information through a sensor carried by the mobile robot, continuously estimates the pose of the mobile robot in the motion process, and creates a map for the surrounding environment, that is, SLAM is mainly used for solving the 'Localization' and 'map creation' of the mobile robot, the mobile robot needs to know the current position (Localization) of the mobile robot on one hand and needs to create a map (map creation) of the surrounding environment on the other hand, wherein SLAM acquiring the environment information only through a camera is called visual SLAM, the most basic function of a three-dimensional map created by the visual SLAM is to be the navigation of the robot and the path planning and obstacle avoidance when the robot executes tasks, however, current visual SLAM-created maps are not well suited for mobile robots to understand the environment and perform tasks, let alone to assist humans in understanding unknown environments.

Disclosure of Invention

The invention mainly aims to provide a map construction method based on visual SLAM, aiming at improving the intelligent levels of positioning, obstacle avoidance and autonomous navigation of a mobile robot. The specific scheme is as follows:

a method of visual SLAM-based mapping, the mapping method comprising the steps of: acquiring current position view screen image data and target position view screen image data; identifying point cloud data of the current position view screen image data and the target position view screen image data, and acquiring the distance between the point cloud of each current view screen image data and the point cloud of the target view screen image data; and constructing a point cloud map according to the distance between the point clouds.

Optionally, the step of constructing a point cloud map according to the distance between the point clouds comprises: acquiring loop detection data; and adjusting the point cloud map according to the loopback detection data.

Optionally, the step of acquiring the current position view screen image data and the target position view screen image data includes: and acquiring a relative motion point cloud according to the current position view screen image data and the target position view screen image data.

Optionally, the step of obtaining a point cloud of relative motion according to the current position view screen image data and the target position view screen image data includes: acquiring point characteristics and line characteristics of the current position and the target position according to the current position video screen image data and the target position video screen image data; and obtaining a relative motion point cloud according to the obtained point characteristics and line characteristics.

Optionally, the step of obtaining the point feature and the line feature of the current position and the target position according to the current position view screen image data and the target position view screen image data includes: selecting a key frame according to the current position video image data and the target position video image data; and extracting point features and line features according to the key frames.

Optionally, the step of constructing a point cloud map according to the distance between the point clouds includes: and constructing a motion track according to the extracted point features and line features.

Optionally, the step of selecting a key frame according to the current position video image data and the target position video image data includes: and when the number of the feature blocks on the new image frame is less than a certain threshold value, the image frame is taken as a new key frame.

Optionally, the step of identifying point cloud data of the current position view screen image data and the target position view screen image data, and obtaining a distance between a point cloud of each current view screen image data and a point cloud of the target position view screen image data includes: acquiring inertial movement data; and calculating the distance between the visual calibration points according to the current position video screen image data, the target position video screen image data, the inertial movement data and the relative motion point cloud.

A navigation system based on a combination of visual SLAM and network map, the navigation system comprising: the map construction method based on the visual SLAM comprises a data communication processing module, a visual collection module, an identification module, a distance acquisition module and a position estimation module, wherein the data communication processing module is respectively connected with the visual collection module, the identification module, the distance acquisition module and the position estimation module, and the visual collection module, the identification module, the distance acquisition module and the position estimation module can realize the steps of the map construction method based on the visual SLAM when in work.

An apparatus having a storage function, having stored thereon program data which, when executed by a processor, implements the steps in the method of visual SLAM-based map construction as described above.

In the technical scheme of the invention, in the synchronous positioning and map construction operation process based on vision, the visual screen image data of the current position and the visual screen image data of the target position are obtained, the point cloud data of the visual screen image data of the current position and the visual screen image data of the target position are identified, the distance between the point cloud of each piece of current visual screen image data and the point cloud of the target visual screen image data is obtained, the distance and the direction between the current position and the target position are further calculated, and a point cloud map is constructed, so that the positioning, obstacle avoidance, autonomous navigation, three-dimensional reconstruction and human-computer interaction of a mobile robot are facilitated.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the structures shown in the drawings without creative efforts.

Fig. 1 is a schematic terminal structure diagram of a hardware operating environment according to an embodiment of the present invention;

FIG. 2 is a schematic flow chart of a first embodiment of a method for visual SLAM-based mapping according to the present invention;

FIG. 3 is a flowchart illustrating a second embodiment of a method for visual SLAM-based mapping according to the present invention;

FIG. 4 is a flowchart illustrating a third embodiment of a method for visual SLAM-based mapping according to the present invention;

FIG. 5 is a flowchart illustrating a fourth embodiment of the method for visual SLAM-based mapping according to the present invention;

FIG. 6 is a flow chart of a fifth embodiment of the method for visual SLAM-based mapping according to the present invention;

FIG. 7 is a flowchart illustrating a sixth embodiment of a method for visual SLAM-based mapping according to the present invention;

FIG. 8 is a flowchart illustrating a seventh embodiment of a method for visual SLAM-based mapping according to the present invention;

fig. 9 is a flowchart illustrating a method for constructing a map based on visual SLAM according to an eighth embodiment of the present invention.

The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The main solution of the embodiment of the invention is as follows: in the synchronous positioning and map building operation process based on vision, by acquiring the view screen image data of the current position and the view screen image data of the target position, identifying the point cloud data of the view screen image data of the current position and the view screen image data of the target position, acquiring the distance between the point cloud of each current view screen image data and the point cloud of the target view screen image data, further calculating the distance and the direction between the current position and the target position, and building a point cloud map so as to facilitate the positioning, obstacle avoidance, autonomous navigation, three-dimensional reconstruction and human-computer interaction of the mobile robot.

Since in the prior art, a mobile robot needs to know where on the one hand it is currently located on a map (positioning), and on the other hand needs to create a map of the surrounding environment (map creation), where a SLAM that acquires only environmental information through a camera is called a visual SLAM, the most basic functions of a three-dimensional map created by the visual SLAM should be navigation of the robot and path planning and obstacle avoidance when the robot performs tasks, however, the map created by the current visual SLAM is not well used for the mobile robot to understand the environment and perform tasks, let alone help the human to understand the unknown environment.

The invention provides a solution, which comprises the steps of obtaining view screen image data of a current position and view screen image data of a target position, identifying point cloud data of the view screen image data of the current position and the view screen image data of the target position, obtaining the distance between the point cloud of each current view screen image data and the point cloud of the target view screen image data, further calculating the distance and the direction between the current position and the target position, and constructing a point cloud map so as to facilitate the positioning, obstacle avoidance, autonomous navigation, three-dimensional reconstruction and human-computer interaction of a mobile robot.

As shown in fig. 1, fig. 1 is a schematic diagram of a hardware operating environment of a terminal according to an embodiment of the present invention.

As shown in fig. 1, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display (Display), an input unit such as a Keyboard (Keyboard), a remote controller, and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (such as a non-volatile memory, e.g., disk storage) and the memory 1005 may optionally also be a storage device separate from the processor 1001.

Those skilled in the art will appreciate that the configuration of the terminal shown in fig. 1 is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.

As shown in fig. 1, a memory 1005, which is one of computer storage media, may include therein a control program of an operating system, a network communication module, a user interface module, and a method of map construction based on visual SLAM.

In the terminal shown in fig. 1, the network interface 1004 is mainly used for connecting to a backend server and performing data communication with the backend server; the user interface 1003 is mainly used for connecting a client (user side) and performing data communication with the client; and the processor 1001 may be configured to call a control program of the method of visual SLAM-based map construction stored in the memory 1005, and perform the following operations: and when an operation execution instruction triggered by a preset control is received, acquiring an operation path corresponding to the preset control.

Further, the processor 1001 may call a control program of the visual SLAM-based map construction method stored in the memory 1005, and also perform the following operations.

The invention provides a map construction method based on visual SLAM.

Referring to fig. 2, fig. 2 is a schematic flowchart of a first embodiment of a mapping method based on visual SLAM according to the present invention.

The embodiment provides a method for constructing a map based on a visual SLAM, which comprises the following steps: step S10, acquiring the current position view screen image data and the target position view screen image data; step S20, identifying point cloud data of the current position view screen image data and the target position view screen image data, and acquiring the distance between the point cloud of each current view screen image data and the point cloud of the target view screen image data; step S30, a point cloud map is constructed according to the distance between the point clouds.

In this embodiment, the target position is a position to be reached by the robot, wherein the shape of the object at the target position includes, but is not limited to, any one or a combination of squares, circles, triangles, or diamonds. And shooting the video image data of the current position and the video image data of the target position through a camera.

The processor performs current position calibration according to the view screen image data of the current position, performs target position calibration according to the view screen image data of the target position, integrates the current position calibration point of the current position and the target position calibration point to form point cloud data, and can acquire the distance between any two visual calibration points after the processor identifies the visual calibration points in the image data.

Since the internal parameters (focal length and distortion parameters) of the camera are known, the processor identifies more than three visual calibration points according to the positions of the identified visual calibration points in the shot image, and the distance and the direction of the camera from the target position can be directly calculated by using a computational vision algorithm.

Specifically, the target position is taken as a coordinate system, the long side is an X axis, the short side is a Y axis, and the gravity axis is a Z axis. The positions of the visual calibration points a, B, C, D in the world coordinate system are denoted as (x1, y1, z1), …, (x4, y4, z4), the positions in the photo are denoted as (u1, v1), …, (u4, v4), two groups of points and camera internal parameters are used as input, the distance and orientation of the camera can be calculated by using solvePnP function under caliib 3D module in OpenCV open source algorithm, the problem is that the Perspective-n-Point problem which has been solved in the field of vision is calculated, which is a technical means which is already known by those skilled in the art, and is not detailed herein.

The technical scheme of the embodiment realizes the supplement and the improvement of the visual SLAM, can create the cognitive point cloud map of the environment, and can be used for realizing the positioning, obstacle avoidance, autonomous navigation, three-dimensional reconstruction and man-machine interaction of the mobile robot.

Further, referring to fig. 3, a second embodiment of the present invention is proposed based on the first embodiment, in this embodiment, the step of constructing the point cloud map according to the distance between the point clouds includes: step S40, loop detection data is obtained; and step S50, adjusting the point cloud map according to the loopback detection data.

The technical scheme of the embodiment realizes the supplement and the improvement of the visual SLAM, can calibrate the cognitive point cloud map, and increases the accuracy of the motion of the mobile robot.

Further, referring to fig. 4, a third embodiment of the present invention is proposed based on the foregoing embodiment, in this embodiment, the step of acquiring the current position view screen image data and the target position view screen image data includes: and step S60, acquiring a relative motion point cloud according to the current position view screen image data and the target position view screen image data.

According to the technical scheme, the point cloud processing of the image data is realized, and the processed point cloud information contains a direct geometric relation, so that the path planning and navigation of the robot become visual.

Further, referring to fig. 5, a fourth embodiment of the present invention is proposed based on the above embodiment, in this embodiment, the step of obtaining a relative motion point cloud according to the current position view screen image data and the target position view screen image data includes: step S61, acquiring point characteristics and line characteristics of the current position and the target position according to the current position video screen image data and the target position video screen image data; and step S62, obtaining a relative motion point cloud according to the obtained point characteristics and line characteristics.

The technical scheme of the embodiment realizes the supplement and the improvement of the visual SLAM, can create the cognitive point cloud map of the environment, and can be used for realizing the positioning, obstacle avoidance, autonomous navigation, three-dimensional reconstruction and man-machine interaction of the mobile robot.

Further, referring to fig. 6, a fifth embodiment of the present invention is proposed based on the above-mentioned embodiments, and in this embodiment, the step of obtaining the point feature and the line feature of the current position and the target position according to the current position view screen image data and the target position view screen image data includes: step S611, selecting key frames according to the current position video image data and the target position video image data; step S612, extracting point features and line features according to the key frame.

The technical scheme of the embodiment realizes the supplement and the improvement of the visual SLAM, can create the cognitive point cloud map of the environment, and can be used for realizing the positioning, obstacle avoidance, autonomous navigation, three-dimensional reconstruction and man-machine interaction of the mobile robot.

Further, referring to fig. 7, a sixth embodiment of the present invention is proposed based on the above embodiment, in this embodiment, the step of constructing the point cloud map according to the distance between the point clouds includes: and step S31, constructing a motion track according to the extracted point features and line features.

The technical scheme of the embodiment realizes the supplement and the improvement of the visual SLAM, can create the cognitive point cloud map of the environment, and can be used for realizing the positioning, obstacle avoidance, autonomous navigation, three-dimensional reconstruction and man-machine interaction of the mobile robot.

Further, referring to fig. 8, a seventh embodiment of the present invention is proposed based on the above embodiment, in this embodiment, after the step of selecting a key frame according to the current position view screen image data and the target position view screen image data, the method further includes: in step S613, when the number of feature blocks in the new image frame is less than a certain threshold, the new image frame is used as a new key frame.

The technical scheme of the embodiment realizes the supplement and the improvement of the visual SLAM, can create the cognitive point cloud map of the environment, and can be used for realizing the positioning, obstacle avoidance, autonomous navigation, three-dimensional reconstruction and man-machine interaction of the mobile robot.

Further, referring to fig. 9, an eighth embodiment of the present invention is provided based on the above embodiment, and in this embodiment, the step of identifying the point cloud data of the current position view screen image data and the target position view screen image data, and acquiring the distance between the point cloud of each current view screen image data and the point cloud of the target position view screen image data includes: step S21, acquiring inertial movement data; step S22, calculating the distance between each visual calibration point according to the current position video screen image data, the target position video screen image data, the inertial movement data and the relative motion point cloud.

The technical scheme of the embodiment realizes the supplement and the improvement of the visual SLAM, can create the cognitive point cloud map of the environment, and can be used for realizing the positioning, obstacle avoidance, autonomous navigation, three-dimensional reconstruction and man-machine interaction of the mobile robot.

The invention also provides a navigation system based on the combination of the visual SLAM and the network map, and the navigation system comprises: the map construction method based on the visual SLAM comprises a data communication processing module, a visual collection module, an identification module, a distance acquisition module and a position estimation module, wherein the data communication processing module is respectively connected with the visual collection module, the identification module, the distance acquisition module and the position estimation module, and the visual collection module, the identification module, the distance acquisition module and the position estimation module can realize the steps of the map construction method based on the visual SLAM when in work.

The present invention also provides a device having a storage function, the device having stored thereon program data, which when executed by a processor, implements the steps in the method for visual SLAM-based mapping in any of the embodiments.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or processor that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or processor. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or processor that comprises the element.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., lcd tv, lcd display, etc.) to execute the method according to the embodiments of the present invention.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and all modifications and equivalents of the present invention, which are made by the contents of the present specification and the accompanying drawings, or directly/indirectly applied to other related technical fields, are included in the scope of the present invention.