阅读说明：本技术 一种极地冰层地图构建系统及其工作方法 (Polar region ice layer map construction system and working method thereof ) 是由 周利 刘浩 丁仕风 闯振菊 赵晓杰 高俊亮 郑思洁 于 2021-09-23 设计创作，主要内容包括：本发明涉及一种极地冰层地图的构建系统,包括激光SLAM模块、视觉SLAM模块、中央处理器以及语义模块。在某一时刻t,激光SLAM模块用来实时采集当前空间地理位置的冰层环境地图以及捕捉当前空间地理位置坐标信息。在时刻t,视觉SLAM模块用来实时采集当前空间地理位置的冰层环境地图以及捕捉当前空间地理位置坐标信息。中央处理器与激光SLAM模块和视觉SLAM模块均进行电连接。协同语义模块,中央处理器根据语义指令对位于船舶四周的冰层进行分类,以实时地生成具有语义信息的、高精度、且附带语义信息的全局冰层代价地图。另外,本发明还公开了一种极地冰层地图构建系统的工作方法。(The invention relates to a polar region ice layer map construction system which comprises a laser SLAM module, a visual SLAM module, a central processing unit and a semantic module. At a certain time t, the laser SLAM module is used for acquiring an ice layer environment map of the current space geographic position in real time and capturing coordinate information of the current space geographic position. At time t, the visual SLAM module is used for acquiring an ice layer environment map of the current spatial geographic position in real time and capturing coordinate information of the current spatial geographic position. The central processing unit is electrically connected with the laser SLAM module and the vision SLAM module. And the central processing unit is used for classifying the ice layers around the ship according to the semantic instruction so as to generate a global ice layer cost map with semantic information, high precision and attached semantic information in real time. In addition, the invention also discloses a working method of the polar region ice layer map construction system.)

1. A polar region ice layer map construction system is characterized by comprising at least one laser SLAM module, at least one visual SLAM module, a central processing unit and a semantic module; the system comprises a laser SLAM module, a first positioning unit and a second positioning unit, wherein the laser SLAM module comprises a first computing unit, a laser radar, a first environment map information acquisition unit, a first positioning unit and a first data transmission unit; at a certain moment t, the first environment map information acquisition unit cooperates with the laser radar to acquire an ice layer environment map of the current spatial geographic position in real time, and meanwhile, the first positioning unit is used for capturing coordinate information of the current spatial geographic position in real time; the first environment map information acquisition unit and the first positioning unit transmit data to the first calculation unit through the first data transmission unit; the vision SLAM module comprises a second computing unit, a vision sensor, a second environment map information acquisition unit, a second positioning unit and a second data transmission unit; at time t, the second environment map information acquisition unit cooperates with the visual sensor to acquire the ice layer environment map of the current spatial geographic position in real time, and meanwhile, the second positioning unit is used for capturing the coordinate information of the current spatial geographic position in real time; the second environment map information acquisition unit and the second positioning unit transmit data to the second computing unit through the second data transmission unit; the central processing unit is used for receiving and processing data and is electrically connected with the first computing unit and the second computing unit; and in cooperation with the semantic module, the central processing unit classifies the ice layers around the ship according to the semantic instructions to generate a global ice layer cost map.

2. The system for constructing the polar ice layer map according to claim 1, wherein the first positioning unit and the second positioning unit are both composed of a GNSS positioning unit and a GPS positioning unit.

3. The polar region ice layer map building system according to claim 1, wherein the central processor comprises an information correction module.

4. The polar region ice layer map building system according to claim 1, wherein the central processor is a desktop computer or a notebook computer.

5. The polar ice layer map building system according to claim 1, wherein the first data transmission unit and the second data transmission unit are both optical fiber transmission units.

6. The polar ice layer map building system according to claim 1, further comprising a back-end optimization module; the rear end optimization module is matched with the central processing unit; and performing loop data correction on the ice layer environment map of the current spatial geographical position acquired through the first environment map information and the ice layer environment map of the current spatial geographical position acquired through the second environment map information, which have the same spatial geographical position coordinate information.

7. A working method of a polar region ice layer map construction system attached to the polar region ice layer map construction system of any one of claims 1-6, comprising the following steps:

a) at a certain time t, the laser SLAM module captures coordinate information omega 1 of the current spatial geographic position and an ice layer environment map beta 1;

b) at the same time t, the visual SLAM module captures coordinate information omega 2 of the current spatial geographic position and an ice layer environment map beta 2; the coordinate information omega 1 of the spatial geographical position is completely the same as the coordinate information omega 2 of the spatial geographical position, and the ice layer environment map beta 1 and the ice layer environment map beta 2 have the same timestamp;

c) the coordinate information omega 1 of the space geographic position, the ice layer environment map beta 1, the coordinate information omega 2 of the space geographic position and the ice layer environment map beta 2 are transmitted to the central processor in a unified mode;

d) fusing the ice layer environment map beta 1 and the ice layer environment map beta 2 through the central processing unit to generate a real-time global environment map;

e) the semantic module is used for classifying the obstacles on the ice layer stored on the global environment map;

f) and generating a real-time global cost map with semantic information according to different ice breaking losses of different types of obstacles.

8. The working method of the polar region ice layer map construction system according to claim 7, wherein the central processor needs to correct the received spatial geographical position information of the ice layer environment map β 1 and the ice layer environment map β 2 before determining the relative position relationship between all local ice layer environment maps with the same timestamp according to the respective spatial geographical position information of the laser SLAM module and the visual SLAM module.

9. The working method of the polar region ice layer map construction system according to claim 8, wherein a least square fitting formula is adopted to correct the spatial geographic position information of the ice layer environment map β 1 and the ice layer environment map β 2.

10. The method as claimed in claim 8, wherein the central processing unit is used to match the corrected spatial geographic location information with the semantic information of the ice layer obstacle obtained by the laser SLAM module and the visual SLAM module.

Background

As global climate warms, the arctic ice layer gradually melts and ships through arctic waters increase year by year. The range of the polar region channel is greatly shortened compared with the traditional Suez canal channel, and the commercial value of the polar region channel is continuously explored by the shipping industry. It is increasingly important to develop vessels suitable for polar navigation. The environment of polar regions is severe, and the ice layer floating on the sea surface poses serious threats to the safety of ships.

The electronic chart applied to the current seaplane has poor timeliness and is difficult to reflect the ice condition around the ship. With the development of scientific technology, the SLAM technology has been widely applied to navigation map drawing, including visual SLAM and laser SLAM, wherein the laser SLAM technology is always dominant. The laser SLAM has the characteristics of high precision, high speed and capability of quickly responding to environmental changes in a dynamic environment, so that the reliability and the safety of the laser SLAM technology are superior to those of a visual SLAM. The main content of laser SLAM technology includes localization and surrounding environment mapping. The visual SLAM can already perceive semantic information of an object, but the precision is poor, and the working effect of the visual SLAM is greatly influenced by ambient light. The Chinese invention patent CN110726409B discloses a map fusion method based on laser SLAM and visual SLAM, which comprises the following steps:

step one, data acquisition and reading

Acquiring the acceleration and the angular velocity of the robot through the IMU; a binocular camera of the robot acquires left and right images, a two-dimensional laser radar acquires a distance value at 360 degrees, a coder acquires the speed of the robot, and an industrial personal computer reads data of a sensor in real time;

step two, data processing

Converting the pulse value acquired by each encoder into a speed value corresponding to each wheel, calculating the overall moving speed of the robot through a motion calculation algorithm, and calculating displacement; integrating the acceleration acquired by the IMU, calculating speed and displacement, and fusing the acceleration acquired in the first step with angular speed and displacement by using extended Kalman filtering to generate a pose;

step three, establishing a graph by using laser SLAM and visual SLAM

Transmitting the two-dimensional laser radar data and the robot attitude and position information obtained in the second step into a laser SLAM, and establishing a 2D grid map; transmitting image information acquired by a binocular camera and acceleration and angular velocity acquired by an IMU (inertial measurement Unit) into a visual SLAM (simultaneous localization and mapping) to generate a three-dimensional sparse point cloud picture;

step four, visual SLAM map conversion

Converting the pose generated by the visual SLAM and the three-dimensional sparse point cloud image into a three-dimensional map represented by a skip list tree, and projecting the three-dimensional map on a plane to form a 2D grid image;

step five, map fusion

And fusing the local grid map generated by the laser SLAM and the converted local grid map of the visual SLAM in real time by using a map fusion algorithm to generate a local fusion map, storing the local fusion map in real time, and repeatedly circulating the processes all the time to form the global fusion map. The map fusion algorithm specifically comprises the following steps: expressing each grid of a local grid map generated by a laser SLAM and a local grid map generated by a visual SLAM by 0-1 probability, setting a threshold T1 of the local grid map generated by the laser SLAM and a threshold T2 of the local grid map obtained by converting the visual SLAM, then comparing the occupancy rate with the preset threshold, if the occupancy rate is greater than the threshold, the occupancy rate is empty if the occupancy rate is less than the threshold, when the occupancy rate is stored and displayed, 1 represents occupancy, 0 represents empty, and-1 represents indeterminacy; and judging the two probabilities according to a grid occupation rule, wherein the two probabilities are empty and are judged to be empty, when the two probabilities are not confirmed, the two probabilities are judged to be uncertain, and when the two probabilities are not confirmed, the other probabilities are judged to be occupied, so that a local fusion map is generated.

Although the technical scheme is adopted, the defects of single SLAM are overcome, so that the fused map is more accurate and more suitable for navigation. However, the map fusion method is not suitable for the ship polar navigation situation, and the reason is the following aspects: 1) when the ship sails in the polar region, the sea ice condition of the polar region is complex, and the situation is particularly complex when the ship sails in the polar region and the quantity of resources consumed by different types of sea ice with different sizes is different; 2) the image information acquired by the laser SLAM and the image information acquired by the visual SLAM lack the corresponding relation of geographic position coordinates, the maps acquired by the laser SLAM and the image information acquired by the visual SLAM are firstly subjected to projection conversion to form a 2D grid map, then the occupancy is compared with a set threshold value, and finally the precision of the obtained global fusion map is greatly influenced by the specific set threshold value. Therefore, it is necessary to perform further adaptive optimization based on the above-mentioned global fusion map construction method.

Disclosure of Invention

Therefore, in view of the above-mentioned problems and drawbacks, the objective group of the present invention collects relevant data, and through multi-party evaluation and consideration, the objective group personnel continuously discuss and simulate the polar sea surface ice condition through physical landscaping, which finally results in the appearance of the polar ice layer mapping system.

In order to solve the technical problem, the invention relates to a polar ice layer map construction system, which comprises at least one laser SLAM module, at least one visual SLAM module, a central processing unit and a semantic module. The laser SLAM module comprises a first computing unit, a laser radar, a first environment map information acquisition unit, a first positioning unit and a first data transmission unit. At a certain moment t, the first environment map information acquisition unit cooperates with the laser radar to acquire the ice layer environment map of the current spatial geographic position in real time, and meanwhile, the first positioning unit is used for capturing the coordinate information of the current spatial geographic position in real time. The first environment map information acquisition unit and the first positioning unit transmit data to the first calculation unit through the first data transmission unit. The vision SLAM module comprises a second computing unit, a vision sensor, a second environment map information acquisition unit, a second positioning unit and a second data transmission unit. At time t, the second environment map information acquisition unit cooperates with the vision sensor to acquire the ice layer environment map of the current spatial geographic position in real time, and meanwhile, the second positioning unit is used for capturing the coordinate information of the current spatial geographic position in real time. The second environment map information acquisition unit and the second positioning unit transmit data to the second computing unit through the second data transmission unit. The central processing unit is used for receiving and processing data and is electrically connected with the first computing unit and the second computing unit. And the central processing unit is used for classifying the ice layers around the ship according to the semantic instruction so as to generate a global ice layer cost map.

As a further improvement of the technical solution of the present invention, the first positioning unit and the second positioning unit are preferably both composed of a GNSS positioning unit and a GPS positioning unit.

As a further improvement of the technical scheme of the invention, the central processing unit comprises an information correction module.

As a further improvement of the technical scheme of the invention, the central processing unit is preferably a desktop computer or a notebook computer.

As a further improvement of the technical solution of the present invention, the first data transmission unit and the second data transmission unit are both preferably optical fiber transmission units.

As a further improvement of the technical scheme of the invention, the construction system of the polar ice layer map also comprises a back-end optimization module. The rear end optimization module is matched with the central processing unit. And performing loop data correction on the ice layer environment map of the current spatial geographic position acquired through the first environment map information and the ice layer environment map of the current spatial geographic position acquired through the second environment map information, wherein the ice layer environment map has the same spatial geographic position coordinate information.

In addition, the invention also discloses a working method of the polar region ice layer map construction system, which comprises the following steps:

a) at a certain time t, capturing coordinate information omega 1 of the current spatial geographic position and an ice layer environment map beta 1 by the laser SLAM module;

b) at the same time t, the visual SLAM module captures the coordinate information omega 2 of the current spatial geographic position and the ice layer environment map beta 2. The coordinate information omega 1 of the spatial geographical position is completely the same as the coordinate information omega 2 of the spatial geographical position, and the ice layer environment map beta 1 and the ice layer environment map beta 2 have the same timestamp;

c) uniformly transmitting the coordinate information omega 1 of the spatial geographic position, the ice layer environment map beta 1, the coordinate information omega 2 of the spatial geographic position and the ice layer environment map beta 2 to a central processing unit;

d) fusing the ice layer environment map beta 1 and the ice layer environment map beta 2 through a central processing unit to generate a real-time global environment map;

e) the semantic module is used for classifying the obstacles on the ice layer stored on the global environment map;

f) and generating a real-time global cost map with semantic information according to different ice breaking losses of different types of obstacles.

As a further improvement of the technical solution of the present invention, before the central processing unit determines the relative position relationship between the local ice layer environment maps with the same timestamp according to the respective spatial geographical position information of the laser SLAM module and the visual SLAM module, the received spatial geographical position information of the ice layer environment map β 1 and the ice layer environment map β 2 needs to be corrected.

As a further improvement of the technical scheme of the invention, a least square fitting formula is adopted to correct the space geographic position information of the ice layer environment map beta 1 and the ice layer environment map beta 2.

As a further improvement of the technical scheme of the invention, the corrected space geographic position information is matched with the semantic information of the ice layer obstacle acquired by the laser SLAM module and the visual SLAM module by the central processing unit.

By adopting the technical scheme for setting, the polar ice layer map building system at least has the following beneficial effects in practical application as follows:

1) the problem that the laser SLAM cannot sense semantic information and the problem that the visual SLAM is poor in precision and depends on the illumination environment is solved perfectly, a global environment map with semantic information, high precision and attached semantic information is generated in real time, a pilot needs to adjust a rudder angle in time according to the ice condition around a ship in the sailing process, and the control difficulty of the pilot is reduced;

2) the global, high-precision and real-time semantic map can be formed, the obstacles in the laser SLAM map can be marked according to the related information of the semantic map, and finally, the real-time global cost map with the semantic information is generated according to the different ice breaking losses of the obstacles of different types, so that a good cushion is made for keeping a relatively small ice breaking loss amount when a ship executes a navigation task in the polar region;

3) the laser SLAM technology and the visual SLAM technology are perfectly integrated, and the ice layer environment map acquired by the laser SLAM technology and the visual SLAM technology is ensured to have the corresponding relation between the geographic position coordinate and the time, so that the generated global ice layer cost map is ensured to be completely matched with the real ice condition around the ship.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic layout diagram of a laser SLAM module and a visual SLAM module on a ship in the polar ice layer map building system according to the present invention.

Fig. 2 is a working principle diagram of the polar region ice layer map construction system for generating an ice layer environment map.

FIG. 3 is a working principle diagram of the polar region ice layer map construction system generating a global ice layer cost map.

Detailed Description

In the following, the present invention will be further described in detail with reference to specific embodiments, and the polar ice layer map construction system includes a plurality of laser SLAM modules, a plurality of visual SLAM modules, a central processing unit, and a semantic module. As shown in fig. 1, the laser SLAM module is preferably arranged on the ship's building in an effort to see a better viewing angle; and the visual SLAM modules are circumferentially and uniformly distributed around the ship shell to be as close to the floating ice position as possible, so that the collected floating ice image has better definition. The laser SLAM module comprises a first computing unit, a laser radar, a first environment map information acquisition unit, a first positioning unit and a first data transmission unit. At a certain moment t, the first environment map information acquisition unit cooperates with the laser radar to acquire the ice layer environment map of the current spatial geographic position in real time, and meanwhile, the first positioning unit is used for capturing the coordinate information of the current spatial geographic position in real time. The first environment map information acquisition unit and the first positioning unit transmit data to the first calculation unit through the first data transmission unit. The vision SLAM module comprises a second computing unit, a vision sensor, a second environment map information acquisition unit, a second positioning unit and a second data transmission unit. At time t, the second environment map information collection unit cooperates with the vision sensor to collect the ice environment map of the current spatial geographical position in real time, and at the same time, the second positioning unit is used to capture the current spatial geographical position coordinate information in real time (as shown in fig. 2). The second environment map information acquisition unit and the second positioning unit transmit data to the second computing unit through the second data transmission unit. The central processor is preferably a desktop computer or a notebook computer, which is used for receiving and processing data and is electrically connected with the first computing unit and the second computing unit. And in cooperation with the semantic module, the central processing unit classifies the ice layers around the ship according to the semantic instructions to generate a global ice layer cost map (as shown in fig. 3).

In order to ensure that the captured current spatial geographical position coordinate information has higher precision and is beneficial to map fusion in the later period, the first positioning unit and the second positioning unit are both preferably composed of a GNSS positioning unit and a GPS positioning unit.

The basic component of the optical fiber is quartz, which only transmits light but is not conductive, is not influenced by an electromagnetic field and cannot be influenced by the electromagnetic field in the process of transmitting optical signals. In addition, the loss rate of optical fiber transmission is extremely low, the loss is hardly changed along with the ambient temperature, and the optical fiber transmission is suitable for polar low-temperature environment and is beneficial to avoiding the occurrence of a trunk line level phenomenon. In view of this, the first data transmission unit and the second data transmission unit are preferably optical fiber transmission units.

The central processing unit comprises an information correction module, collects sensor data from the laser SLAM module and the vision SLAM module in real time, performs loop detection by utilizing the GNSS positioning unit and the GPS positioning unit, and corrects the positions between different SLAM modules (including the positions between the laser SLAM module and the laser SLAM module, between the laser SLAM module and the vision SLAM module and between the vision SLAM module and the vision SLAM module) in a least square fitting mode.

As shown in fig. 2, it can also be known that the polar ice layer map construction system is further configured with a back-end optimization module. The rear end optimization module is matched with the central processing unit. And performing loop data correction on the ice layer environment map of the current spatial geographic position acquired through the first environment map information and the ice layer environment map of the current spatial geographic position acquired through the second environment map information, wherein the ice layer environment map has the same spatial geographic position coordinate information.

By adopting the technical scheme for setting, the polar ice layer map building system at least has the following beneficial effects in practical application as follows:

1) the problem that the laser SLAM cannot sense semantic information and the problem that the visual SLAM is poor in precision and depends on the illumination environment is solved perfectly, a global environment map with semantic information, high precision and attached semantic information is generated in real time, a pilot needs to adjust a rudder angle in time according to the ice condition around a ship in the sailing process, and the control difficulty of the pilot is reduced;

2) the global, high-precision and real-time semantic map can be formed, the obstacles in the laser SLAM map can be marked according to the related information of the semantic map, and finally, the real-time global cost map with the semantic information is generated according to the different ice breaking losses of the obstacles of different types, so that a good cushion is made for keeping a relatively small ice breaking loss amount when a ship executes a navigation task in the polar region;

3) the laser SLAM technology and the visual SLAM technology are perfectly integrated, and the ice layer environment map acquired by the laser SLAM technology and the visual SLAM technology is ensured to have the corresponding relation between the geographic position coordinate and the time, so that the generated global ice layer cost map is ensured to be completely matched with the real ice condition around the ship.

In addition, the invention also discloses a working method of the polar region ice layer map construction system, as shown in fig. 2 and 3, the working method comprises the following steps:

a) at a certain time t, capturing coordinate information omega 1 of the current spatial geographic position and an ice layer environment map beta 1 by the laser SLAM module;

b) at the same time t, the visual SLAM module captures the coordinate information omega 2 of the current spatial geographic position and the ice layer environment map beta 2. The coordinate information omega 1 of the spatial geographical position is completely the same as the coordinate information omega 2 of the spatial geographical position, and the ice layer environment map beta 1 and the ice layer environment map beta 2 have the same timestamp;

c) uniformly transmitting the coordinate information omega 1 of the spatial geographic position, the ice layer environment map beta 1, the coordinate information omega 2 of the spatial geographic position and the ice layer environment map beta 2 to a central processing unit;

d) fusing the ice layer environment map beta 1 and the ice layer environment map beta 2 through a central processing unit to generate a real-time global environment map;

e) the semantic module is used for classifying the obstacles on the ice layer stored on the global environment map;

f) and generating a real-time global cost map with semantic information according to different ice breaking losses of different types of obstacles.

Before the central processor determines the relative position relationship between the local ice layer environment maps with the same timestamp according to the respective space geographic position information of the laser SLAM module and the visual SLAM module, the received space geographic position information of the ice layer environment map beta 1 and the ice layer environment map beta 2 needs to be corrected.

And correcting the space geographic position information of the ice layer environment map beta 1 and the ice layer environment map beta 2 by adopting a least square fitting formula.

And matching the corrected spatial geographical position information with the semantic information of the ice layer obstacle acquired by the laser SLAM module and the visual SLAM module by the central processing unit.

In order to ensure the consistency of the local map, all the laser SLAM modules and the visual SLAM modules need to synchronously acquire the ice layer environment map of the corresponding space geographic position.

Let the description in the environment be si ═ (xi, yi, zi, vi), where (xi, yi, zi) represents the location coordinate of the ith point in the environment relative to the origin of the sensor, and v represents the semantic information of that point. Let the data from the j-th vision SLAM module be Mk ═ { si, s ∈ R4, i · · · · · · · · · · · · · · · ·, N } where N denotes the data volume collected by the vision sensor per cycle, the data from the laser SLAM module be LM ═ { si, s ∈ R4, i · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · N }, where N denotes the data collected by the data volume per cycle by the laser sensor per cycle, and N.

Ideally, the position of the jth visual SLAM module in the world coordinate system is measured as Lj ═ (xj, yj, zj,0), and the position matrix formed by the m visual SLAM modules is Lj ═ { li, l ∈ R4, i ═ 0 · · · m }.

Assuming that the horizontal directions of the visual SLAM modules are consistent and a very small direction error delta theta exists, the global map obtained by calculation according to the position matrix is represented as

Where R represents the extended rotation matrix corresponding to δ θ, which can be expressed as

Similarly, the global map obtained by the laser SLAM module is the LM.

After the CM and each point semantic information v are transmitted into the central processing unit, the CM1 with semantic information is formed.

And corresponding semantic information is given to each part of the LM through matching of the CM1 and the LM, so that a real-time global environment map LM1 with the semantic information is formed.

Assuming that LM1 is divided into k different parts in total, and each part in LM1 is classified according to semantics to obtain the ice breaking loss wi of the part, the global loss vector W ═ { wi, i ═ 0 · · · · k }.

And matching W with LM1 to form a real-time global cost map LM2 with semantic information.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.