阅读说明：本技术 建立室内3d地图的方法和无人机 (Method for establishing indoor 3D map and unmanned aerial vehicle ) 是由 李选富 何庭波 许占 胡慧 陈海 于 2018-06-22 设计创作，主要内容包括：本申请提供了一种建立室内3D地图的方法和无人机。该方法包括：确定目标房间的至少一个待扫描子区域；确定至少一个待扫描子区域中的每个子区域的最佳扫描点；根据当前扫描点和至少一个待扫描子区域中的每个子区域的最佳扫描点,确定目标扫描路径；沿目标扫描路径,对目标房间的至少一个待扫描子区域进行扫描；当目标房间的全部子区域完成扫描时,根据通过扫描获得的目标房间的3D点云,建立目标房间的3D地图。本申请可以在扫描时依据待扫描区域实时规划扫描路径,能够实现对室内的完整的扫描,进而构建出更准确的3D地图。(The application provides a method for establishing an indoor 3D map and an unmanned aerial vehicle. The method comprises the following steps: determining at least one sub-area to be scanned of a target room; determining an optimal scanning point of each subarea in at least one subarea to be scanned; determining a target scanning path according to the current scanning point and the optimal scanning point of each subarea in at least one subarea to be scanned; scanning at least one sub-area to be scanned of a target room along a target scanning path; and when all the sub-areas of the target room are scanned, establishing a 3D map of the target room according to the 3D point cloud of the target room obtained through scanning. According to the method and the device, the scanning path can be planned in real time according to the area to be scanned during scanning, indoor complete scanning can be achieved, and a more accurate 3D map can be constructed.)

1. A method of creating an indoor 3D map, wherein each sub-area of a target room is scanned, comprising:

determining at least one sub-area to be scanned of a target room;

determining an optimal scanning point of each sub-area in the at least one sub-area to be scanned, wherein when the optimal scanning point of each sub-area scans each sub-area, the scanning degree of each sub-area reaches a first preset scanning degree;

determining a target scanning path according to the current scanning point and the optimal scanning point of each subarea in the at least one subarea to be scanned;

starting from the current scanning point, scanning the at least one sub-area to be scanned according to the target scanning path;

and when all the sub-areas of the target room are scanned, establishing a 3D map of the target room according to the 3D point cloud of the target room obtained by scanning.

2. The method of claim 1, wherein the method further comprises:

determining a subarea with the largest area in the at least one subarea to be scanned as a target subarea;

determining a target scanning path according to the current scanning point and the optimal scanning point of each sub-area in the at least one sub-area to be scanned, wherein the determining step comprises the following steps:

and determining the target scanning path according to the current scanning point and the optimal scanning point of the target sub-area, wherein the starting point of the target scanning path is the current scanning point, and the end point of the target scanning path is the optimal scanning point of the target sub-area.

3. The method of claim 2, wherein determining the target scan path based on the current scan point and the optimal scan point for the target sub-region comprises:

and determining the scanning path with the shortest path in the plurality of scanning paths with the starting point as the current scanning point and the end point as the optimal scanning point of the target sub-area as the target scanning path.

4. The method of claim 2, wherein determining the target scan path based on the current scan point and the optimal scan point for the target sub-region comprises:

determining a path cost of each scanning path in a plurality of scanning paths with a starting point as the current scanning point and an end point as the optimal scanning point of the target sub-region according to the length of each scanning path in the plurality of scanning paths and the scanning area when the scanning path passes through the optimal scanning point of other sub-regions to be scanned during scanning, wherein the other sub-regions to be scanned are sub-regions except the target sub-region in the at least one sub-region to be scanned;

and determining the path with the minimum path cost in the multiple paths as the target scanning path.

5. The method of claim 4, wherein the scan cost for each path is obtained according to the following formula:

C＝α·d-β·S

wherein, C represents the path cost of the path, the numerical value of d is in direct proportion to the path length, the numerical value of S is in direct proportion to the area of other sub-areas to be scanned which can be scanned when the sub-areas to be scanned pass through the optimal scanning point of the other sub-areas to be scanned, and alpha and beta are weight coefficients.

6. The method of claim 1, wherein the starting point of the target scan path is the current scan point, the target scan path passes through the best scan point for each of the at least one sub-area to be scanned, and the ending point of the target scan path is the best scan point for one of the at least one sub-area to be scanned.

7. The method of claim 6, wherein determining a target scan path based on the current scan point and the optimal scan point for each of the at least one sub-area to be scanned comprises:

and determining a scanning path with the shortest path among the multiple scanning paths as the target scanning path, wherein each scanning path in the multiple paths is a path with a starting point as the current scanning point, a best scanning point passing through each sub-area in the at least one sub-area to be scanned and an end point as the best scanning point of one sub-area in the at least one sub-area to be scanned.

8. The method of any one of claims 1-7, wherein determining at least one sub-area of the target room to be scanned comprises:

acquiring boundary information of the target room;

constructing a bounding box of the target room according to the boundary information;

constructing an initial grid map of the target room according to the bounding box, wherein the initial grid map is at least one of a top view, a bottom view, a left view, a right view, a front view and a back view of the target room;

projecting the boundary information to the initial grid map to obtain a target grid map;

determining a region outside the 3D point cloud projection region in the target grid map as a region to be scanned, wherein the region to be scanned comprises the at least one sub-region to be scanned.

9. The method of any one of claims 1-8, wherein determining the optimal scan point for each of the at least one sub-region to be scanned comprises:

determining the geometric gravity center of a geometric body corresponding to the three-dimensional space to which each sub-region belongs;

determining a reference plane passing through the geometric center of gravity and an edge nadir of an upper surface edge of the geometric body and perpendicular to a bottom surface of each sub-region;

determining a target line segment on the reference plane, wherein the starting point of the target line segment is the geometric center of gravity, the length of the target line segment is a preset length, and the included angle between the target line segment and the bottom surface of each sub-region is a preset included angle;

determining the position of the end point of the target line segment as the position of the optimal scanning point of each sub-area;

and determining the direction in which the end point of the target line segment points to the starting point of the target line segment as the scanning attitude of the optimal scanning point.

10. An unmanned aerial vehicle, comprising:

the scanning device comprises a scanning-waiting subarea determining module, a scanning-waiting subarea determining module and a scanning-waiting subarea determining module, wherein the scanning-waiting subarea determining module is used for determining at least one scanning-waiting subarea of a target room;

the sub-region to be scanned determining module is further configured to determine an optimal scanning point of each sub-region in the at least one sub-region to be scanned, where when the optimal scanning point of each sub-region scans each sub-region, a scanning degree of each sub-region reaches a first preset scanning degree;

the path planning module is used for determining a target scanning path according to the current scanning point and the optimal scanning point of each subarea in the at least one subarea to be scanned;

the motion and scanning module is used for scanning the at least one sub-area to be scanned from the current scanning point according to the target scanning path;

and the mapping module is used for establishing a 3D map of the target room according to the 3D point cloud of the target room obtained through scanning when all the sub-areas of the target room are scanned.

11. The drone of claim 10, wherein the sub-region to scan determination module is further to:

determining a subarea with the largest area in the at least one subarea to be scanned as a target subarea;

the path planning module is configured to determine the target scanning path according to the current scanning point and the optimal scanning point of the target sub-region, where a starting point of the target scanning path is the current scanning point, and an end point of the target scanning path is the optimal scanning point of the target sub-region.

12. The drone of claim 11, wherein the path planning module is to:

and determining the scanning path with the shortest path in the plurality of scanning paths with the starting point as the current scanning point and the end point as the optimal scanning point of the target sub-area as the target scanning path.

13. The drone of claim 11, wherein the path planning module is to:

determining a path cost of each scanning path according to the length of each scanning path in the plurality of scanning paths, and the scanning area when the scanning is performed along each scanning path and passes through the optimal scanning point of the at least one sub-region to be scanned, in a plurality of scanning paths of which the starting point is the current scanning point and the end point is the optimal scanning point of the target sub-region;

and determining the path with the minimum path cost in the multiple paths as the target scanning path.

14. A drone as claimed in claim 13, wherein the scan cost per path is obtained according to the formula:

C＝α·d-β·S

wherein, C represents the path cost of the path, the numerical value of d is in direct proportion to the path length, the numerical value of S is in direct proportion to the area of other sub-areas to be scanned which can be scanned when the sub-areas to be scanned pass through the optimal scanning point of the other sub-areas to be scanned, and alpha and beta are weight coefficients.

15. The drone of claim 10, wherein the start point of the target scan path is the current scan point, the target scan path passes through the best scan point of each of the at least one sub-area to be scanned, and the end point of the target scan path is the best scan point of one of the at least one sub-area to be scanned.

16. The drone of claim 15, wherein the path planning module is to:

and determining a scanning path with the shortest path among the multiple scanning paths as the target scanning path, wherein each scanning path in the multiple paths is a path with a starting point as the current scanning point, a best scanning point passing through each sub-area in the at least one sub-area to be scanned and an end point as the best scanning point of one sub-area in the at least one sub-area to be scanned.

17. A drone as claimed in any one of claims 10 to 16, wherein the sub-region to scan determination module is to:

acquiring boundary information of the target room;

constructing a bounding box of the target room according to the boundary information;

constructing an initial grid map of the target room according to the bounding box, wherein the initial grid map is at least one of a top view, a bottom view, a left view, a right view, a front view and a back view of the target room;

projecting the boundary information to the initial grid map to obtain a target grid map;

determining a region outside the 3D point cloud projection region in the target grid map as a region to be scanned, wherein the region to be scanned comprises the at least one sub-region to be scanned.

18. A drone as claimed in any one of claims 10-17, wherein the sub-region to scan determination module is to:

determining the geometric gravity center of a geometric body corresponding to the three-dimensional space to which each sub-region belongs;

determining a reference plane passing through the geometric center of gravity and an edge nadir of an upper surface edge of the geometric body and perpendicular to a bottom surface of each sub-region;

determining a target line segment on the reference plane, wherein the starting point of the target line segment is the geometric center of gravity, the length of the target line segment is a preset length, and the included angle between the target line segment and the bottom surface of each sub-region is a preset included angle;

determining the position of the end point of the target line segment as the position of the optimal scanning point of each sub-area;

and determining the direction in which the end point of the target line segment points to the starting point of the target line segment as the scanning attitude of the optimal scanning point.

Technical Field

The present application relates to the field of unmanned aerial vehicle technology, and more particularly, to a method of establishing an indoor 3D map and an unmanned aerial vehicle.

Background

Along with the rapid development of the unmanned aerial vehicle technology, the indoor unmanned aerial vehicle is applied to a certain extent. Indoor unmanned aerial vehicle can freely remove in the interior space, can accomplish the various work that ground robot can't accomplish. Before the indoor unmanned aerial vehicle works indoors, the indoor unmanned aerial vehicle needs to be completely scanned indoors to generate an indoor 3D map (the 3D map can represent an indoor space structure), and then the indoor unmanned aerial vehicle works based on the 3D map.

The traditional scheme generally scans a room according to a preset rule, and establishes a 3D map according to a 3D point cloud obtained by scanning. For example, the unmanned aerial vehicle is controlled to fly along the obstacle, the indoor space is scanned during flying, and when scanning is completed, a 3D map is built according to the 3D point cloud obtained through scanning.

In the conventional scheme, incomplete scanning may occur when the indoor space is directly scanned according to a preset rule, so that the finally established 3D map is not accurate enough.

Disclosure of Invention

The application provides a method for establishing an indoor 3D map and an unmanned aerial vehicle, so that complete indoor scanning is achieved, and a more accurate 3D map is constructed.

In a first aspect, a method for creating an indoor 3D map is provided, where the method is used to scan each sub-area of a target room, and the method specifically includes: determining at least one sub-area to be scanned of a target room; determining an optimal scanning point of each subarea in at least one subarea to be scanned; determining a target scanning path according to the current scanning point and the optimal scanning point of each subarea in at least one subarea to be scanned; starting from the current scanning point, scanning at least one sub-area to be scanned of the target room according to the target scanning path; and when all the sub-areas of the target room are scanned, establishing a 3D map of the target room according to the 3D point cloud of the target room obtained by scanning.

When each sub-area is scanned at the optimal scanning point of each sub-area, the scanning degree of each sub-area reaches a first preset scanning degree.

The method can be executed by an unmanned aerial vehicle, and the target room can be a room needing 3D mapping.

Optionally, the starting point of the target scan path is a current scan point, and the end point of the target scan path is located in any one of the at least one sub-area to be scanned.

Optionally, any two sub-regions in the at least one region to be scanned are not communicated with each other.

It should be understood that the 3D point cloud of the target room obtained in the scanning process specifically comprises 3D coordinate information already scanned in the target room. The 3D map of the target room may refer to a grid map containing the occupancy of objects in the target room.

According to the method and the device, only the non-scanned area is concerned during path planning, so that the planned scanning path is more pertinent, the overall scanning efficiency during room scanning can be improved, and the effect of the 3D map can be finally constructed.

With reference to the first aspect, in certain implementations of the first aspect, the method further includes: determining a subarea with the largest area in at least one subarea to be scanned as a target subarea; determining a target scanning path according to the current scanning point and the optimal scanning point of each subarea in at least one subarea to be scanned, wherein the method comprises the following steps: and determining a target scanning path according to the current scanning point and the optimal scanning point of the target sub-area, wherein the starting point of the target scanning path is the current scanning point, and the end point of the target scanning path is the optimal scanning point of the target sub-area.

In the scanning process, the sub-region with a larger area is preferentially scanned, so that the larger area can be scanned as much as possible in the scanning process, and the scanning efficiency can be improved to a certain extent.

With reference to the first aspect, in some implementations of the first aspect, determining the target scanning path according to the current scanning point and the optimal scanning point of the target sub-region includes: determining a plurality of scanning paths of which the starting point is the current scanning point and the end point is the optimal scanning point of the target sub-area; and determining the scanning path with the shortest path in the plurality of scanning paths as a target scanning path.

With reference to the first aspect, in some implementations of the first aspect, determining the target scanning path according to the current scanning point and the optimal scanning point of the target sub-region includes: and determining the scanning path with the shortest path in the plurality of scanning paths with the starting point as the current scanning point and the end point as the optimal scanning point of the target sub-area as the target scanning path.

By taking the shortest scanning path as the target scanning path, the moving distance can be reduced in the scanning process, the scanning time can be reduced to a certain extent, and the scanning efficiency is improved.

Alternatively, the shortest scan path of the plurality of scan paths may be a straight line path from the current scan point to the optimal scan point of the target sub-area.

It should be appreciated that if an obstacle exists between the current scanning point and the optimal scanning point of the target sub-area, it may result in the optimal scanning point of the target sub-area not being reached along a straight-line path between the current scanning point and the optimal scanning point of the target sub-area. The target scanning path is then the shortest path from the current scanning point, around the obstacle and to the optimal scanning point of the target sub-area.

With reference to the first aspect, in some implementations of the first aspect, determining the target scanning path according to the current scanning point and the optimal scanning point of the target sub-region includes: determining a plurality of scanning paths of which the starting point is the current scanning point and the end point is the optimal scanning point of the target sub-area; determining the path cost of each scanning path according to the length of each scanning path in the plurality of scanning paths and the scanning area when the scanning is carried out along each scanning path and passes through the optimal scanning point of other sub-areas to be scanned; and determining the path with the minimum path cost in the plurality of paths as the target scanning path.

With reference to the first aspect, in some implementations of the first aspect, determining the target scanning path according to the current scanning point and the optimal scanning point of the target sub-region includes: determining the path cost of each scanning path in a plurality of scanning paths with the starting point as the current scanning point and the end point as the optimal scanning point of the target sub-area according to the length of each scanning path in the plurality of scanning paths and the scanning area when the scanning path passes through the optimal scanning point of other sub-areas to be scanned during scanning, wherein the other sub-areas to be scanned are sub-areas except the target sub-area in at least one sub-area to be scanned; and determining the path with the minimum path cost in the plurality of paths as the target scanning path.

Wherein, the other sub-regions to be scanned refer to sub-regions of the at least one sub-region to be scanned except for the target sub-region.

When determining the target scan path, a path with the smallest path cost may be selected from multiple paths by using a Bellman-Ford (BF) algorithm or integer programming supporting negative weight edge, and other greedy algorithms such as Genetic Algorithm (GA).

By integrating the path length of each path and the areas of other sub-areas to be scanned which can be scanned when scanning is carried out along the path, a more reasonable path can be selected from a plurality of paths to be used as a target scanning path, and the scanning efficiency is improved.

With reference to the first aspect, in some implementations of the first aspect, the scanning cost of each path is obtained according to the following formula:

C＝α·d-β·S

wherein, C represents the path cost of the path, the numerical value of d is in direct proportion to the path length, the numerical value of S is in direct proportion to the area of other sub-areas to be scanned which can be scanned when the sub-areas to be scanned pass through the optimal scanning point of the other sub-areas to be scanned, and alpha and beta are weight coefficients.

With reference to the first aspect, in certain implementations of the first aspect, the starting point of the target scan path is a current scan point, the target scan path passes through a best scan point of each of the at least one sub-area to be scanned, and the end point of the target scan path is a best scan point of one of the at least one sub-area to be scanned.

With reference to the first aspect, in certain implementations of the first aspect, determining the target scanning path according to the current scanning point and the optimal scanning point of each of at least one sub-area to be scanned includes: determining a plurality of scanning paths of which the starting point is the current scanning point, the starting point passes through the optimal scanning point of each subarea in at least one subarea to be scanned, and the end point is the optimal scanning point of one subarea in at least one subarea to be scanned; and determining the scanning path with the shortest path in the plurality of scanning paths as a target scanning path.

With reference to the first aspect, in certain implementations of the first aspect, determining the target scanning path according to the current scanning point and the optimal scanning point of each of at least one sub-area to be scanned includes: and determining the scanning path with the shortest path in the multiple scanning paths as a target scanning path, wherein each scanning path in the multiple paths is a path which has a starting point as a current scanning point, passes through the optimal scanning point of each sub-area in at least one sub-area to be scanned and has an end point as the optimal scanning point of one sub-area in at least one sub-area to be scanned.

By taking the shortest scanning path of the plurality of scanning paths as a target scanning path, scanning of at least one sub-area to be scanned can be realized from the current scanning point by moving the shortest distance as much as possible, so that the scanning time can be reduced to a certain extent, and the scanning efficiency is improved.

With reference to the first aspect, in certain implementations of the first aspect, determining at least one sub-area of the target room to be scanned includes: acquiring boundary information of a target room; constructing a bounding box of the target room according to the boundary information; constructing an initial grid map of the target room according to the bounding box, wherein the initial grid map is at least one of a top view, a bottom view, a left view, a right view, a front view and a back view of the target room; projecting the boundary information to the initial grid map to obtain a target grid map; determining a region outside the 3D point cloud projection region in the target grid map as a region to be scanned, wherein the region to be scanned comprises at least one sub-region to be scanned.

It should be understood that the initial value of each grid in the initial grid map is an invalid value, indicating that a region in the initial grid map has not been scanned.

The boundary information may be a 3D point cloud obtained by scanning.

Optionally, the bounding box is a cube corresponding to the target room, and the length, width and height of the cube respectively correspond to the length, width and height of the target room and are used for representing the size of the target space.

It should be understood that when constructing an initial grid map (alternatively referred to as a grid map) of a current room from bounding boxes, a top grid map, a bottom grid map, a left grid map, a right grid map, a front grid map, and a back grid map of the room may be constructed from the bounding boxes and one of the grid maps selected as the initial grid map. Or it is also possible to construct only a mesh map of one view and determine the mesh map of the view as the initial mesh map.

With reference to the first aspect, in certain implementations of the first aspect, determining an optimal scanning point for each of at least one sub-area to be scanned includes: determining the geometric gravity center of a geometric body corresponding to the three-dimensional space to which each sub-region belongs; determining a reference plane which passes through the geometric center of gravity and the edge lowest point of the edge of the upper surface of the geometric body and is vertical to the bottom surface of each sub-area; determining a target line segment on a reference plane, wherein the starting point of the target line segment is the geometric gravity center, the length of the target line segment is a preset length, and the included angle between the target line segment and the bottom surface of each subarea is a preset included angle; determining the position of the end point of the target line segment as the position of the optimal scanning point of each sub-area; and determining the direction in which the end point of the target line segment points to the starting point of the target line segment as the scanning attitude of the optimal scanning point.

Optionally, the method further includes: determining an outlet of the current room according to the real-time 3D point cloud information of the current room; and when the scanning degree of the current room meets a second preset scanning degree, moving to the exit of the current room, and scanning other rooms except the exit of the current room.

In the scanning process, the exit of the current room is identified, so that continuous scanning of a plurality of rooms can be realized, and the scanning effect can be improved.

In a second aspect, there is provided a drone comprising respective modules for performing the method of the first aspect above.

In a third aspect, a computer-readable storage medium is provided, which stores program code, wherein the program code comprises instructions for performing the method of any one of the implementations in the first aspect.

When the above-mentioned computer-readable storage medium is provided inside a drone, the drone may execute the program code in the computer-readable storage medium, and when the drone executes the program code in the computer-readable storage medium, the drone may execute the method in any one of the implementations of the first aspect.

Drawings

Fig. 1 is a schematic flow chart of a method of building an indoor 3D map according to an embodiment of the present application;

FIG. 2 is a schematic illustration of determining an optimal scanning point for a sub-area to be scanned;

FIG. 3 is a schematic diagram of a bounding box;

FIG. 4 is a schematic diagram of a top view grid diagram;

FIG. 5 is a schematic diagram of a top view grid map after expanding a boundary;

FIG. 6 is a grid map containing 3D point cloud information;

FIG. 7 is a schematic diagram of a grid map;

FIG. 8 is a schematic illustration of forming an occlusion region;

FIG. 9 is a schematic flow chart of scanning a plurality of rooms;

fig. 10 is a schematic block diagram of a drone of an embodiment of the present application;

fig. 11 is a schematic block diagram of a drone of an embodiment of the present application.

Detailed Description

The technical solution in the present application will be described below with reference to the accompanying drawings.

The scanning point in the present application may also be referred to as an observation point or a detection point, and the scanning point may be some key points that are passed when scanning indoors. In the application, the scanning path is planned according to the scanning points by acquiring the scanning points, so that complete indoor scanning is realized.

Fig. 1 shows a schematic flow chart of a method for building an indoor 3D map according to an embodiment of the present application. The method shown in fig. 1 may be performed by a drone or other device having automatic flight and scanning capabilities. The method shown in fig. 1 specifically includes steps 110 to 160, and the steps 110 to 160 are described in detail below.

110. At least one sub-area to be scanned of the target room is determined.

The target room may be a room in which 3D mapping is required.

120. An optimal scanning point for each of the at least one sub-area to be scanned is determined.

When the optimal scanning point of each sub-area is to scan each sub-area, the scanning degree of each sub-area reaches a first preset scanning degree.

The scanning degree of a certain sub-region can be expressed by the ratio of the scanning area of the sub-region to the whole area of the sub-region. For example, the first preset scanning degree may be 60%, and in this case, taking the first sub-region as an example, the optimal scanning point of the first sub-region is that when the first sub-region is scanned, a ratio of a scanning area of the first sub-region to an entire area of the first sub-region is greater than or equal to 60%.

130. And determining a target scanning path according to the current scanning point and the optimal scanning point of each subarea in at least one subarea to be scanned.

The at least one sub-area to be scanned may be all sub-areas to be scanned (or referred to as non-scanned sub-areas) in the target room.

Optionally, determining a target scanning path according to the current scanning point and the optimal scanning point of each sub-area in the at least one sub-area to be scanned, including: and determining the starting point as the current scanning point, and determining the path passing through at least one sub-area to be scanned of the target room as a target scanning path.

The scanning of the region to be scanned can be realized by planning a path which starts from the current scanning point and passes through at least one sub-region to be scanned of the target room.

Alternatively, the target path may be a scan path from the current scan point to a sub-area in the at least one sub-area to be scanned, or may be a scan path from the current scan point and passing through the best scan point of all sub-areas in the at least one sub-area to be scanned.

The specific process of determining the target scan path in each of these two cases will be described in detail below.

In the first case: the starting point of the target scanning path is a current scanning point, and the end point is an optimal scanning point of a certain subarea in at least one subarea to be scanned.

In the first case, a path from the current scanning point to the optimal scanning point of the sub-region with the largest area in the at least one region to be scanned may be planned as the target scanning path, and the specific process is as follows:

(1) determining a subarea with the largest area in at least one subarea to be scanned as a target subarea;

(2) and determining a target scanning path according to the current scanning point and the optimal scanning point of the target sub-area, wherein the starting point of the target scanning path is the current scanning point, and the end point of the target scanning path is the path of the optimal scanning point of the target sub-area.

In the application, the scanning process can scan a larger area as much as possible by preferentially scanning the sub-area with the larger area, and the scanning efficiency can be improved to a certain extent.

Further, in order to reduce the length of the path as much as possible during the scanning process, the shortest available path may be selected as the target scanning path, and the specific process is as follows:

(3) and determining the scanning path with the shortest path in the plurality of scanning paths with the starting point as the current scanning point and the end point as the optimal scanning point of the target sub-area as the target scanning path.

In the application, the shortest scanning path is taken as the target scanning path, so that the moving distance can be reduced in the scanning process, the scanning time can be reduced to a certain extent, and the scanning efficiency is improved.

Alternatively, the length of the shortest scanning path of the plurality of scanning paths may be a straight-line distance from the current scanning point to the optimal scanning point of the target sub-area.

Alternatively, an obstacle existing between the current scanning point and the optimal scanning point of the target sub-area causes no straight-line path to exist along the current scanning point to the optimal scanning point of the target sub-area. The target scanning path is then the shortest path from the current scanning point, around the obstacle and to the optimal scanning point of the target sub-area.

In the first case, in addition to determining the target scanning path by considering the distance of the path, the target scanning path may be determined by comprehensively considering the scanning area when the path having the starting point as the current scanning point and the end point as the optimal scanning point of the target sub-region passes through the optimal scanning point of at least one sub-region to be scanned, and the specific process is as follows:

(4) determining the path cost of each scanning path in a plurality of scanning paths with the starting point as the current scanning point and the end point as the optimal scanning point of the target sub-area according to the length of each scanning path in the plurality of scanning paths and the scanning area when the scanning is carried out along each scanning path and the optimal scanning point of other sub-areas to be scanned passes through;

(5) and determining the path with the minimum path cost in the plurality of paths as the target scanning path.

Wherein, the other sub-regions to be scanned refer to sub-regions of the at least one sub-region to be scanned except for the target sub-region.

When determining the target scan path, a path with the smallest path cost may be selected from multiple paths by using a Bellman-Ford (BF) algorithm supporting negative weight edge or integer programming, or other greedy algorithm such as a Genetic Algorithm (GA).

In the application, by integrating the path length of each path and the areas of other sub-regions to be scanned which can be scanned when scanning along the path, a more reasonable path can be selected from a plurality of paths as a target scanning path, and the scanning efficiency is improved.

The path cost of each path may be determined according to equation (1).

C＝α·d-β·S (1)

Wherein, C represents the path cost of the path, the value of d is proportional to the path length, the value of S is proportional to the area of other sub-regions to be scanned which can be scanned when passing through the optimal scanning point of other sub-regions to be scanned, α and β are weighting coefficients (α is the weighting coefficient of d, β is the weighting coefficient of S).

In the second case: the starting point of the target scanning path is the current scanning point, and the target scanning path passes through all the subareas in at least one subarea to be scanned,

in the second case, the starting point is the current scanning point, and there are multiple paths passing through all sub-areas of at least one sub-area to be scanned, and one path may be directly selected from the multiple paths as the target scanning path, and the specific process is as follows:

(6) and determining a target scanning path according to the current scanning point and the optimal scanning point of each subarea in at least one subarea to be scanned.

Specifically, the starting point of the target scanning path determined in the above process (6) is the current scanning point, the end point is the path of the optimal scanning point of one sub-area of the at least one sub-area to be scanned, and the target scanning path passes through the optimal scanning point of each sub-area of the at least one sub-area to be scanned.

Further, in the second case, a path with the shortest path may also be selected from the multiple selectable paths as the target scanning path, and the specific process is as follows:

(7) and determining the scanning path with the shortest path in the multiple scanning paths as a target scanning path, wherein each scanning path in the multiple paths is a path which has a starting point as a current scanning point, passes through the optimal scanning point of each sub-area in at least one sub-area to be scanned and has an end point as the optimal scanning point of one sub-area in at least one sub-area to be scanned.

In the application, the shortest scanning path is taken as the target scanning path, scanning of at least one to-be-scanned subarea can be realized from the current scanning point by moving the distance as less as possible, the scanning time can be reduced to a certain extent, and the scanning efficiency is improved.

140. Starting from the current scanning point, scanning at least one sub-area to be scanned of the target room according to the target scanning path;

it should be understood that step 140 is to scan at least one sub-area to be scanned of the target room by moving along the target scanning path from the current scanning point.

150. It is determined whether all sub-areas of the target room have completed scanning.

The fact that all the sub-areas (all the sub-areas) of the target room are scanned is equivalent to that the target room is scanned, and whether the target room is scanned or not can be judged according to the scanning degree of the target room.

In particular, when the target room is scanned to a second preset scanning degree, the sub-area of the target room may be considered to be completely scanned. For example, when the proportion of the scanned area of the target room to the total area of the target room reaches 95% or more, it can be considered that the scanning of the target room is completed.

160. And establishing a 3D map of the target room according to the 3D point cloud of the target room obtained by scanning.

It should be understood that the 3D point cloud of the target room obtained in the scanning process specifically includes the scanned points already scanned in the target room, and 3D coordinate information of these scanned points, category attribute information (semantic information, color texture information, etc.) of the points. The 3D map of the target room may be an occupancy grid map, a 3D mesh map expressed by an octree (octree), a semantic map having semantic information, or the like.

According to the method and the device, only the non-scanned area is concerned during path planning, so that the planned scanning path is more pertinent, the overall scanning efficiency during room scanning can be improved, and the effect of the 3D map can be finally constructed.

Optionally, as an embodiment, determining an optimal scanning point of each sub-area of the at least one sub-area to be scanned includes: determining the geometric gravity center of a geometric body corresponding to the three-dimensional space to which each sub-region belongs; determining a reference plane which passes through the geometric center of gravity and the edge lowest point of the edge of the upper surface of the geometric body and is vertical to the bottom surface of each sub-area; determining a target line segment on a reference plane, wherein the starting point of the target line segment is the geometric gravity center, the length of the target line segment is a preset length, and the included angle between the target line segment and the bottom surface of each subarea is a preset included angle; determining the position of the end point of the target line segment as the position of the optimal scanning point of each sub-area; and determining the direction in which the end point of the target line segment points to the starting point of the target line segment as the scanning attitude of the optimal scanning point.

It should be understood that the geometric body corresponding to the three-dimensional space to which each sub-region belongs may be a geometric body formed by the three-dimensional space to which each sub-region belongs, or the geometric body is formed by the three-dimensional space occupied by the object in each sub-region, and the geometric body may be regular or irregular.

As shown in fig. 2, the geometric body corresponding to the sub-area to be scanned is an irregular cylinder, the center of gravity a of the cylinder and the lowest point B of the edge of the upper surface of the cylinder form a reference plane along the height direction of the cylinder, and the reference plane is perpendicular to the bottom surface of the cylinder, so that a line segment with a preset length (which may be set to 2 meters or other values according to the optimal observation distance of the sensor in the drone) and a certain included angle (for example, 30 degrees) with the height direction of the cylinder can be drawn from the point a on the reference plane, the position of the end point of the line segment is the position of the optimal scanning point of the sub-area to be scanned, and the included angle formed by the opposite direction of the line segment and the height direction of the cylinder is the scanning attitude (scanning angle) of the optimal scanning point of the sub-area to be scanned.

It should be understood that before the scanning of the at least one sub-region to be scanned, the region to be scanned in the target image may also be determined, and the specific process of determining the region to be scanned in the target image is as follows:

(8) acquiring boundary information of a target room;

(9) constructing a bounding box of the target room according to the boundary information;

(10) constructing an initial grid map of the target room according to the bounding box, wherein the initial grid map is at least one of a top view, a bottom view, a left view, a right view, a front view and a back view of the target room;

(11) projecting the boundary information to the initial grid map to obtain a target grid map;

(12) determining a region outside the 3D point cloud projection region in the target grid map as a region to be scanned, wherein the region to be scanned comprises at least one sub-region to be scanned.

The boundary information may be information such as the approximate length and width of the target room obtained at the time of the initial scan. For example, when the method of the present application is performed by a drone, the boundary information may be obtained by roughly scanning around the target room after the drone is initially lifted.

In addition, the initial value of each mesh in the initial mesh map is an invalid value, indicating that the region in the initial mesh map is not scanned.

Specifically, after the unmanned aerial vehicle takes off, a camera or other detectors can be adopted to roughly detect the indoor space so as to obtain the indoor space, and then the indoor bounding box is constructed according to the indoor space.

In addition, the flying height of the unmanned aerial vehicle can be set according to the room height in a certain proportion mode, and can also be set according to a mode of keeping a certain safe distance from the top of the room.

As shown in fig. 3, the bounding box may be a cube, and the length, width and height of the cube may respectively correspond to the length, width and height of the room, and are used to indicate the size of the room space.

In constructing an initial grid map (alternatively referred to as a grid map) of a current room from bounding boxes, a top grid map, a bottom grid map, a left grid map, a right grid map, a front grid map, and a back grid map of the room may be constructed from the bounding boxes, and one of the grid maps may be selected as the initial grid map. Or it is also possible to construct only a mesh map of one view and determine the mesh map of the view as the initial mesh map.

It should be understood that the length and width of the top and bottom grid views constructed from the indoor bounding box are the same as the length and width, respectively, of the room bounding box, while the length and width of the left, right, front and back grid views correspond to the length and height, or width and height, of the indoor bounding box.

For example, a top-down grid view based on a bounding box configuration is shown in FIG. 4, and the top-down grid view shown in FIG. 4 may be used directly as the initial grid view. It will be appreciated that the length and width of the top view grid diagram shown in FIG. 4 is the same as the length and width, respectively, of the bounding box shown in FIG. 3. In addition, the top grid diagram shown in fig. 4 also includes a plurality of grids, which can conveniently determine the scanning area and the area to be scanned.

It should be understood that the top view may be divided into grids according to the size of 0.1 × 0.1m (or divided according to other sizes), so as to obtain the top view grid shown in fig. 4, and the grids are used to record whether the grid area is scanned, so as to determine the scanning area and the area to be scanned.

In addition, when the initial grid map is determined, the boundary of the obtained grid map at a certain angle of view may be expanded, and the expanded grid map may be determined as the initial grid map. For example, the overhead grid diagram shown in fig. 4 is expanded to obtain the overhead grid diagram after the boundary expansion shown in fig. 5, and the grid on the boundary in the overhead grid diagram shown in fig. 5 is used to represent the wall around the room, wherein the value of the partial grid on the boundary is the height h of the wall₀And other grids on the boundary do not have values due to shielding or exceeding the detection distance of the sensor of the unmanned aerial vehicle.

When the initial mesh map is the top-view mesh map shown in fig. 5, the currently acquired 3D point cloud information may be projected (projected) into the top-view mesh map shown in fig. 5, resulting in the mesh map shown in fig. 6. In fig. 6, the area where the symbol "+" is located represents the area where the 3D point cloud is projected, and the unit area where the symbol "-" is located represents the area where the 3D point cloud is not projected, so the area on the right side of fig. 6 is the area to be scanned, where the grid where h is located represents the boundary between the area to be scanned and the scanning area, and the value of h represents the height value of the boundary. It should be understood that the height value at the intersection between the region to be scanned and the scanned region shown in fig. 6 may have a certain difference, and h in the figure is only a symbol indicating the height of the intersection, and does not indicate the actual height. The region to be scanned shown in fig. 6 is a complete region, and in this case, the region to be scanned is substantially composed of a sub-region.

As shown in fig. 7, the region to be scanned includes two independent regions, and in this case, the region to be scanned is substantially composed of two sub-regions (a sub-region 1 to be scanned and a sub-region 2 to be scanned).

In addition, during the process of scanning indoors, due to the shielding of the object, a shielding region may be generated during the scanning process, that is, there may be a shielding region in the region to be scanned, and the related content of the shielding region is described below.

For example, as shown in fig. 8, due to occlusion of the piano in the current room, two boundary lines, a boundary line 1 and a boundary line 2, which are continuous inside the image but discontinuous in the 3D coordinate system, may be acquired when scanning the piano, and the positions of the boundary line 1 and the boundary line 2 in the 3D coordinate system constitute a visual occlusion region. It should be understood that since the occlusion region formed by occlusion is also included in the range of the sub-region to be scanned, the present application actually takes the occlusion region into account when planning the path according to the region to be scanned.

It should be understood that after the target room is scanned, other unscanned rooms may also be scanned by entering other rooms through the exit of the target room, thereby achieving continuous scanning in multiple rooms, as described in detail below in conjunction with fig. 9.

Fig. 9 is a schematic flow chart of scanning a plurality of rooms. The process shown in fig. 9 may be performed by a drone or other device having automatic flight and scanning capabilities. The process shown in fig. 9 specifically includes steps 2001 to 2006, and the steps 2001 to 2006 are described in detail below.

2001. Starting;

when there are multiple rooms to scan, the scan can be started from any one room.

2002. And scanning the current room, and determining the exit information of the current room according to the scanning information of the current room.

It should be understood that the current room is equivalent to the target room in the method shown in fig. 1, and the detailed process of scanning the current room can be referred to the method shown in fig. 1 above, and the detailed process is not repeated.

Specifically, the exit information of the current room may be determined by the scanning information obtained during the scanning of the current room (the exit information of the current room is determined while scanning), or may be determined after the scanning of the current room is completed.

The exit information may include the size of an exit (door, landing, etc.) of the current room, the location of the exit in the current room, and the like.

When determining the exit information of the current room, the image of the current room and the 3D point cloud obtained by scanning may be obtained first, and then the exit information of the current room is determined according to the image of the current room and the 3D point cloud.

After the image of the current room is obtained, target detection can be performed on the image of the current room (the obtained image can be subjected to target detection through a neural network or other target detection algorithms), the exit of the current room is obtained, and after the exit of the current room is identified, 3D position information of the exit can be obtained by combining with 3D point cloud obtained through scanning.

Specifically, the indoor exit generally includes a door, a window, a staircase, and the like, and when the exit is identified, some pictures with labeled positions of the door and the staircase can be collected, the neural network is trained, and after the neural network is trained, the network model is used to input the picture with the door or the staircase, and the neural network can output the position of the door or the staircase in the image. It should be understood that the exit position identified by using the neural network is a 2D position, and in order to obtain the true position of the exit of the current room, the 3D point cloud information of the current room needs to be combined again, the point cloud is segmented based on the 2D position, and then the segmented point cloud data belonging to the door estimates the 3D information of the door through a neural network (e.g., a frutum-pointe network) algorithm.

2003. Determining whether the current room is scanned completely;

if the current room is not scanned, returning to the previous step, and continuing to execute step 2002; if the current room has been scanned, then step 2004 is performed.

If the scanning degree of the current room reaches a second preset scanning degree (for example, when the scanning degree reaches 90% or more), the scanning of the current room can be considered to be finished; if the scanning degree of the current room does not reach the second preset scanning degree (less than 90%, that is, more than 10% of the area is not scanned yet), it can be considered that the scanning of the current room is not finished.

The second preset scanning degree may be set according to actual conditions, and may be set to 90%, 95%, or the like, for example.

Specifically, the second preset scanning degree may be a scanning completion degree, for example, a proportion of a space where scanning is completed to a whole space, or may be a scanning fineness, that is, a rich degree of details of a scanned space included in scanning, and the like, which is not limited.

2004. Determining whether an unexploited exit exists in the room that has been scanned;

if there is an exit that has not passed in the room that has been scanned, then step 2005 is performed; if there is no room currently passing through, then step 2006 is performed.

It will be appreciated that during the scanning process, the status of the exit from each room may be recorded, the initial status of each exit being non-passed, and if a certain exit is passed, the status of that exit being passed may be recorded.

In step 2004, it may also be determined whether there is an unexplored exit in the current room. If the current room has an outlet which is not passed through, entering other unscanned rooms through the outlet which is not passed through of the current room for scanning; and if the current room has no outlet which is not passed through, returning to the previous room of the current room. Judging whether an outlet which is not passed exists in the previous room after the previous room is returned, if so, entering other rooms which are not scanned from the outlet which is not passed of the previous room for scanning, otherwise, continuing to return the previous room until the room which is initially scanned is returned, and if so, continuing to scan other rooms which are scanned through the outlet which is not passed; if the room initially scanned does not have an exit that has not passed, the scan is ended.

2005. Other unscanned rooms are scanned by passing through an unpassed exit into the other unscanned rooms.

After step 2005 is executed, step 2002 is executed again until all the rooms not scanned are scanned.

The scanning process when scanning other unscanned rooms is the same as the scanning process of the target room in the method shown in fig. 1, and the description thereof is not repeated.

2006. And (6) ending.

After the scanning is finished, a 3D map of each room can be constructed according to the acquired 3D point cloud of each room.

The method for building an indoor 3D map according to the embodiment of the present application is described in detail above with reference to fig. 1 to 9, and the drone according to the embodiment of the present application is described below with reference to fig. 10 and 11, it should be understood that the drone shown in fig. 10 and 11 is capable of performing the steps of the method for building an indoor 3D map according to the embodiment of the present application, specifically, the drone shown in fig. 10 and 11 is capable of performing the steps in fig. 1 and 9, and repeated descriptions are appropriately omitted in describing the drone according to the embodiment of the present application in order to avoid unnecessary repetition.

Fig. 10 is a schematic block diagram of a drone of an embodiment of the present application. The drone 3000 shown in fig. 10 includes:

a to-be-scanned sub-region determining module 3001, configured to determine at least one to-be-scanned sub-region of the target room;

the to-be-scanned sub-region determining module 3001 is further configured to determine an optimal scanning point of each sub-region of the at least one to-be-scanned sub-region, where a scanning degree of each sub-region reaches a first preset scanning degree when the optimal scanning point of each sub-region scans each sub-region;

a path planning module 3002, configured to determine a target scanning path according to the current scanning point and the optimal scanning point of each sub-area in the at least one sub-area to be scanned;

a moving and scanning module 3003, configured to scan, starting from the current scanning point, at least one sub-area to be scanned of the target room according to the target scanning path;

a mapping module 3004, configured to build a 3D map of the target room according to the 3D point cloud of the target room obtained through the scanning when all the sub-areas of the target room complete the scanning.

According to the method and the device, only the non-scanned area is concerned during path planning, so that the planned scanning path is more pertinent, the overall scanning efficiency during room scanning can be improved, and the effect of the 3D map can be finally constructed.

Fig. 11 is a schematic block diagram of a drone of an embodiment of the present application.

The drone 4000 shown in fig. 11 includes: a detection unit 4001, a flight unit 4002 and a control and processing unit 4003.

The control and processing unit 4003 is equivalent to the sub-area to be scanned determining module 3001, the path planning module 3002 and the mapping module 3004 described above; the detection unit 4001 corresponds to a scanning module in the motion and scanning module 3003, and the flying unit 4002 corresponds to a motion module in the motion and scanning module 3003; the various modules in drone 4000 may also implement the functionality of the various modules in drone 3000.

It should be understood that the control and processing unit 4003 may correspond to a processor inside the drone, which may be specifically a Central Processing Unit (CPU) or an artificial intelligence chip.

The detection unit 4001 may correspond to a camera of the drone and other detectors, and the like. The flying unit 4002 may correspond to a motor, a propeller, etc. of the unmanned aerial vehicle for driving the unmanned aerial vehicle to fly.

In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of a unit or a module in the apparatus is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted or not executed.

In addition, each unit or module in the apparatus of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The methods of the present application, if implemented in software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.