阅读说明：本技术 一种车载点云着色中最优关联影像选择方法及系统 (Optimal associated image selection method and system in vehicle-mounted point cloud coloring ) 是由 颜源 杨晶 汪开理 陈海佳 翁国康 于 2021-05-25 设计创作，主要内容包括：本发明涉及一种车载点云着色中最优关联影像选择方法及系统,该方法包括：对各个影像依次进行判断,确定点云中任一待着色物方点的最优关联影像为满足避开盲区和不存在遮挡的条件的影像；确定影像的有效区域,将物方点投影到影像上得到投影点,投影点在有效区域内时确定该影像满足避开盲区的条件；物方点与某影像对应的摄影中心的连接线段的附近设定距离范围内不存在其它物方点时确定该影像满足不存在遮挡的条件。选择未被遮挡的影像作为最优关联影像,可解决由于遮挡导致的着色错误问题。(The invention relates to a method and a system for selecting an optimal associated image in vehicle-mounted point cloud coloring, wherein the method comprises the following steps: sequentially judging each image, and determining the optimal associated image of any object space point to be colored in the point cloud as an image meeting the conditions of avoiding a blind area and having no shielding; determining an effective area of an image, projecting object space points on the image to obtain projection points, and determining that the image meets the condition of avoiding a blind area when the projection points are in the effective area; and when no other object point exists in the set distance range near the connecting line segment of the object point and the shooting center corresponding to a certain image, determining that the image meets the condition of no occlusion. The image which is not blocked is selected as the optimal related image, so that the problem of shading errors caused by blocking can be solved.)

1. An optimal associated image selection method in vehicle-mounted point cloud coloring is characterized by comprising the following steps:

sequentially judging each image, and determining the optimal associated image of any object space point to be colored in the point cloud as an image meeting the conditions of avoiding a blind area and having no shielding;

determining an effective area of the image, projecting the object point onto the image to obtain a projection point, and determining that the image meets the condition of avoiding a blind area when the projection point is in the effective area;

and determining a photographing center corresponding to the image, and determining that the image meets the condition of no occlusion when other object space points do not exist in a set distance range near a connecting line segment between the object space point and the photographing center.

2. The method of claim 1, wherein before sequentially determining each image, the method further comprises: and dividing the point cloud into a plurality of blocks according to the coordinate range of the point cloud, and sequentially determining the optimal associated image of each object space point in each block of point cloud.

3. The method of claim 2, wherein the step of dividing the point cloud into blocks according to the coordinate range of the point cloud comprises:

dividing the point cloud into a plurality of blocks with side length d according to X, Y coordinates; when the coordinate range of the point cloud is (xmin, ymin, zmin) to (xmax, ymax, zmax), the coordinate range of the i-th block is (xmin + (i-1) d, ymin + (i-1) d) to (xmin + i d, ymin + i d).

4. The method as claimed in claim 1, wherein the active area is selected by a user using a polygon frame.

5. The method of claim 3, wherein determining that the image satisfies the condition of absence of occlusion comprises: dividing a block where a point to be colored is located into three-dimensional grids, determining each grid through which the connecting line segment passes, calculating the distance from each object point in each grid to the connecting line segment, and determining that the object point is not shielded under the shooting visual angle of the shooting center when the distances are all larger than a set distance threshold.

6. The method of claim 1, wherein the block where the point to be rendered is located is divided into three-dimensional grids, each grid through which the connecting line segment passes is determined, the object point and the object point in each grid are projected onto the image, and if the pixel distance between the object point to be rendered and an object point in a grid is smaller than a set threshold, the object point is considered to be occluded under the view angle of the image.

7. The method of claim 1, wherein after the selection of the optimal related image is completed, the color attribute of the projection point of the object point on the optimal related image is assigned to the object point.

8. An optimal associated image selection system in vehicle-mounted point cloud coloring is characterized by comprising the following components: the device comprises a blind area avoiding judgment module, a shielding judgment module and an optimal associated image determination module;

the blind area avoiding judgment module is used for determining an effective area of the image, projecting the object point onto the image to obtain a projection point, and determining that the image meets the condition of avoiding the blind area when the projection point is in the effective area;

the shielding judgment module is used for determining a photographing center corresponding to the image, and determining that the image meets the condition of no shielding when other object space points do not exist in the set distance range near the connecting line segment of the object space point and the photographing center;

and the optimal associated image determining module is used for sequentially judging the images and determining the optimal associated image of any object space point to be colored in the point cloud as the image meeting the conditions of avoiding the blind area and having no shielding.

9. An electronic device, comprising a memory and a processor, wherein the processor is configured to implement the steps of the method for selecting the optimal correlation image in the vehicle-mounted point cloud coloring according to any one of claims 1 to 7 when executing a computer management program stored in the memory.

10. A computer-readable storage medium, on which a computer management program is stored, wherein the computer management program, when executed by a processor, implements the steps of the method for selecting an optimal correlation image in vehicle-mounted point cloud coloring according to any one of claims 1 to 7.

Technical Field

The invention relates to the field of laser point cloud data processing, in particular to a method and a system for selecting an optimal associated image in vehicle-mounted point cloud coloring.

Background

The information of the laser point cloud data and the optical image data has a great deal of complementarity, and after the laser point cloud data and the optical image data are fused, the point cloud data with both spatial information and semantic information can be obtained, so that the advantages are exerted. In the vehicle-mounted mobile measurement system, through the registration of the point cloud and the image, an image point corresponding to each point P in the laser point cloud can be found in the panoramic image, the color attribute of the image point is given to the point P, and vehicle-mounted color point cloud data is obtained. Fig. 1 is a flow chart of a method for generating a vehicle-mounted true-color point cloud in the prior art. The vehicle-mounted panoramic image has a high overlapping rate. One laser point can find corresponding image points in a plurality of panoramic images, and the optimal associated image is determined to directly influence the quality of the color point cloud, which is an essential link for obtaining the vehicle-mounted color point cloud.

The prior art has a method for determining an optimal associated image, which comprises the following steps: 1. the optimal associated image is determined according to the included angle between the vertical line from the laser point to the road side line and the photographic beam, the smaller the included angle is, the closer the collection visual angles of the object point in the laser scanner and the panoramic camera are proved, at the moment, the closer the laser point is to the image, the higher the imaging precision is, and the truer the texture is. 2. And searching the panoramic image with the exposure time closest to the point cloud acquisition time as an optimal associated image.

However, since the collection view angles of the laser scanner and the panoramic camera are different, a certain point collected by the laser scanner may be blocked by other ground objects under the collection view angle of the panoramic camera, thereby causing a coloring error.

Disclosure of Invention

The invention provides a method and a system for selecting an optimal associated image in vehicle-mounted point cloud coloring, aiming at the technical problems in the prior art, wherein an object point is projected in an effective area on an image, and the image which is not shielded in the process of shooting the object point by the image is used as the optimal associated image; when coloring, the selected image is shielded and judged under the acquisition visual angle of the image, the object space point and the panoramic photography center form a line segment, if other object space points exist near the line segment, the selected image is considered to be shielded by other ground objects when the object space point is shot, and the image which is not shielded is selected as the optimal associated image, so that the problem of coloring errors caused by shielding can be solved.

According to a first aspect of the invention, a method for selecting an optimal associated image in vehicle-mounted point cloud coloring is provided, which comprises the following steps:

sequentially judging each image, and determining the optimal associated image of any object space point to be colored in the point cloud as an image meeting the conditions of avoiding a blind area and having no shielding;

determining an effective area of the image, projecting the object point onto the image to obtain a projection point, and determining that the image meets the condition of avoiding a blind area when the projection point is in the effective area;

and determining a photographing center corresponding to the image, and determining that the image meets the condition of no occlusion when other object space points do not exist in a set distance range near a connecting line segment between the object space point and the photographing center.

On the basis of the technical scheme, the invention can be improved as follows.

Optionally, before sequentially determining each image, the method further includes: and dividing the point cloud into blocks according to the coordinate range of the point cloud, and sequentially determining the optimal associated image of each object space point in each block of point cloud.

Optionally, the process of dividing the point cloud into blocks according to the coordinate range of the point cloud includes:

dividing the point cloud into a plurality of blocks with side length d according to X, Y coordinates; when the coordinate range of the point cloud is (xmin, ymin, zmin) to (xmax, ymax, zmax), the coordinate range of the i-th block is (xmin + (i-1) d, ymin + (i-1) d) to (xmin + i d, ymin + i d).

Optionally, the active area is selected by a user with a polygon box.

Optionally, the process of determining that the image satisfies the condition of no occlusion includes: dividing the point cloud block where the point to be colored is located into three-dimensional grids, determining each grid through which the connecting line segment passes, calculating the distance from each object space point in each grid to the connecting line segment, and determining that the object space point is not shielded under the shooting visual angle of the shooting center when the distances are all larger than a set distance threshold value.

Optionally, the point cloud block where the point to be colored is located is divided into three-dimensional grids, each grid through which the connecting line segment passes is determined, the point to be colored and the object space point in each grid are projected onto the image, and if the pixel distance between the point to be colored and an object space point after projection is smaller than or equal to a set threshold, the object space point is considered to be shielded by the object space point in the grid.

Optionally, after the optimal associated image is selected, the color attribute of the projection point of the object point on the optimal associated image is given to the object point.

According to a second aspect of the present invention, there is provided a system for selecting an optimal associated image in vehicle-mounted point cloud coloring, including: the device comprises a blind area avoiding judgment module, a shielding judgment module and an optimal associated image determination module;

the blind area avoiding judgment module is used for determining an effective area of the image, projecting the object point onto the image to obtain a projection point, and determining that the image meets the condition of avoiding the blind area when the projection point is in the effective area;

the shielding judgment module is used for determining a photographing center corresponding to the image, and determining that the image meets the condition of no shielding when other object space points do not exist in the set distance range near the connecting line segment of the object space point and the photographing center;

and the optimal associated image determining module is used for sequentially judging the images and determining the optimal associated image of any object space point to be colored in the point cloud as the image meeting the conditions of avoiding the blind area and having no shielding.

According to a third aspect of the present invention, an electronic device is provided, which includes a memory and a processor, wherein the processor is configured to implement the steps of the method for selecting the optimal associated image in the vehicle cloud rendering when executing a computer management class program stored in the memory.

According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having stored thereon a computer management-like program, which when executed by a processor, performs the steps of a method for selecting an optimal associated image in vehicle-mounted point cloud coloring.

According to the optimal associated image selection method, the system, the electronic equipment and the storage medium in vehicle-mounted point cloud coloring, the point cloud is divided into a plurality of blocks according to the coordinate range of the point cloud, the block where the point cloud is located is determined according to the coordinates of each object space point, the block where the point cloud is located is divided into the three-dimensional grid, then shielding judgment is carried out by judging whether other object space points near the line segment exist in the grid through which the connecting line segment of the object space point and the image shooting center passes, if other object space points exist near the line segment, the image to be selected is considered to be shielded by other ground objects when the object space point is shot, and the image which is not shielded is selected as the optimal associated image, so that the problem of coloring errors caused by shielding can be solved.

Drawings

FIG. 1 is a flow chart of a prior art method for generating a vehicle-mounted true color point cloud;

fig. 2 is a flowchart of a method for selecting an optimal associated image in vehicle-mounted point cloud coloring according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of an occlusion situation according to the present invention;

fig. 4 is a structural block diagram of a method for selecting an optimal associated image in vehicle-mounted point cloud coloring according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a hardware structure of a possible electronic device provided in the present invention;

fig. 6 is a schematic diagram of a hardware structure of a possible computer-readable storage medium according to the present invention.

Detailed Description

The principles and features of this invention are described below in conjunction with the following drawings, which are set forth by way of illustration only and are not intended to limit the scope of the invention.

The invention provides an optimal associated image selection method in vehicle-mounted point cloud coloring, which comprises the following steps:

and sequentially judging each image, and determining the optimal associated image of any object space point to be colored in the point cloud as the image meeting the conditions of avoiding the blind area and having no shielding.

That is, the optimal associated image should satisfy the following conditions:

(1) the projection point of the object space point on the optimal correlation image is in the effective area of the image, and the shooting blind area of the panoramic image is avoided.

(2) There is no occlusion between the point to be colored and the panoramic photography center.

The method for determining the image meeting the condition of avoiding the blind area comprises the following steps: and determining an effective area of the image, projecting the object space point on the image to obtain a projection point, and determining that the image meets the condition of avoiding the blind area when the projection point is in the effective area.

The method for determining the image meeting the condition of no occlusion comprises the following steps: and determining a photographing center corresponding to the image, and determining that the image meets the condition of no occlusion when other object points do not exist in a set distance range near a connecting line segment between the object point and the photographing center.

Selecting an image which is not shielded in the process of shooting the object point by the image as an optimal associated image, wherein the projection of the object point on the image is in an effective area; and when the image is shaded and judged, the object space point and the shooting center corresponding to the image form a line segment, if other object space points exist near the line segment, the image to be selected is considered to be shaded by other ground objects when the object space point is shot by the image to be selected, and the image which is not shaded is selected as the optimal associated image, so that the problem of shading errors caused by shading can be solved.

Example 1

Embodiment 1 provided in the present invention is an embodiment of a method for selecting an optimal associated image in vehicle-mounted point cloud coloring, and fig. 2 shows a flowchart of the method for selecting an optimal associated image in vehicle-mounted point cloud coloring, and as can be seen from fig. 2, the embodiment of the method for selecting an optimal associated image includes:

step 1, determining a polygonal effective area of the panoramic image.

Specifically, the effective area is selected by the user with a polygon frame.

Since the effective areas of all images in the same group of data are the same, the user only needs to select the images once.

And 2, dividing the point cloud into N blocks according to the coordinate range of the point cloud, and enabling i to be 0 and j to be 0, wherein i represents the serial number of the point cloud blocks, and j represents the serial number of the object space point.

Specifically, the process of dividing the point cloud into N blocks according to the coordinate range of the point cloud includes:

assuming that the coordinate ranges of the point clouds are (xmin, ymin, zmin) to (xmax, ymax, zmax), the coordinate range of the i-th block is (xmin + (i-1) d, ymin + (i-1) d) to (xmin + i d, ymin + i d).

And 3, sequentially determining the optimal associated image of each object space point in each point cloud. And determining the optimal associated image of any object space point to be colored in the point cloud as an image meeting the conditions of avoiding a blind area and having no shielding.

Specifically, step 3 includes:

step 301, an ith point cloud is taken out, the total number of object space points in the point cloud is M, whether i is less than N is judged, if yes, step 302 is executed, and if not, the whole process is ended.

Step 302, a jth object space point P in the ith point cloud is taken out.

Step 303, find the nearest center S in the image X to the point P, and project the point P onto the image X. X represents an ID of the image to be determined.

In step 304, it is determined whether the projection point is within the effective area of the polygon, if so, step 305 is performed, otherwise, step 306 is performed.

The method for determining the image meeting the condition of avoiding the blind area comprises the following steps: and projecting the object space point onto the image to obtain a projection point, and determining that the image meets the condition of avoiding the blind area when the projection point is in the effective area.

Step 305, judging whether the shielding exists between the point P and the shooting center S, if so, executing step 306, otherwise, executing step 307.

Preferably, the method for determining the image satisfying the condition of no occlusion includes: and determining a photographing center corresponding to the image, and determining that the image meets the condition of no occlusion when other object points do not exist in a set distance range near a connecting line segment between the object point and the photographing center.

Preferably, the process of determining that the image satisfies the condition of no occlusion includes: dividing a block where the point cloud to be colored is located into three-dimensional grids, determining each grid through which a connecting line segment passes, calculating the distance from each object space point in each grid to the connecting line segment, and determining that the object space point is not shielded under the shooting visual angle of the shooting center when the distances are all larger than a set distance threshold.

As shown in fig. 3, which is a schematic diagram of an occlusion situation provided by the present invention, it can be known from fig. 3 that, assuming that a line segment formed by a certain point P and a photographing center S in a point cloud passes through c grids, whether the points in the grids occlude a light ray PS is sequentially determined. As shown in fig. 3, assuming that points in the c meshes are P1, P2, P3 and P4, the distances from P1, P2, P3 and P4 to the straight line PS are calculated in sequence, and if the distance from any one point to the straight line is smaller than a distance threshold, it is considered that the point P is blocked in the imaging angle of view of the imaging center S.

In another embodiment provided by the present invention, each grid through which a connecting line segment passes is determined, a color point to be colored and an object point in each grid are projected onto the image, and if a pixel distance between a projection point of an object point and a projection point of a point to be colored is smaller than or equal to a set threshold, it is determined that the object point is shielded by the object point in the grid.

Step 306, find the next image, and let the value of X equal to the ID of the next image, go to step 303.

Step 307, assigning the RGB values of the projection point to the point P, and after j equals j +1, determining whether j < M is true, if so, executing step 302, otherwise, after i equals i +1 and j equals 0, executing step 301.

Preferably, traversing the point cloud, selecting an optimal associated image for each object space point to be colored, and giving the color attribute of the projection point of the object space point on the optimal associated image to the object space point, thereby finishing coloring.

Example 2

Embodiment 2 provided by the present invention is an embodiment of a system provided by the present invention, and as shown in fig. 4, is a structural diagram of an optimal associated image selection system in vehicle-mounted point cloud coloring provided by the embodiment of the present invention, and as shown in fig. 4, is an optimal associated image selection system in vehicle-mounted point cloud coloring, including: the device comprises a blind area avoiding judgment module, a shielding judgment module and an optimal associated image determination module;

the blind area avoiding judgment module is used for determining an effective area of the image, projecting the object space point onto the image to obtain a projection point, and determining that the image meets the condition of avoiding the blind area when the projection point is in the effective area;

the shielding judgment module is used for determining a photographing center corresponding to the image, and determining that the image meets the condition of no shielding when other object space points do not exist in the range of the set distance near the connecting line segment between the object space point and the photographing center;

and the optimal associated image determining module is used for sequentially judging the images and determining the optimal associated image of any object space point to be colored in the point cloud as the image meeting the conditions of avoiding the blind area and having no shielding.

It can be understood that the system for selecting the optimal associated image in vehicle-mounted point cloud coloring provided by the present invention corresponds to the method for selecting the optimal associated image in vehicle-mounted point cloud coloring provided by the foregoing embodiments, and the relevant technical features of the system for selecting the optimal associated image in vehicle-mounted point cloud coloring may refer to the relevant technical features of the method for selecting the optimal associated image in vehicle-mounted point cloud coloring, and are not described herein again.

Referring to fig. 5, fig. 5 is a schematic view of an embodiment of an electronic device according to an embodiment of the invention. As shown in fig. 5, an embodiment of the present invention provides an electronic device, which includes a memory 1310, a processor 1320, and a computer program 1311 stored in the memory 1320 and executable on the processor 1320, where the processor 1320 executes the computer program 1311 to implement the following steps:

sequentially judging each image, and determining the optimal associated image of any object space point to be colored in the point cloud as an image meeting the conditions of avoiding a blind area and having no shielding;

determining an effective area of an image, projecting object space points on the image to obtain projection points, and determining that the image meets the condition of avoiding a blind area when the projection points are in the effective area;

and determining a photographing center corresponding to the image, and determining that the image meets the condition of no occlusion when other object points do not exist in a set distance range near a connecting line segment between the object point and the photographing center.

Referring to fig. 6, fig. 6 is a schematic diagram of an embodiment of a computer-readable storage medium according to the present invention. As shown in fig. 6, the present embodiment provides a computer-readable storage medium 1400, on which a computer program 1411 is stored, which computer program 1411, when executed by a processor, implements the steps of:

sequentially judging each image, and determining the optimal associated image of any object space point to be colored in the point cloud as an image meeting the conditions of avoiding a blind area and having no shielding;

determining an effective area of an image, projecting object space points on the image to obtain projection points, and determining that the image meets the condition of avoiding a blind area when the projection points are in the effective area;

and determining a photographing center corresponding to the image, and determining that the image meets the condition of no occlusion when other object points do not exist in a set distance range near a connecting line segment between the object point and the photographing center.

According to the optimal associated image selection method, system and storage medium in vehicle-mounted point cloud coloring, the point cloud is divided into a plurality of blocks according to the coordinate range of the point cloud, the block where the point to be colored is located is divided into three-dimensional grids, each grid through which a connecting line segment passes is determined, the distance from each object space point in each grid to the connecting line segment is calculated, and when the distances are all larger than a set distance threshold value, it is determined that the object space point is not shielded under the shooting visual angle of the shooting center. The image which is not blocked is selected as the optimal related image, so that the problem of shading errors caused by blocking can be solved.

It should be noted that, in the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to relevant descriptions of other embodiments for parts that are not described in detail in a certain embodiment.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.