阅读说明：本技术 位置识别方法、装置和电子设备 (Position identification method and device and electronic equipment ) 是由 贾金让 于 2020-05-27 设计创作，主要内容包括：本申请公开了一种位置识别方法、装置和电子设备,涉及智能交通领域。具体实现方案为：通过获取三维的地面点云图,以及获取二维的相机图,根据地面点云图与相机图的映射关系,从地面点云图的各点中确定出目标点,确定在世界坐标系下目标点的地面位置,根据目标点的地面位置,对相机图中具有映射关系的像素进行定位,以得到像素呈现对象在世界坐标系下的位置。该方法通过高精度的三维地面点云图与相机图的映射关系,确定障碍物的位置,提高了自动驾驶和路侧感知障碍物的精准度。(The application discloses a position identification method and device and electronic equipment, and relates to the field of intelligent transportation. The specific implementation scheme is as follows: the method comprises the steps of determining a target point from each point of a ground point cloud picture according to the mapping relation between the ground point cloud picture and a camera picture by acquiring a three-dimensional ground point cloud picture and a two-dimensional camera picture, determining the ground position of the target point in a world coordinate system, and positioning pixels with the mapping relation in the camera picture according to the ground position of the target point to obtain the position of a pixel presentation object in the world coordinate system. According to the method, the position of the obstacle is determined through the high-precision mapping relation between the three-dimensional ground point cloud picture and the camera picture, and the accuracy of automatic driving and road side obstacle sensing is improved.)

1. A method of location identification, the method comprising:

acquiring a three-dimensional ground point cloud picture and acquiring a two-dimensional camera picture;

determining a target point from each point of the ground point cloud picture according to the mapping relation between the ground point cloud picture and the camera picture; the target point has a mapping relation with pixels in the camera image;

determining the ground position of the target point under a world coordinate system;

and positioning the pixels with the mapping relation in the camera image according to the ground position of the target point to obtain the position of the pixel presentation object in the world coordinate system.

2. The method according to claim 1, wherein the determining a target point from each point of the ground point cloud image according to the mapping relationship between the ground point cloud image and the camera image comprises:

acquiring first coordinates carried by each point in the ground point cloud picture; the first coordinate is the coordinate of each point in a radar coordinate system;

mapping the first coordinates of each point to a camera coordinate system according to the mapping relation between the ground point cloud picture and the camera picture to obtain second coordinates in the camera coordinate system;

determining a boundary range of the camera image in the camera coordinate system;

and taking the point of the ground point cloud picture, of which the second coordinate is within the boundary range, as the target point.

3. The method according to claim 2, wherein the mapping the first coordinates of the points to a camera coordinate system according to the mapping relationship between the ground point cloud image and the camera image to obtain second coordinates in the camera coordinate system comprises:

according to the external parameters of the radar, mapping the first coordinates of each point from a radar coordinate system where the ground point cloud picture is located to a world coordinate system to obtain intermediate coordinates under the world coordinate system;

and mapping the middle coordinate in the world coordinate system to the camera coordinate system according to the external parameters of the camera to obtain a second coordinate in the camera coordinate system.

4. The position recognition method according to claim 3,

the data precision of the first coordinate is a floating point type;

the data precision of the intermediate coordinate and the second coordinate is higher than that of the floating point type.

5. The position recognition method according to any one of claims 1 to 4, wherein the ground position is indicated by a ground normal vector; the positioning the pixels with mapping relation in the camera image according to the ground position of the target point to obtain the position of the pixel presentation object in the world coordinate system includes:

superposing the ground normal vector on each target point which is mapped to the same pixel in the camera image to obtain a superposition vector;

and positioning the pixels with the mapping relation in the camera image according to the superposition normal vector to obtain the position of the pixel presentation object in the world coordinate system.

6. The method according to any one of claims 1 to 4, wherein the acquiring a three-dimensional ground point cloud picture comprises:

generating a local point cloud picture of each frame according to data of each frame collected by a radar;

performing feature matching on each frame of local point cloud image to obtain the position relation among each frame of local point cloud image;

performing morphological filtering on each frame of local point cloud image to obtain a filtered local point cloud image;

and splicing the local point cloud pictures after filtering of each frame according to the position relation among the local point cloud pictures of each frame to obtain the ground point cloud picture.

7. The method according to claim 6, wherein the step of splicing the filtered local point cloud maps of the frames according to the position relationship between the local point cloud maps of the frames to obtain the ground point cloud map further comprises:

filtering by adopting a voxel grid filter;

and/or eliminating noise points from the ground point cloud picture.

8. A position recognition apparatus, comprising:

the acquisition module is used for acquiring a three-dimensional ground point cloud picture and acquiring a two-dimensional camera picture;

the determining module is used for determining a target point from each point of the ground point cloud picture according to the mapping relation between the ground point cloud picture and the camera picture; the target point has a mapping relation with pixels in the camera image;

the reconstruction module is used for determining the ground position of the target point in a world coordinate system;

and the positioning module is used for positioning the pixels with the mapping relation in the camera image according to the ground position of the target point so as to obtain the position of the pixel presentation object in the world coordinate system.

9. The position recognition apparatus of claim 8, wherein the determining module comprises:

the acquisition unit is used for acquiring first coordinates carried by each point in the ground point cloud picture; the first coordinate is the coordinate of each point in a radar coordinate system;

the mapping unit is used for mapping the first coordinates of each point to a camera coordinate system according to the mapping relation between the ground point cloud picture and the camera picture to obtain second coordinates in the camera coordinate system;

a first determination unit configured to determine a boundary range of the camera map in the camera coordinate system;

and the second determining unit is used for taking the point of the ground point cloud picture, of which the second coordinate is within the boundary range, as the target point.

10. The position recognition apparatus of claim 9, wherein the mapping unit is further configured to:

according to the external parameters of the radar, mapping the first coordinates of each point from a radar coordinate system where the ground point cloud picture is located to a world coordinate system to obtain intermediate coordinates under the world coordinate system;

and mapping the coordinates in the world coordinate system to the camera coordinate system according to the external parameters of the camera to obtain second coordinates in the camera coordinate system.

11. The position recognition apparatus according to claim 10,

the data precision of the first coordinate is a floating point type;

the data precision of the intermediate coordinate and the second coordinate is higher than that of the floating point type.

12. The position identifying device of any one of claims 8-11, wherein the ground position is indicated by a ground normal vector; the positioning module further comprises:

the superposition unit is used for superposing the ground normal vector on each target point which is mapped with the same pixel in the camera image to obtain a superposition vector;

and the positioning unit is used for positioning the pixels with the mapping relation in the camera image according to the superposition vector to obtain the positions of the pixel presentation objects in the world coordinate system.

13. The position recognition apparatus according to any one of claims 8 to 11, wherein the obtaining module includes:

the generating unit is used for generating a local point cloud picture of each frame according to data of each frame collected by the radar;

the matching unit is used for carrying out feature matching on each frame of local point cloud picture to obtain the position relation among each frame of local point cloud picture;

the filtering unit is used for performing morphological filtering on each frame of local point cloud image to obtain a filtered local point cloud image;

and the splicing unit is used for splicing the local point cloud pictures after filtering each frame according to the position relation among the local point cloud pictures of each frame to obtain the ground point cloud picture.

14. The position recognition apparatus of claim 13, wherein the obtaining module comprises:

the filtering unit is also used for filtering by adopting a voxel grid filter; and/or the rejecting unit is used for rejecting noise points from the ground point cloud picture.

15. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the location identification method of any of claims 1-7.

16. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the location identification method according to any one of claims 1 to 7.

Technical Field

The present application relates to the field of computer vision technologies in the field of intelligent transportation, and in particular, to a method and an apparatus for identifying a location, and an electronic device.

Background

With the development of the automatic driving technology, an intelligent traffic system integrating a plurality of functions of environment dynamic collaborative perception, data information processing, transmission and storage, traffic control and management and the like is developed, so that the modern advanced technology is fully utilized, and all subjects such as people, vehicles, roads and the like become safer, more intelligent and more efficient.

In practical application, in a roadside perception scene based on a roadside camera, due to the fact that the camera lacks depth information, when the real position of an obstacle in a two-dimensional image in a three-dimensional world is calculated, a ground equation of the position of the obstacle, namely a ground normal vector, is needed. And the more accurate the ground equation is, the more accurate the finally determined position of the obstacle in the world coordinate system is.

Disclosure of Invention

The application provides a position identification method, a position identification device, position identification equipment and a storage medium.

An embodiment of a first aspect of the present application provides a location identification method, including:

acquiring a three-dimensional ground point cloud picture and acquiring a two-dimensional camera picture;

determining a target point from each point of the ground point cloud picture according to the mapping relation between the ground point cloud picture and the camera picture; the target point has a mapping relation with pixels in the camera image;

determining the ground position of the target point under a world coordinate system;

and positioning the pixels with the mapping relation in the camera image according to the ground position of the target point to obtain the position of the pixel presentation object in the world coordinate system.

An embodiment of a second aspect of the present application provides a position identification apparatus, including:

the acquisition module is used for acquiring a three-dimensional ground point cloud picture and acquiring a two-dimensional camera picture;

the determining module is used for determining a target point from each point of the ground point cloud picture according to the mapping relation between the ground point cloud picture and the camera picture; the target point has a mapping relation with pixels in the camera image;

the reconstruction module is used for determining the ground position of the target point in a world coordinate system;

and the positioning module is used for positioning the pixels with the mapping relation in the camera image according to the ground position of the target point so as to obtain the position of the pixel presentation object in the world coordinate system.

An embodiment of a third aspect of the present application provides an electronic device, including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the location identification method of the embodiments of the first aspect.

A fourth aspect of the present application provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the position identification method of the first aspect.

One embodiment in the above application has the following advantages or benefits: the method comprises the steps of determining a target point from each point of a ground point cloud picture according to the mapping relation between the ground point cloud picture and a camera picture by acquiring a three-dimensional ground point cloud picture and a two-dimensional camera picture, determining the ground position of the target point in a world coordinate system, and positioning pixels with the mapping relation in the camera picture according to the ground position of the target point to obtain the position of a pixel presentation object in the world coordinate system. According to the method, the position of the obstacle is determined through the high-precision mapping relation between the three-dimensional ground point cloud picture and the camera picture, and the accuracy of automatic driving and road side obstacle sensing is improved.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:

fig. 1 is a schematic flowchart of a location identification method according to an embodiment of the present application;

fig. 2 is a schematic flowchart of a location identification method according to a second embodiment of the present application;

fig. 3 is a schematic flowchart of a location identification method according to a third embodiment of the present application;

fig. 4 is a schematic flowchart of a position identification method according to a fourth embodiment of the present application;

fig. 5 is an exemplary diagram of a location identification method according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of a position identification device according to a fifth embodiment of the present application;

fig. 7 is a block diagram of an electronic device for implementing a method of location identification according to an embodiment of the present application.

Detailed Description

The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

A position recognition method, apparatus, electronic device, and storage medium of embodiments of the present application are described below with reference to the accompanying drawings.

Fig. 1 is a schematic flowchart of a location identification method according to an embodiment of the present application.

The embodiment of the present application is exemplified by the location identification method being configured in a location identification device, and the question answering processing device can be applied to any electronic equipment, so that the electronic equipment can execute a location identification function.

The electronic device may be a Personal Computer (PC), a cloud device, a mobile device, and the like, and the mobile device may be a hardware device having various operating systems, such as a mobile phone, a tablet Computer, a Personal digital assistant, a wearable device, and an in-vehicle device.

As shown in fig. 1, the position identification method may include the following steps:

step S101, acquiring a three-dimensional ground point cloud picture and acquiring a two-dimensional camera picture.

In the embodiment of the present application, the three-dimensional ground point cloud is an original data point group used for representing three-dimensional scanning, And is obtained by measuring through a laser Detection And measurement (Light Detection And Ranging, abbreviated as LiDAR) device, that is, by measuring through a laser radar. LiDAR carries out laser scanning by utilizing a Global positioning System (GPS for short) and an Inertial Measurement Unit (IMU for short), measured data of the LiDAR is represented by discrete points of a Digital Surface Model (DSM for short), and the data contain space three-dimensional information and laser intensity information. That is, three-dimensional ground point clouds can be measured by LiDAR.

The advantages of lidar are that the range frequency is high, accurate, and the measurement yields relatively fixed errors and is independent of range. Lidar is a very useful range finding sensor in navigation, and laser has the advantage that it can directly obtain accurate distance information from the surface of an object to a point, so the lidar can be used for positioning and mapping at the same time.

The objects contained in the ground point cloud picture may be pedestrians, vehicles, trees, buildings, and the like.

In the embodiment of the application, the roadside camera shoots an object in a visual field to obtain a two-dimensional camera image, and the two-dimensional camera image is sent to the electronic equipment, so that the electronic equipment can obtain the two-dimensional camera image. Similarly, the two-dimensional camera image captured by the roadside camera may include pedestrians, vehicles, trees, buildings, and the like.

It should be noted that the LiDAR and the roadside camera capture the same object, and the electronic device and the roadside camera may be integrally arranged or may be independently arranged, which is not limited in this application. For example, the electronic device and roadside camera may be disposed in an unmanned vehicle.

And S102, determining a target point from each point of the ground point cloud picture according to the mapping relation between the ground point cloud picture and the camera picture.

The target point and the pixel in the camera image have a mapping relation.

In the application, the three-dimensional ground point cloud picture and the two-dimensional camera picture which are acquired by the electronic equipment are obtained by shooting the same object, so that a fixed mapping relation exists between the three-dimensional point cloud in the ground point cloud picture and the pixel coordinates of the image in the camera picture.

In the embodiment of the application, after the electronic device acquires the three-dimensional ground point cloud picture and the two-dimensional camera picture, a target point having a mapping relation with a pixel of the camera can be determined from each point of the ground point cloud picture according to the mapping relation between the ground point cloud picture and the camera picture.

And step S103, determining the ground position of the target point in the world coordinate system.

It should be noted that, according to the mapping relationship between the ground point cloud image and the camera image, the target point determined from the ground point cloud image is a point in the camera coordinate system, and considering the uncertainty of the camera coordinate system, the target point in the camera coordinate system needs to be converted into the world coordinate system, and the ground position of the target point in the world coordinate system is determined by three-dimensionally reconstructing the ground point cloud image.

For example, the transformation relationship between the camera coordinate system and the world coordinate system can be represented by a rotation matrix R and a translation matrix t, wherein the transformation process is shown as the following formula:

wherein in the above formula (X)_c,Y_c,Z_c) Is point cloud coordinate under camera coordinate system, (X)_w,Y_w,Z_w) Is the point cloud coordinate in the world coordinate system, t is the three-dimensional translation vector, and t is [ t ═ t [ [ t ]_x,t_y,t_z]The rotation matrix R is a3 × 3 unit orthogonal matrix, and the matrix elements satisfy the following conditions:

according to the method and the device, after a three-dimensional ground point cloud picture which has a mapping relation with pixels in a two-dimensional camera picture is obtained, the three-dimensional reconstruction is carried out on the ground point cloud picture through the registration and fusion of point cloud data, and therefore the ground position of a target point in a world coordinate system is determined.

The three-dimensional reconstruction is to depict a real scene into a mathematical model which accords with the logical expression of a computer through the processes of depth data acquisition, preprocessing, point cloud registration and fusion, surface generation and the like.

As a possible implementation manner, after acquiring a three-dimensional ground point cloud image, the electronic device may perform preprocessing operations such as simplification and noise reduction on the three-dimensional cloud image to enhance the three-dimensional cloud image, further convert point cloud data in the three-dimensional cloud image into a world coordinate system according to a change relationship between the camera coordinate system and the world coordinate system, and fuse the converted data, and further may generate a completed three-dimensional surface by using a mobile cube method in a classical voxel-level reconstruction algorithm to obtain a three-dimensional scene of the ground, so that a ground position of a target point in the world coordinate system may be determined according to coordinates of each target point converted into the world coordinate system.

And step S104, positioning the pixels with the mapping relation in the camera image according to the ground position of the target point to obtain the position of the pixel presentation object in the world coordinate system.

In the embodiment of the application, after the ground position of the target point in the world coordinate system is determined, the pixels in the camera map, which have a mapping relationship with each target point in the world coordinate system, can be located according to the mapping relationship between the ground point cloud map and the camera map, so as to determine the position of the object represented by each pixel in the world coordinate system.

The pixel presenting object may be an obstacle such as a pedestrian, a vehicle, a tree, a building, and the like.

It should be noted that, the above steps S101 and S102 can be regarded as an offline part, and the steps S103 and S104 are an online part, in this application, the target point is determined from each point of the ground point cloud graph in an offline state, and the determined target point can be applied to online piecewise ground equation calculation, so as to speed up the speed of position identification.

According to the position identification method, the three-dimensional ground point cloud picture is obtained, the two-dimensional camera picture is obtained, the target point is determined from each point of the ground point cloud picture according to the mapping relation between the ground point cloud picture and the camera picture, the ground position of the target point in the world coordinate system is determined, and the pixel with the mapping relation in the camera picture is positioned according to the ground position of the target point, so that the position of the pixel presentation object in the world coordinate system is obtained. According to the method, the position of the obstacle is determined through the high-precision mapping relation between the three-dimensional ground point cloud picture and the camera picture, and the accuracy of automatic driving and road side obstacle sensing is improved.

On the basis of the above embodiments, the present application proposes another position identification method.

Fig. 2 is a schematic flowchart of a location identification method according to a second embodiment of the present application.

As shown in fig. 2, the position recognition method may include the steps of:

step S201, a three-dimensional ground point cloud picture is obtained, and a two-dimensional camera picture is obtained.

In the embodiment of the present application, the implementation process of step S201 may refer to the implementation process of step S101 in the foregoing embodiment, and details are not described here.

Step S202, first coordinates carried by each point in the ground point cloud picture are obtained.

The first coordinate is the coordinate of each point in the ground point cloud picture in a radar coordinate system.

In the embodiment of the application, the three-dimensional ground point cloud is obtained according to laser radar measurement, so that coordinates in a radar coordinate system and laser reflection intensity are carried in each point cloud. The radar coordinate system can use the position of the laser radar as the origin of coordinates, the direction pointing to the right side of the radar as the positive direction of an X axis, and the direction pointing to the front of the radar as the positive direction of a Y axis.

For the sake of convenience of distinction, in the present application, the coordinates of each point in the ground point cloud image in the radar coordinate system are referred to as first coordinates, and the coordinates of each point in the camera coordinate system are referred to as second coordinates.

And step S203, mapping the first coordinates of each point to a camera coordinate system according to the mapping relation between the ground point cloud picture and the camera picture to obtain second coordinates in the camera coordinate system.

As a possible implementation manner, after acquiring the first coordinates of each point in the ground point cloud picture in the radar coordinate system, the electronic device maps the first coordinates of each point from the radar coordinate system where the ground point cloud picture is located to the world coordinate system according to the external reference of the radar, so as to obtain the intermediate coordinates in the world coordinate system. Further, according to the external parameters of the camera, the middle coordinates in the world coordinate system are mapped to the camera coordinate system to obtain second coordinates in the camera coordinate system.

The external parameters of the camera determine the relative position relationship between the camera coordinate system and the world coordinate system, and can be a rotation matrix and a translation matrix, and the rotation matrix and the translation matrix jointly describe how to convert each point from the world coordinate system to the camera coordinate system. Wherein the rotation matrix describes the direction of the coordinate axes of the world coordinate system relative to the camera coordinate axes; the translation matrix describes the position of the spatial origin in the camera coordinate system.

Therefore, according to the coordinates of each point in the ground point cloud picture in the radar coordinate system, the second coordinates of each point in the camera coordinate system are determined, and therefore the coordinates of each pixel point in the two-dimensional camera picture are determined.

It should be noted that, each point in the three-dimensional ground point cloud map is not mapped into the world coordinate system from the radar coordinate system immediately after acquiring the data of each point, because the data precision of the first coordinate of each point cloud in the radar coordinate system is a floating point type, the middle coordinate in the world coordinate system and the second coordinate in the camera coordinate system are both higher than the floating point type, and when the effective number of the floating point type is insufficient, the precision of the point is lost, so the coordinates of each point in the radar coordinate system are stored by using the floating point type, and the coordinates of each point are mapped into the world coordinate system when in use.

As an example, a file for storing point cloud data in a radar coordinate system generally only supports point cloud data of a floating point type, while data precision in a world coordinate system is generally 6-bit or even 7-bit, and the precision of a point is lost due to insufficient effective numbers of the floating point type, so that the file is used for storing the result in the radar coordinate system, and coordinates of each point are mapped into the world coordinate system when used.

In step S204, in the camera coordinate system, a boundary range of the camera image is determined.

In the embodiment of the present application, the boundary range of the camera image may be determined in the camera coordinate system according to the coordinates of each pixel in the two-dimensional camera image.

And step S205, taking the point of the ground point cloud picture with the second coordinate in the boundary range as a target point.

In the embodiment of the application, after the second coordinates of each point in the ground point cloud picture in the camera coordinate system are determined, whether the second coordinates corresponding to each point are within the boundary range of the camera picture can be determined according to the second coordinate system corresponding to each point, so as to determine whether each point is a target point.

As a possible situation, if the second coordinate corresponding to the point in the ground point cloud picture is not in the boundary range of the camera picture, it is determined that the corresponding point in the ground point cloud picture is not in the range shot by the roadside camera, and the point is removed.

As another possible case, if the second coordinate of the point in the ground point cloud image is within the boundary range, the point in the ground point cloud image where the second coordinate is within the boundary range is used as the target point, so that the target point can be used to generate the high-precision segmented ground equation. Therefore, whether each point is a target point or not is determined through the second coordinate of each point in the ground point cloud picture in the camera coordinate system, so that dense and uniform ground point cloud in the high-precision map is determined, a high-precision ground equation can be generated, and the accuracy of automatic driving and road side perception is improved.

Step S206, determining the ground position of the target point in the world coordinate system.

Step S207, positioning the pixels with the mapping relation in the camera image according to the ground position of the target point to obtain the position of the pixel presentation object in the world coordinate system.

In the embodiment of the present application, the implementation processes of step S206 and step S207 may refer to the implementation processes of step S103 and step S104 in the foregoing embodiment, and are not described herein again.

According to the position identification method, a three-dimensional ground point cloud picture is obtained, a two-dimensional camera picture is obtained, and first coordinates carried by each point in the ground point cloud picture are obtained; mapping the first coordinates of each point to a camera coordinate system according to the mapping relation between the ground point cloud picture and the camera picture to obtain second coordinates in the camera coordinate system; determining a boundary range of a camera image in a camera coordinate system; and determining the ground position of the target point in the world coordinate system by taking the point of the ground point cloud picture with the second coordinate within the boundary range as the target point, and positioning the pixels with the mapping relation in the camera picture according to the ground position of the target point to obtain the position of the pixel presentation object in the world coordinate system. Therefore, the ground point cloud is scanned through the laser radar to obtain a dense ground point cloud map, and points which are not in the boundary range of the camera image in the ground point cloud map are removed, so that point cloud data with high enough precision is generated, and the precision of automatic driving and road side perception is improved.

On the basis of the above embodiment, as a possible case, the ground position of the target point in the world coordinate system may be indicated by a ground normal vector, and thus, the ground normal vector may be superimposed on each target point of the same pixel in the mapping camera image to obtain a superimposed vector, so as to position, according to the superimposed vector, the object represented by the pixel having the mapping relationship in the camera image in the world coordinate system. The above process is described in detail with reference to fig. 3, and fig. 3 is a schematic flowchart of a location identification method according to a third embodiment of the present application.

As shown in fig. 3, the position identification method may further include the following steps:

step S301, a three-dimensional ground point cloud picture is obtained, and a two-dimensional camera picture is obtained.

Step S302, determining a target point from each point of the ground point cloud picture according to the mapping relation between the ground point cloud picture and the camera picture.

Step S303, determining the ground position of the target point in the world coordinate system.

In the embodiment of the present application, the implementation process of step S301 to step S303 may refer to the implementation process of step S101 to step S103 in the above embodiment, and is not described herein again.

Step S304, the ground normal vector is superposed on each target point of the same pixel in the mapping camera image to obtain a superposition vector.

In the method, the ground point cloud picture is reconstructed in three dimensions to obtain the ground position of the target point in the world coordinate system. Wherein the ground position of the target point is indicated by a ground normal vector.

In a possible case, the target points determined from each point in the three-dimensional ground point cloud picture are mapped to each target point with the same pixel in the camera picture, and the ground normal vectors of the target points can be superposed to obtain a superposition normal vector.

As an example, assuming that the target points a1, a2, and A3 are determined from each point of the ground point cloud image, and a1 and a2 have a mapping relation with the same pixel in the camera image, in this case, the ground normal vectors of the target points a1 and a2 may be superimposed to obtain a superimposed vector.

Step S305, according to the superposition vector, positioning the pixels with the mapping relation in the camera image to obtain the position of the pixel display object in the world coordinate system.

In the embodiment of the application, the ground normal vectors corresponding to the target points are superposed on the target points of the same pixel in the mapping camera image to obtain the superposition vector, and the object represented by the pixel with the mapping relation in the camera image is positioned in the world coordinate system according to the superposition vector.

Therefore, after the ground point cloud picture is reconstructed, the ground normal vectors of the target points in the world coordinate system are obtained, the ground normal vectors of the target points of the same pixel in the mapping camera picture are superposed, so that the pixel with the mapping relation in the camera picture is positioned according to the superposed normal vectors, the position of the pixel presenting object in the world coordinate system is obtained, the position of the obstacle can be accurately positioned in the automatic driving and roadside sensing processes, and the accuracy of automatic driving and roadside sensing is improved.

As a possible implementation manner, when a three-dimensional ground point cloud picture is obtained, a corresponding frame local point cloud picture may be generated according to each frame of data acquired by a radar, so as to perform feature matching and morphological filtering on each frame of local point cloud picture, and further, according to the position relationship of each frame of local point cloud picture, the local point cloud pictures after each frame of filtering are spliced to obtain the ground point cloud picture. The above process is described in detail with reference to fig. 4, and fig. 4 is a schematic flowchart of a location identification method according to a fourth embodiment of the present application.

As shown in fig. 4, the position identification method may further include the following steps:

step S401, according to each frame of data collected by the radar, each frame of local point cloud picture is generated.

As an implementation manner, after each frame of data is acquired by radar, a map of the data acquired by the radar may be created by using a LOAM to generate a local point cloud map of each frame.

The overall idea of LOAM is to separate the complex time-based localization and mapping problem into high-frequency motion estimation and low-frequency environment mapping.

And S402, performing feature matching on each frame of local point cloud images to obtain the position relation among the frames of local point cloud images.

In the embodiment of the application, feature matching is performed on each frame of local point cloud image, which can be understood as matching between two frames of laser radar data. For example, if the local point cloud image of the current frame is A and the local point cloud image of another frame matched with the current frame is B, if A is taken as the starting frame and B is taken as the target frame, A is transformed to B through a relative translation and rotation, and the position relation between the two local point cloud images can be determined by determining the relative translation amount and the rotation angle of A.

It should be noted that, the process of performing feature matching on each frame of local point cloud image is less in calculation amount, and therefore, the process can be performed multiple times to determine the position relationship of each frame of local point cloud image.

And step S403, performing morphological filtering on each frame of local point cloud image to obtain a filtered local point cloud image.

It should be explained that morphology is a concept in biology by nature, but for image processing, morphology refers to morphological filtering in mathematical terms, in particular to filtering processing of images. The image denoising method has the same nature as other filters, and can perform denoising, enhancing and other effects on the image.

In the method, after the corresponding frame local point cloud picture is generated according to each frame of data collected by the radar, each frame of local point cloud picture is subjected to morphological filtering to obtain the filtered local point cloud picture.

Because the data volume of each frame collected by the radar is large, the number of the point clouds is linearly increased along with the increase of time, the filtering of the whole point cloud image occupies a large amount of memory resources, and the time consumption is large, so that the filtering speed of the point cloud image is improved by performing morphological filtering on each frame of local point cloud image.

And S404, splicing the local point cloud pictures after filtering of each frame according to the position relation among the local point cloud pictures of each frame to obtain the ground point cloud picture.

In the embodiment of the application, after the position relationship between the local point cloud pictures of each frame is determined, the filtered local point cloud pictures of each frame can be spliced according to the position relationship between the local point cloud pictures of each frame to obtain the overall ground point cloud picture.

For example, the filtered local point cloud images of each frame may be superimposed according to the position relationship between the local point cloud images of each frame to obtain a complete ground point cloud image.

In a possible case, the moving speed of each frame of the radar is possibly inconsistent when the data are collected, so that the point cloud density of each position is inconsistent, and meanwhile, the final ground equation calculation preferably uses uniform point cloud, and after a ground point cloud picture is obtained, a voxel grid filter can be adopted to filter the whole ground point cloud picture. The ground point cloud picture can keep the original geometric form to the maximum extent after being filtered by the voxel grid filter, and the point cloud can be more uniform.

In the present application, the voxel grid filter is used to down-sample the point cloud in the point cloud map. If the point clouds are collected by using equipment such as a high-resolution camera, the point clouds are often dense, however, the subsequent segmentation work is difficult due to the excessive number of the point clouds. Therefore, in the application, the ground point cloud picture is filtered by adopting the voxel grid filter, so that the function of down-sampling without destroying the geometrical structure of the point cloud can be achieved. The geometrical structure of the point cloud is not only a macroscopic geometrical shape, but also a microscopic arrangement thereof, such as a similar size in the transverse direction and a same distance in the longitudinal direction. The existing random down-sampling mode has higher sampling efficiency than a voxel grid filter, but can damage the point cloud microstructure. Therefore, the voxel grid filter is adopted to filter the ground point cloud picture, and the original geometric structure of the ground point cloud picture can be kept to the maximum extent.

In another possible case, after obtaining the complete ground point cloud picture, noise points in the ground point cloud picture can be removed to obtain point cloud data with high enough precision.

In another possible case, after obtaining the complete ground point cloud picture, filtering may be performed by using a voxel grid filter, so that the ground point cloud picture can maintain its original geometric structure to the maximum extent, so that the point cloud of the ground point cloud picture is more uniform, and further noise points in the ground point cloud picture are removed, so as to obtain high-precision point cloud data.

According to the position identification method, each frame of local point cloud picture is generated according to each frame of data collected by a radar, feature matching is conducted on each frame of local point cloud picture, the position relation among each frame of local point cloud picture is obtained, morphological filtering is conducted on each frame of local point cloud picture, the filtered local point cloud pictures are obtained, and the filtered local point cloud pictures of each frame are spliced according to the position relation among each frame of local point cloud picture, so that the ground point cloud picture is obtained. According to the method, the local point cloud pictures after filtering of each frame are spliced according to the position relation of each frame of local point cloud picture to obtain the dense and uniform ground point cloud picture, and the technical problems that point clouds in a high-precision map are not dense enough and a segmented ground equation cannot be generated at any position in the prior art are solved.

Illustratively, referring to fig. 5, the execution process of the location identification method of the present application is divided into an offline part and an online part, wherein the offline part may include the following processes: the method comprises the steps of establishing a map of data collected by a radar by using LOAM to generate a local point cloud picture of each frame, performing morphological filtering on the local point cloud picture of each frame to obtain a filtered local point cloud picture, splicing the filtered local point cloud pictures of each frame according to the position relation among the local point cloud pictures of each frame, filtering the spliced point cloud pictures by using a voxel grid filter, and removing noise points from the filtered point cloud pictures to obtain a high-precision ground point cloud picture.

The online portion may include the following processes: and mapping the coordinates of each point in the ground point cloud picture to a world coordinate system according to the external parameters of the radar to obtain intermediate coordinates in the world coordinate system, and mapping the intermediate coordinates in the world coordinate system to a camera coordinate system according to the external parameters of the camera to obtain second coordinates in the camera coordinate system. Further, points which are not in the shooting range of the camera are removed, and a target point in the ground point cloud picture is obtained. The high-precision segmented ground equation can be generated according to the high-precision and uniform target points, so that the road side perception and the automatic driving precision can be improved.

It should be noted that the above-mentioned process is only an exemplary expression, and the implementation process of the specific location identification method is not limited herein.

In order to implement the above embodiments, the present application proposes a position recognition apparatus.

Fig. 6 is a schematic structural diagram of a position identification device according to a fifth embodiment of the present application.

As shown in fig. 6, the position recognition apparatus 500 may include: an acquisition module 510, a determination module 520, a reconstruction module 530, and a location module 540.

The obtaining module 510 is configured to obtain a three-dimensional ground point cloud image and obtain a two-dimensional camera image.

A determining module 520, configured to determine a target point from each point of the ground point cloud image according to a mapping relationship between the ground point cloud image and the camera image; and the target point has a mapping relation with the pixel in the camera image.

A reconstruction module 530 for determining a ground location of the target point in the world coordinate system.

And the positioning module 540 is configured to position the pixel having the mapping relationship in the camera image according to the ground position of the target point, so as to obtain a position of the pixel presentation object in the world coordinate system.

As a possible case, the determining module 520 may further include:

the acquisition unit is used for acquiring first coordinates carried by each point in the ground point cloud picture; the first coordinate is the coordinate of each point in the radar coordinate system.

The mapping unit is used for mapping the first coordinates of each point to a camera coordinate system according to the mapping relation between the ground point cloud picture and the camera picture to obtain second coordinates in the camera coordinate system;

a first determining unit for determining a boundary range of the camera image in the camera coordinate system;

and the second determining unit is used for taking the point of the ground point cloud picture with the second coordinate in the boundary range as the target point.

As another possible case, the mapping unit is further configured to:

according to external parameters of the radar, mapping the first coordinates of each point from a radar coordinate system where a ground point cloud picture is located to a world coordinate system to obtain intermediate coordinates under the world coordinate system; and mapping the coordinates in the world coordinate system to a camera coordinate system according to the external parameters of the camera to obtain second coordinates in the camera coordinate system.

As another possible case, the data precision of the first coordinate is of a floating point type; the data precision of the intermediate coordinate and the second coordinate is higher than that of a floating point type.

As another possible scenario, the ground position is indicated by a ground normal vector; the positioning module 540 may further include:

the superposition unit is used for superposing the ground normal vector on each target point of the same pixel in the mapping camera image to obtain a superposition vector;

and the positioning unit is used for positioning the pixels with the mapping relation in the camera image according to the superposition vector to obtain the position of the pixel presentation object in the world coordinate system.

As another possible case, the obtaining module 510 may further include:

the generating unit is used for generating a local point cloud picture of each frame according to data of each frame collected by the radar;

the matching unit is used for carrying out feature matching on each frame of local point cloud picture to obtain the position relation among each frame of local point cloud picture;

the filtering unit is used for performing morphological filtering on each frame of local point cloud image to obtain a filtered local point cloud image;

and the splicing unit is used for splicing the local point cloud pictures after filtering each frame according to the position relation among the local point cloud pictures of each frame to obtain the ground point cloud picture.

As another possible case, the filtering unit is further configured to perform filtering by using a voxel grid filter; and/or, the obtaining module further comprises: and the eliminating unit is used for eliminating noise points from the ground point cloud picture.

It should be noted that the explanation of the embodiment of the position identification method is also applicable to the position identification apparatus of the embodiment, and is not repeated herein.

The position recognition device of the embodiment of the application determines a target point from each point of the ground point cloud picture by acquiring a three-dimensional ground point cloud picture and a two-dimensional camera picture according to the mapping relation between the ground point cloud picture and the camera picture, determines the ground position of the target point in a world coordinate system, and positions the pixel with the mapping relation in the camera picture according to the ground position of the target point so as to obtain the position of a pixel presentation object in the world coordinate system. According to the method, the position of the obstacle is determined through the high-precision mapping relation between the three-dimensional ground point cloud picture and the camera picture, and the accuracy of automatic driving and road side obstacle sensing is improved.

According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.

Fig. 7 is a block diagram of an electronic device according to the location identification method of the embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.

As shown in fig. 7, the electronic apparatus includes: one or more processors 601, memory 602, and interfaces for connecting the various components, including a high-speed interface and a low-speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). Fig. 7 illustrates an example of a processor 601.

The memory 602 is a non-transitory computer readable storage medium as provided herein. Wherein the memory stores instructions executable by at least one processor to cause the at least one processor to perform the method of location identification provided herein. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the method of location identification provided herein.

The memory 602, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules (e.g., the obtaining module 510, the determining module 520, the reconstructing module 530, and the positioning module 540 shown in fig. 6) corresponding to the method of location identification in the embodiments of the present application. The processor 601 executes various functional applications of the server and data processing, i.e., implementing the method of location identification in the above-described method embodiments, by running non-transitory software programs, instructions, and modules stored in the memory 602.

The memory 602 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the electronic device identified by the location, and the like. Further, the memory 602 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 602 optionally includes memory located remotely from the processor 601, and these remote memories may be connected to the location-identifying electronic device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The electronic device of the method of location identification may further include: an input device 603 and an output device 604. The processor 601, the memory 602, the input device 603 and the output device 604 may be connected by a bus or other means, and fig. 7 illustrates the connection by a bus as an example.

The input device 603 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the position-recognized electronic apparatus, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointing stick, one or more mouse buttons, a track ball, a joystick, or other input devices. The output devices 604 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.

Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

According to the technical scheme of the embodiment of the application, a three-dimensional ground point cloud picture is obtained, a two-dimensional camera picture is obtained, a target point is determined from each point of the ground point cloud picture according to the mapping relation between the ground point cloud picture and the camera picture, the ground position of the target point in a world coordinate system is determined, and pixels with the mapping relation in the camera picture are positioned according to the ground position of the target point, so that the position of a pixel presentation object in the world coordinate system is obtained. According to the method, the position of the obstacle is determined through the high-precision mapping relation between the three-dimensional ground point cloud picture and the camera picture, and the accuracy of automatic driving and road side obstacle sensing is improved.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.

The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.