阅读说明：本技术 激光雷达测距方法、装置、计算机设备和存储介质 (Laser radar ranging method and device, computer equipment and storage medium ) 是由 何一雄 于 2019-12-10 设计创作，主要内容包括：一种激光雷达测距方法,包括：获取激光雷达接收的回波图像(302)；判断回波图像是否存在串扰像素点(304)；当回波图像存在串扰像素点时,校正串扰像素点,得到校正后的回波图像(306)；根据校正后的回波图像计算得到物体位姿信息(308)。(A laser radar ranging method, comprising: acquiring an echo image (302) received by a laser radar; judging whether crosstalk pixel points exist in the echo image (304); when crosstalk pixel points exist in the echo image, the crosstalk pixel points are corrected to obtain a corrected echo image (306); and calculating the object pose information according to the corrected echo image (308).)

1. A laser radar ranging method, comprising:

acquiring an echo image received by a laser radar;

judging whether crosstalk pixel points exist in the echo image or not;

when crosstalk pixel points exist in the echo image, the crosstalk pixel points are corrected to obtain a corrected echo image; and

and calculating to obtain object pose information according to the corrected echo image.

2. The method of claim 1, wherein the echo image comprises a depth image, and the determining whether crosstalk pixel points exist in the echo image comprises:

acquiring a first edge pixel point of the depth image;

judging whether the first edge pixel point meets a preset crosstalk condition or not; and

and when the first edge pixel point meets a preset crosstalk condition, determining the first edge pixel point as a crosstalk pixel point of the echo image.

3. The method of claim 2, wherein the echo image further comprises a grayscale image, and wherein the obtaining a first edge pixel point of the depth image comprises:

acquiring a second edge pixel point of the gray image;

searching a corresponding pixel point of the second edge pixel point in the depth image; and

and determining the corresponding pixel points as first edge pixel points of the depth image.

4. The method of claim 2, wherein the determining whether the first edge pixel meets a predetermined crosstalk condition comprises:

acquiring an edge distance measurement value and a gradient direction of the first edge pixel point;

acquiring first average ranging values of the first edge pixel point and all surrounding pixel points on the same side of the gradient direction;

acquiring second average ranging values of the first edge pixel point and all surrounding pixel points on the side opposite to the gradient direction;

and when the absolute value of the difference value between the edge ranging value and the first average ranging value is greater than a preset distance threshold value and the absolute value of the difference value between the edge ranging value and the second average ranging value is greater than the preset distance threshold value, the first edge pixel point meets a preset crosstalk condition.

5. The method of claim 4, wherein the surrounding pixels comprise pixels adjacent to the first edge pixel.

6. The method of claim 1, wherein the operating frequency of the lidar includes at least two different modulation frequencies, and the determining whether crosstalk pixel points exist in the echo image includes:

when the working frequency of the laser radar is a first modulation frequency, acquiring a first ranging value of each pixel point in the echo image;

when the working frequency of the laser radar is a second modulation frequency, obtaining a second distance measurement value of each pixel point in the echo image;

the first distance measurement value and the second distance measurement value of each pixel point are subjected to difference to obtain a distance measurement difference value;

and when the distance measurement difference value is not zero, the pixel point with the distance measurement difference value not being zero is a crosstalk pixel point of the echo image.

7. The method according to any one of claims 1 to 6, wherein the correcting the crosstalk pixel point to obtain a corrected echo image comprises:

selecting a pixel point adjacent to the crosstalk pixel point as a standby pixel point; and

and copying the information of the standby pixel points into the crosstalk pixel points to obtain the corrected echo image.

8. The method according to any one of claims 1 to 6, wherein the correcting the crosstalk pixel point to obtain a corrected echo image comprises:

and deleting the information of the crosstalk pixel points to obtain a corrected echo image.

9. The method according to any one of claims 1 to 6, wherein the correcting the crosstalk pixel point to obtain a corrected echo image comprises:

dividing the crosstalk pixel point into a plurality of sub pixel points;

and copying the information of the pixel points adjacent to the sub-pixel points into the sub-pixel points to obtain the corrected echo image.

10. A lidar ranging device comprising:

the acquisition module is used for acquiring an echo image received by the laser radar;

the judging module is used for judging whether crosstalk pixel points exist in the echo image;

the correction module is used for correcting crosstalk pixel points when the crosstalk pixel points exist in the echo image to obtain a corrected echo image; and

and the calculation module is used for calculating to obtain object pose information according to the corrected echo image.

11. The apparatus according to any one of claims 10, wherein the determining module is further configured to obtain a first edge pixel point of the depth image; judging whether the first edge pixel point meets a preset crosstalk condition or not; and when the first edge pixel point meets a preset crosstalk condition, determining the first edge pixel point as a crosstalk pixel point of the echo image.

12. The apparatus according to claim 11, wherein the determining module is further configured to obtain a second edge pixel point of the grayscale image; searching a corresponding pixel point of the second edge pixel point in the depth image; and determining the corresponding pixel point as a first edge pixel point of the depth image.

13. The apparatus according to claim 11, wherein the determining module is further configured to obtain an edge distance measurement value and a gradient direction of the first edge pixel point; acquiring first average ranging values of the first edge pixel point and all surrounding pixel points on the same side of the gradient direction; acquiring second average ranging values of the first edge pixel point and all surrounding pixel points on the opposite side of the gradient direction; and when the absolute value of the difference value between the edge ranging value and the first average ranging value is greater than a preset distance threshold value and the absolute value of the difference value between the edge ranging value and the second average ranging value is greater than the preset distance threshold value, the first edge pixel point meets a preset crosstalk condition.

14. The apparatus of claim 13, wherein the surrounding pixels comprise pixels adjacent to the first edge pixel.

15. The apparatus according to claim 10, wherein the determining module is further configured to obtain a first distance measurement value of each pixel point in the echo image when the operating frequency of the lidar is a first modulation frequency; when the working frequency of the laser radar is a second modulation frequency, obtaining a second distance measurement value of each pixel point in the echo image; the first distance measurement value and the second distance measurement value of each pixel point are subjected to difference to obtain a distance measurement difference value; and when the distance measurement difference value is not zero, the pixel point with the distance measurement difference value not being zero is a crosstalk pixel point of the echo image.

16. The apparatus according to any one of claims 10 to 15, wherein the calibration module is further configured to select a pixel adjacent to the crosstalk pixel as a spare pixel; and copying the information of the standby pixel points into the crosstalk pixel points to obtain corrected echo images.

17. The apparatus according to any one of claims 10 to 15, wherein the correction module is further configured to delete information of the crosstalk pixel, so as to obtain a corrected echo image.

18. The apparatus according to any one of claims 10 to 15, wherein the correction module is further configured to divide the crosstalk pixel into a plurality of sub-pixels, and copy information of pixels adjacent to the sub-pixels into the sub-pixels to obtain the corrected echo image.

19. A computer device comprising a memory and one or more processors, the memory having stored therein computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to perform the steps of:

acquiring an echo image received by a laser radar;

judging whether crosstalk pixel points exist in the echo image or not;

when crosstalk pixel points exist in the echo image, the crosstalk pixel points are corrected to obtain a corrected echo image; and

and calculating to obtain object pose information according to the corrected echo image.

20. One or more non-transitory computer-readable storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:

acquiring an echo image received by a laser radar;

judging whether crosstalk pixel points exist in the echo image or not;

when crosstalk pixel points exist in the echo image, the crosstalk pixel points are corrected to obtain a corrected echo image; and calculating to obtain object pose information according to the corrected echo image.

Technical Field

The application relates to a laser radar ranging method, a laser radar ranging device, computer equipment and a storage medium.

Background

The laser radar can provide real-time and accurate three-dimensional scene information, has the inherent advantage of environmental perception, has a large ranging range and high precision, and is widely applied to the fields of security monitoring, surveying and mapping, traffic management, automatic driving and the like. The laser radar transmits a detection signal to the object to be detected, receives an echo signal reflected by the object to be detected, and calculates the distance of the object to be detected according to the phase difference between the echo signal and the detection signal.

The Flash laser radar belongs to all-solid-state scanning laser radar, has no moving part, and has good system stability and reliability. The receiving end of the device adopts a pixel array to receive echo signals reflected by an object, and the pixel array calculates the distance of the object to be detected according to the echo signals and detection signals. With the increase of the detection distance, a single pixel of the receiving end can receive echo signals reflected by different objects, so that the problem of crosstalk between pixels can be caused, and the ranging accuracy of the laser radar is further influenced.

Disclosure of Invention

According to various embodiments disclosed in the present application, a laser radar ranging method, a laser radar ranging device, a computer device, and a storage medium are provided, which can improve the accuracy of ranging of an object to be measured.

A laser radar ranging method comprises the following steps:

acquiring an echo image received by a laser radar;

judging whether crosstalk pixel points exist in the echo image or not;

when crosstalk pixel points exist in the echo image, the crosstalk pixel points are corrected to obtain a corrected echo image; and

and calculating to obtain object pose information according to the corrected echo image.

A laser radar ranging apparatus includes:

the acquisition module is used for acquiring an echo image received by the laser radar;

the judging module is used for judging whether crosstalk pixel points exist in the echo image;

the correction module is used for correcting crosstalk pixel points when the crosstalk pixel points exist in the echo image to obtain a corrected echo image; and

and the calculation module is used for calculating to obtain object pose information according to the corrected echo image.

A computer device comprising a memory and one or more processors, the memory having stored therein computer-readable instructions that, when executed by the processors, cause the one or more processors to perform the steps of:

acquiring an echo image received by a laser radar;

judging whether crosstalk pixel points exist in the echo image or not;

when crosstalk pixel points exist in the echo image, the crosstalk pixel points are corrected to obtain a corrected echo image; and

and calculating to obtain object pose information according to the corrected echo image.

One or more non-transitory computer-readable storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:

acquiring an echo image received by a laser radar;

judging whether crosstalk pixel points exist in the echo image or not;

when crosstalk pixel points exist in the echo image, the crosstalk pixel points are corrected to obtain a corrected echo image; and

and calculating to obtain object pose information according to the corrected echo image.

The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below. Other features and advantages of the application will be apparent from the description and drawings, and from the claims.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

FIG. 1 is a diagram of an environment in which a lidar ranging method may be used in accordance with one or more embodiments.

FIG. 2 is a schematic diagram illustrating a crosstalk phenomenon between pixels according to one or more embodiments.

FIG. 3 is a flow diagram of a lidar ranging method in accordance with one or more embodiments.

Fig. 4 is a schematic flow chart illustrating a step of determining whether crosstalk pixel points exist in an echo image according to one or more embodiments.

Fig. 5 is a flowchart illustrating a step of obtaining first edge pixel points of a depth image according to one or more embodiments.

Fig. 6 is a flowchart illustrating a step of determining whether a first edge pixel meets a predetermined crosstalk condition according to one or more embodiments.

Fig. 7 is a block diagram of a depth image of a 3 × 3 pixel array formed by a first edge pixel and all its neighboring pixels according to one or more embodiments.

Fig. 8 is a block diagram of a depth image of a 3 × 3 pixel array formed by a first edge pixel and all its neighboring pixels in another embodiment.

Fig. 9 is a schematic flowchart of a step of determining whether crosstalk pixel exists in an echo image in another embodiment.

FIG. 10 is a schematic diagram of a detection signal emitted by a lidar in accordance with one or more embodiments.

Fig. 11 is a schematic diagram illustrating the operation of a lidar at multiple modulation frequency switching in accordance with one or more embodiments.

Fig. 12 is a block diagram of a corrected echo image obtained after dividing a crosstalk pixel into a plurality of sub-pixels according to one or more embodiments.

Fig. 13 is a block diagram of a corrected echo image obtained after dividing a crosstalk pixel into a plurality of sub-pixels in another embodiment.

Fig. 14 is a schematic diagram of dividing an echo image according to a detection accuracy requirement from high to low to obtain divided image regions according to one or more embodiments.

FIG. 15 is a block diagram of a lidar ranging device in accordance with one or more embodiments.

FIG. 16 is a block diagram of a computer device in accordance with one or more embodiments.

Detailed Description

In order to make the technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The laser radar ranging method provided by the application can be applied to the application environment shown in fig. 1. The laser radar 102 transmits a detection signal to the object to be detected, and receives an echo signal reflected by the object to be detected through a pixel array of a receiving end. The laser radar 102 processes the echo signal to obtain an echo image. Lidar 102 transmits the echo image to computer device 104. The computer device 104 determines whether the echo image has crosstalk pixel points. When crosstalk pixel points exist in the echo image, the computer device 104 corrects the crosstalk pixel points to obtain a corrected echo image. The computer device 104 calculates object pose information from the corrected echo image.

For flash lidar, the angular resolution depends on the field angle of the receiving lens and the number of pixels of the pixel array at the receiving end. Illustratively, the field angle of the receiving lens of the flash lidar is 60 ° × 45 °, the number of pixel points of the pixel array is 320 × 240, and the calculated angular resolutions of both the horizontal direction and the vertical direction are 0.1875 °. Accordingly, the view field corresponding to a single pixel point at the distance R can be estimated to be a square, and the area S of the view field is as follows:

S＝4R²tan²(0.1875°/2) (1)

such as R₁10m, then S₁＝10.71cm²(ii) a If R is₂50m, then S₂＝267.75cm². It can be seen that the field of view S corresponding to a single pixel point of the flash lidar increases with increasing distance R. Therefore, in an object at a longer distance, the field area corresponding to a single pixel point is larger, and the field corresponding to the single pixel point has a larger probability of falling at the edges of two objects at different distances. As shown in fig. 2, a pixel point 2 in the figure receives echo signals reflected by two objects (e.g., an object a and an object B in the figure), which is called "crosstalk between pixels", and the pixel point 2 is a crosstalk pixel point. At this time, if the distance measurement value of the crosstalk pixel point is calculated according to the previous distance calculation method, the distance measurement value Rx is a function of the actual distance between the object a and the object B, that is, Rx ═ f (R)_A，R_R) An erroneous calculation result will be obtained. This may cause a series of pixels with wrong distance to appear at the boundary between two (or more) objects in the depth image obtained after the flash lidar detects, and in the lidar point cloud, a comet-like point cloud trailing phenomenon may appear at the boundary between the objects.

The accuracy of the flash laser radar in distance measurement of the object boundary is influenced by crosstalk among pixels, the error point rate of a depth image is increased, the point cloud quality is damaged, and the application of the flash laser radar in the fields of automatic driving, object recognition, environment modeling and the like is limited to a great extent. The laser radar ranging method is used for solving the problem of crosstalk between pixels.

The depth camera also has the problem of crosstalk between pixels. The distance measurement method provided by the application can also be used for solving the problem of crosstalk between pixels of the depth camera.

In one embodiment, as shown in fig. 3, a lidar ranging method is provided, which is described by taking the method as an example applied to the computer device in fig. 1, and includes the following steps:

and step 302, acquiring an echo image received by the laser radar.

The laser radar can be flash laser radar, and the transmission array transmission detection signal of laser radar transmitting terminal illuminates the field of view, returns echo signal after the detection signal is by the internal object reflection of field of view, and the pixel array of receiving terminal receives echo signal, and laser radar processes echo signal and obtains the echo image, sends echo image to computer equipment. Each pixel point in a receiving end of the laser radar corresponds to a view field in an angle range, and echo signals reflected by the same object in the view field can be received by the corresponding pixel points. The echo image can be an image with gray scale information and space information of the whole field of view acquired by the laser radar in the detection process, and comprises a gray scale image and a depth image.

And step 304, judging whether crosstalk pixel points exist in the echo image.

There are various methods for determining whether crosstalk pixel exists.

One of them is to obtain a first edge pixel point of the depth image. And judging whether the first edge pixel point meets a preset crosstalk condition or not. And if the preset crosstalk condition is met, determining the first edge pixel point as a crosstalk pixel point.

Alternatively, the operating frequencies of the lidar may be different. And comparing the distance measurement values of all the pixel points under different working frequencies. And if the distance measurement values are different, determining the pixel points with different distance measurement values as crosstalk pixel points.

The method for judging the crosstalk pixel points is not limited, and the computer equipment can adopt one method for judging the crosstalk pixel points independently or adopt multiple methods for judging the crosstalk pixel points simultaneously.

And step 306, correcting the crosstalk pixel points to obtain a corrected echo image.

If it is determined in step 304 that there is a crosstalk pixel, the echo signals received by the crosstalk pixel include echo signals reflected by at least two objects. The distance measurement value obtained by resolving according to the echo signal received by the crosstalk pixel point is wrong, and the distance information of the detected object cannot be truly reflected. Therefore, it is necessary to correct the pixel crosstalk point determined to have the crosstalk problem so as to avoid obtaining erroneous detection information.

There are various methods for correcting crosstalk pixel points. One of them is to delete the information of the crosstalk pixel point. One of the alternatives is to copy the information of the spare pixel point adjacent to the pixel crosstalk point into the pixel crosstalk point. One of the optional methods is to divide the pixel crosstalk point into a plurality of sub-pixel points, and copy the information of the pixel points adjacent to the sub-pixel points into the sub-pixel points.

And after the crosstalk pixel points are corrected, the corrected echo image is obtained, so that the accuracy of the echo image is improved.

And 308, calculating to obtain object pose information according to the corrected echo image.

The corrected echo image corrects or partially corrects the error information of the crosstalk pixel points, and the accuracy of the echo image is improved. The echo image with high accuracy is used as a calculation data source, and the calculated object pose information is more accurate.

There are various methods for obtaining object pose information through calculation. One of them is, continuous wave type incoherent detection, in which a detection signal is emitted after being modulated by a carrier, and distance information is obtained by resolving a phase difference between an echo signal and the detection signal. An alternative is pulsed incoherent detection. The ITOF type is pulse integration ranging, which periodically and continuously transmits a detection signal with a wide pulse width, acquires echo signals in different integration time windows, and solves the flight time through a proportional relationship to obtain distance information. The DTOF type periodically and continuously transmits a detection signal with a narrow pulse width and a large peak power, and detects an echo signal to obtain the flight time, thereby calculating and obtaining distance information. One of them is coherent detection, where the echo signal and the local reference signal meet the condition of wavefront matching, beat frequency or coherence is superimposed at the receiving end and received by the receiving end, and the distance information is obtained by resolving according to the received coherent signal.

Step 310, the echo image is kept unchanged.

If it is determined in step 304 that there is no crosstalk pixel, the echo signals received by the pixels at the receiving end are all accurate and do not need to be further corrected.

And step 312, calculating to obtain object pose information according to the echo image.

And calculating the obtained object pose information through the accurate echo signals. Referring to step 308, a specific method for calculating pose information of an object may be described.

In this embodiment, the computer device obtains the echo image received by the laser radar, then determines whether the echo image has crosstalk pixel points, determines and identifies crosstalk pixel points having crosstalk between pixels in the pixel points, where the crosstalk pixel points receive echo signals reflected by different objects, and the echo signals received by the crosstalk pixel points cannot accurately correspond to distance information of the object to be measured, which greatly affects the accuracy of ranging of the laser radar. And next, when crosstalk pixel points exist in the echo image, the crosstalk pixel points are corrected, the pixel crosstalk points are corrected according to the judgment result, error information of the crosstalk pixel points is corrected or partially corrected, the influence of pixel crosstalk on the accuracy of the echo image is effectively reduced, and the accurate echo image is obtained. And finally, calculating to obtain object pose information according to the corrected echo image, wherein the information in the echo image for calculation is accurate because crosstalk pixel points do not exist in the echo image, the accuracy of the object pose information obtained by resolving is improved, and the accuracy and the reliability of laser radar detection are improved.

There are two methods for determining whether crosstalk pixel points exist in the echo image.

One way, in one embodiment, the echo image includes a depth image, as shown in fig. 4, step 304: the step of judging whether the echo image has crosstalk pixel points specifically comprises the following steps:

step 402, obtaining a first edge pixel point of the depth image.

There are various methods for obtaining edge pixel points of a depth image. One of the optional methods is to extract the edge features of the object by using the gray image, and obtain a second edge pixel point of the object. And mapping second edge pixel points of the gray-scale image into the depth image according to the corresponding relation between the gray-scale image and the depth image, wherein the corresponding pixel points in the depth image are the first edge pixel points. One of the alternatives is to directly adopt an edge detection operator to the depth image to obtain a first edge pixel point of the depth image. The depth image is not influenced by the irradiation direction of the detection signal of the transmitting end and the surface reflection characteristic of the object to be detected, and the shadow does not exist, so that the three-dimensional information of the object can be more accurately expressed. The detection to edge pixel is also more accurate, and pixel crosstalk often takes place in the edge of two at least objects of crisscross different distances, and it is high to detect the probability that pixel crosstalk in the first edge pixel that obtains to directly obtain first edge pixel from the degree of depth image, it is direct simple, reduce the operation degree of difficulty, improve the arithmetic speed.

The echo image comprises pixel points for normally receiving echo signals and pixel points for generating crosstalk, and the pixel crosstalk often occurs at the edges of a plurality of objects, so that the edge pixel points are judged, processing steps can be simplified, the calculation amount is reduced, and the calculation speed is improved.

Step 404, determining whether the first edge pixel point meets a preset crosstalk condition.

And judging pixel crosstalk according to the first edge pixel point obtained in the last step. Because the first edge pixel points in the depth image do not all have the problem of pixel crosstalk, when the first edge pixel points only detect the edge of one object, the problem of pixel crosstalk does not exist. As shown in fig. 2, pixel point 1 detects the edge of object a, but there is no pixel crosstalk at this point. The acquired first edge pixel points need to be judged one by one to determine whether the first edge pixel points are pixel crosstalk points or not.

Specifically, the step of determining whether the first edge pixel point meets the preset crosstalk condition further includes: acquiring an edge distance measurement value and a gradient direction of a first edge pixel point; acquiring first average ranging values of the first edge pixel point and all surrounding pixel points on the same side of the gradient direction; acquiring second average ranging values of the first edge pixel point and all surrounding pixel points on the opposite side of the gradient direction; and when the absolute value of the difference value between the edge ranging value and the first average ranging value is greater than the preset distance threshold value and the absolute value of the difference value between the edge ranging value and the second average ranging value is greater than the preset distance threshold value, the first edge pixel point meets the preset crosstalk condition.

Wherein, the surrounding pixel points comprise pixel points adjacent to the first edge pixel point. The first distance measurement average value of the surrounding pixel points on the same side in the gradient direction is similar to the edge distance measurement value of the first edge pixel point in value, which indicates that the edge distance measurement value has no larger deviation compared with the correct first distance measurement average value, and the first edge pixel point and the surrounding pixel points on the same side in the gradient direction detect the same object. Therefore, pixel crosstalk does not occur in the first edge pixel point, and the edge distance measurement value is correct and reliable.

Similarly, the second distance measurement average value of the peripheral pixel points on the side opposite to the gradient direction is similar to the edge distance measurement value of the first edge pixel point in value, so that the first edge pixel point and the peripheral pixel points on the side opposite to the gradient direction detect the same object, and the first edge pixel point does not generate pixel crosstalk.

When the edge distance measurement value is different from both the first distance measurement average value and the second distance measurement average value greatly, for example, is greater than a preset distance threshold, the first edge pixel point is a crosstalk pixel point.

And 406, when the first edge pixel point meets a preset crosstalk condition, determining the first edge pixel point as a crosstalk pixel point of the echo image.

And if the difference between the edge distance measurement value of the first edge pixel point and the first distance measurement average value and the second distance measurement average value is larger than a preset distance threshold value, the first edge pixel point is a crosstalk pixel point.

And step 408, when the first edge pixel does not meet the preset crosstalk condition, the first edge pixel is not a crosstalk pixel of the echo image.

In this embodiment, the computer device obtains the first edge pixel points of the depth image, determines whether the first edge pixel points satisfy the preset crosstalk condition one by one, and determines that the first edge pixel points are crosstalk pixel points if the preset crosstalk condition is satisfied. Because the depth image is not influenced by the irradiation direction of the detection signal of the transmitting end and the surface reflection characteristic of the object to be detected, and shadow does not exist, the three-dimensional information of the object can be more accurately expressed; the detection of the edge pixel points is more accurate, pixel crosstalk often occurs at the edges of at least two staggered objects with different distances, the probability of detecting the pixel crosstalk in the first edge pixel point is high, and the processing efficiency of computer equipment is improved; and the first edge pixel point is directly obtained from the depth image, so that the method is direct and simple, the operation difficulty is reduced, and the operation speed is improved.

In one embodiment, as shown in FIG. 5, step 402: acquiring a first edge pixel point of a depth image, acquiring a second edge pixel point from a gray image according to the corresponding relation between the gray image and the depth image, mapping to the depth image to find a corresponding pixel point, wherein the step specifically comprises the following steps:

step 502, obtaining a second edge pixel point of the gray image.

The computer device identifies pixels in the grayscale image where the brightness changes significantly, and significant changes in the image attributes typically reflect significant events and changes in attributes, which can include (a) discontinuities in depth, (b) discontinuities in surface orientation, (c) changes in material attributes, and (d) changes in scene illumination.

The operators for edge detection may include Sobel operators, Laplacian operators, Canny operators, and the like. Because the edge characteristics of the depth image and the gray image have consistency, the computer equipment can search corresponding pixel points in the depth image of the echo image according to the second edge pixel points, and therefore the corresponding pixel points are used as first edge pixel points of the depth image.

Illustratively, the edge detection is performed by using a Sobel operator. The Sobel operator comprises two sets of 3 × 3 matrices T₁And T₂Corresponding to the lateral and longitudinal directions of the gray-scale image, respectively. And performing plane convolution on the Sobel operator and the gray level image so as to obtain the horizontal and longitudinal brightness difference approximate values in the gray level image. The two sets of matrices for the Sobel operator are:

if A represents the initial gray image, G_XAnd G_YGray values, G, representing the transverse and longitudinal images, respectively_XAnd G_YThe calculation formula of (a) is as follows:

the calculation formula of the gradient amplitude of a single pixel point in the gray image can be as follows:

the computer equipment compares the gradient amplitude of each pixel point in the gray image with a preset gradient threshold; and when the gradient amplitude of one pixel point is larger than the preset gradient threshold, the pixel point is the second edge pixel point. P for the second edge pixel₀(X₀，Y₀) To express, the gradient direction of the second edge pixel point is:

the gray level image comprises depth information and surface information, and contains more discontinuous information, and the condition of missing detection of edge pixel points is less. Meanwhile, an operator for edge detection of the gray level image is verified, and the reliability of an operation result is good.

Step 504, finding a corresponding pixel point of the second edge pixel point in the depth image.

The echo image obtained after the detection of the laser radar comprises a gray level image and a depth image, so that pixel points of the gray level image and pixel points of the depth image have a corresponding relation. There are various methods for obtaining the correspondence between the grayscale image and the depth image.

One of the optional methods is to extract feature points of objects in the grayscale image and the depth image, such as a straight line or a corner of a road edge, an end point of a street lamp post, and the like, and align the feature points of the corresponding objects in the grayscale image and the depth image to obtain mapping between the grayscale image and the depth image. The method for determining the corresponding relation only needs to match the characteristic points, and has small operand and high operation speed.

One of the optional methods is to extract features of an object in the grayscale image and the depth image, such as a frame of the object, and align all pixel points in the frame of the corresponding object in the grayscale image and the depth image to obtain mapping between the grayscale image and the depth image. The method for determining the corresponding relation matches all pixel points in the object frame, and the operation accuracy is high.

One of the selectable methods is that the grayscale image and the depth image are both obtained by the same laser radar, so that each pixel point of the grayscale image and the depth image contains time information, and the pixel points with the same time information are aligned to obtain mapping of the grayscale image and the depth image. The method for determining the corresponding relation fully utilizes the time corresponding relation between the images obtained by the laser radar, utilizes the existing data, simplifies the operation process and has high operation accuracy.

And mapping a second edge pixel point in the gray level image to the depth image according to the obtained mapping of the depth image and the gray level image to obtain a corresponding pixel point.

Step 506, the corresponding pixel point is determined as a first edge pixel point of the depth image.

Because the gray image and the depth image are obtained by detecting the same field of view, the contents of the gray image and the depth image have consistency. Therefore, the corresponding pixel point obtained by mapping the second edge pixel point can be determined as the first edge pixel point of the depth image.

In this embodiment, the computer device extracts the edge feature of the object by using the grayscale image, obtains a second edge pixel point of the object, and maps the second edge pixel point of the grayscale image into the depth image according to the correspondence between the grayscale image and the depth image, where a corresponding pixel point in the depth image is the first edge pixel point. Because the gray level image comprises depth information and surface information, such as image attribute information, object depth information, object surface information, scene illumination information and the like, the contained discontinuous information is more comprehensive, and the omission factor of the edge pixel points can be reduced. Moreover, the operator for edge detection of the gray level image is verified, the reliability of the operation result is good, and the accuracy of edge pixel detection is improved.

In one embodiment, as shown in FIG. 6, step 404: the step of judging whether the first edge pixel point meets the preset crosstalk condition specifically comprises the following steps:

step 602, an edge distance measurement value and a gradient direction of the first edge pixel point are obtained.

It can be known from the foregoing embodiment that the gradient direction of the second edge pixel point can be obtained through calculation by the formula (5), the second edge pixel point and the first edge pixel point have a corresponding relationship, and the gradient direction is the gradient direction of the first edge pixel point.

Or, an edge detection operator can be directly adopted for the depth image to obtain the depth difference in the depth image, so that the gradient direction of the first edge pixel point in the depth image is obtained. The specific method can refer to the gray level image to obtain the gradient direction of the second edge pixel point.

The distance measurement value is one of the object pose information, and thus the method for obtaining the edge distance measurement value of the first edge pixel point may refer to a plurality of methods for obtaining the object pose information through calculation in the foregoing embodiments, which are not described herein again.

Step 604, obtain the first average ranging values of the first edge pixel point and all surrounding pixel points on the same side of the gradient direction.

Step 606, obtain the second average ranging values of the first edge pixel point and all surrounding pixel points on the opposite side of the gradient direction.

And setting a dividing line passing through the first edge pixel point in a direction perpendicular to the gradient direction of the first edge pixel point, respectively obtaining average distance measurement values of surrounding pixel points at two sides of the dividing line, and comparing the average distance measurement values with the edge distance measurement values of the first edge pixel point. And the average ranging value of all surrounding pixel points on the same side with the gradient direction is the first average ranging value. According to the methods for calculating the object pose information in the embodiments, the distance measurement value of each surrounding pixel point is obtained, and the distance measurement values of all surrounding pixel points are averaged to obtain the first average distance measurement value. The surrounding pixels include pixels adjacent to the first edge pixel. Similarly, the average distance measurement value of all surrounding pixel points on the side opposite to the gradient direction is the second average distance measurement value, and the calculation method may be the same as the first average distance measurement value.

In step 608, when the absolute value of the difference between the edge ranging value and the first average ranging value is greater than the preset distance threshold and the absolute value of the difference between the edge ranging value and the second average ranging value is greater than the preset distance threshold, the first edge pixel satisfies the preset crosstalk condition.

The first distance measurement average value of the surrounding pixel points on the same side in the gradient direction is similar to the edge distance measurement value of the first edge pixel point in value, which indicates that the edge distance measurement value has no larger deviation compared with the correct first distance measurement average value, and the first edge pixel point and the surrounding pixel points on the same side in the gradient direction detect the same object. Therefore, pixel crosstalk does not occur in the first edge pixel point, and the edge distance measurement value is correct and reliable.

Similarly, the second distance measurement average value of the peripheral pixel points on the side opposite to the gradient direction is similar to the edge distance measurement value of the first edge pixel point in value, so that the first edge pixel point and the peripheral pixel points on the side opposite to the gradient direction detect the same object, and the first edge pixel point does not generate pixel crosstalk.

The preset distance threshold is determined, the preset distance threshold is related to the ranging precision of the laser radar, and the higher the ranging precision is, the smaller the preset distance threshold is; and also relates to the ranging distance of the laser radar, wherein the longer the ranging distance is, the larger the preset distance threshold value is.

Because the surrounding pixel points are the pixel points adjacent to the first edge pixel point; if the first edge pixel point has no pixel crosstalk, the surrounding pixel points on the same side and/or the opposite side of the gradient direction of the first edge pixel point are the same object as the first edge pixel point, and the obtained distance measurement value should be equal. And taking errors and loss into consideration, averaging the distance measurement values of surrounding pixel points on the same side and/or the opposite side of the gradient direction of the first edge pixel point. When the edge distance measurement value is different from both the first distance measurement average value and the second distance measurement average value, for example, is greater than a preset distance threshold value, it is indicated that the first edge pixel point is different from the objects detected by the surrounding pixel points on the same side and the opposite side of the gradient direction, and the first edge pixel point is a crosstalk pixel point.

And the division line perpendicular to the gradient direction of the first edge pixel point passes through other pixel points besides the first edge pixel point. The information continuity of the depth images on the two sides of the dividing line changes, so that the pixel points on the dividing line are changed edge pixel points, and whether the pixel points on the dividing line meet the preset crosstalk condition can be further judged, so that the detection rate of crosstalk pixel points is improved, the omission factor is reduced, and the detection accuracy of the laser radar is improved; the specific judgment method can be referred to the above method. If the first edge pixel point meets the preset crosstalk condition, the probability that the pixel point on the partition line also meets the preset crosstalk condition is high.

For example, a depth image of a 3 × 3 pixel array formed by the first edge pixel and all the adjacent pixels thereof is taken as an example for description. As shown in fig. 7, the point P is a first edge pixel, and the direction of the arrow on the point P is the gradient direction of the first edge pixel. All surrounding pixel points of the P point are sequentially marked as 1, 2, 3, 4, 6, 7, 8 and 9.

And a dividing line passing through the point P is arranged in the direction perpendicular to the gradient direction of the point P, the dividing line passes through the point 3, the point P and the point 7, and the information continuity of the depth images on the two sides of the dividing line is changed. The peripheral pixel points in the same direction as the gradient direction of the P point are 1, 2 and 4, and the peripheral pixel points in the opposite direction to the gradient direction of the P point are 6, 8 and 9. The computer equipment respectively calculates and obtains the distance measurement values of the pixel points 1, 2 and 4 as R₁、R₂And R₄(ii) a Calculating the average value of the distance measuring values of the pixel points 1, 2 and 4 to obtain a first average distance measuring value (R)_i+R₂+R₄)/3. The computer equipment respectively calculates and obtains the distance measurement values of the pixel points 6, 8 and 9 as R₆、R₈And R₉Calculating the average value of the distance measuring values of the pixel points 6, 8 and 9 to obtain a second average distance measuring value (R)₆+R₈+R₉)/3. Meanwhile, the computer device calculates the distance measurement value of the obtained P point to be R_P。

To obtain a difference D between the P-point range value and the first average range value₁And taking the difference between the P point distance measurement value and the first average distance measurement value and taking the absolute value, wherein the calculation formula is as follows:

D₁＝|R_P-(R₁+R₂+R₄)/3| (6)

in the same way, P-point measurementDifference D between the distance value and the second average distance measurement value₂The calculation formula is:

D₂＝|R_P-(R₆+R₈+R₉)/3| (7)

when D is present₁When the distance is less than e (a preset distance threshold value), the point P and the pixel points 1, 2 and 4 are considered to be on the surface of the same object, and pixel crosstalk does not exist; when D is present₂When the pixel values are less than e, the P point and the pixel points 6, 8 and 9 are considered to be on the surface of the same object, and pixel crosstalk does not exist; when D is present₁E and D₂When the current value is more than e, the P point meets the preset crosstalk condition, the P point is considered not to be on the same object surface with the pixel points 1, 2 and 4, but not on the same object surface with the pixel points 6, 8 and 9, and the pixel crosstalk exists in the P point.

Similarly, the dividing line also passes through the pixel points 3 and 7, and when the point P meets the preset crosstalk condition, the pixel points 3 and 7 have a larger possibility of also having the problem of pixel crosstalk. Therefore, with the above method, it is determined whether the pixel points 3 and 7 satisfy the predetermined crosstalk condition, respectively.

For example, a depth image of a 3 × 3 pixel array formed by the first edge pixel and all the adjacent pixels thereof is taken as an example for description. As shown in fig. 8, the point P is a first edge pixel, and the direction of the arrow on the point P is the gradient direction of the first edge pixel. All surrounding pixel points of the P point are sequentially marked as 1, 2, 3, 4, 6, 7, 8 and 9.

And a dividing line passing through the point P is arranged in the direction perpendicular to the gradient direction of the point P, the dividing line passes through the point 2, the point P and the point 8, and the information continuity of the depth images on the two sides of the dividing line is changed. The peripheral pixel points in the same direction as the gradient direction of the P point are 1, 4 and 7, and the peripheral pixel points in the opposite direction to the gradient direction of the P point are 3, 6 and 9. The computer equipment respectively calculates and obtains the distance measurement values of the pixel points 1, 4 and 7 as R₁、R₄And R₇(ii) a Calculating the average value of the distance measuring values of the pixel points 1, 2 and 4 to obtain a first average distance measuring value (R)₁+R₄+R₇)/3. The computer equipment respectively calculates and obtains the distance measurement values of the pixel points 3, 6 and 9 as R₃、R₆And R₉Calculating the average value of the distance measuring values of the pixel points 3, 6 and 9 to obtain a second average distance measuring value (R)₃+R₆+R₉)/3. Meanwhile, the computer device calculates the distance measurement value of the obtained P point to be R_P。

Difference D between P point ranging value and first average ranging value_I＝|R_P-(R₁+R₄+R₇) The difference D between the P point distance measurement value and the second average distance measurement value₂＝|R_P-(R₃+R₆+R₉)/3|。

When D is present₁When the distance is less than e (a preset distance threshold value), the point P and the pixel points 1, 4 and 7 are considered to be on the surface of the same object, and pixel crosstalk does not exist; when D is present₂When the pixel values are less than e, the P point and the pixel points 3, 6 and 9 are considered to be on the surface of the same object, and pixel crosstalk does not exist; when D is present₁E and D₂When the current value is more than e, the P point meets the preset crosstalk condition, the P point is considered not to be on the same object surface with the pixel points 1, 4 and 7, but not on the same object surface with the pixel points 3, 6 and 9, and the pixel crosstalk exists in the P point.

In this embodiment, the computer device sets a dividing line passing through the P point in a direction perpendicular to the gradient direction of the P point by obtaining the edge distance measurement value and the gradient direction of the first edge pixel point, information continuity of the depth image on both sides of the dividing line changes, and the pixel point passing through the dividing line is more likely to be located at a junction of two objects, which is more likely to cause pixel crosstalk. The computer equipment acquires the first average distance measurement value of all surrounding pixel points on the same side of the gradient direction of the first edge pixel point and the second average distance measurement value of all surrounding pixel points on the opposite side of the gradient direction, and then acquires the difference value between the edge distance measurement value and the first average distance measurement value and the difference value between the edge distance measurement value and the second average distance measurement value. Because the surrounding pixel points are the pixel points adjacent to the first edge pixel point; if the first edge pixel point has no pixel crosstalk, the surrounding pixel points on the same side and/or the opposite side of the gradient direction of the first edge pixel point are the same object as the first edge pixel point, and the obtained distance measurement value should be similar. When the edge distance measurement value is different from both the first distance measurement average value and the second distance measurement average value, for example, is greater than a preset distance threshold value, it is indicated that the first edge pixel point is different from the objects detected by the surrounding pixel points on the same side and the opposite side of the gradient direction, and the first edge pixel point is a crosstalk pixel point. By the method, the first edge pixel points which possibly have pixel crosstalk are judged one by one, so that the identification accuracy of the crosstalk pixel points is improved; and the judgment precision can be adjusted by adjusting the preset distance threshold value so as to meet the application of laser radars with different system precisions and ranging distances, and the universality is good.

Another method for determining whether crosstalk pixel points exist in the echo image is that, in one embodiment, the working frequency of the laser radar includes at least two different modulation frequencies, as shown in fig. 9, in step 204: the step of judging whether the echo image has crosstalk pixel points specifically comprises the following steps:

step 902, when the working frequency of the laser radar is the first modulation frequency, obtaining a first ranging value of each pixel point in the echo image.

And 904, when the working frequency of the laser radar is the second modulation frequency, obtaining a second distance measurement value of each pixel point in the echo image.

Step 906, the first distance measurement value and the second distance measurement value of each pixel point are subtracted to obtain a distance measurement difference value.

Step 908, when the distance measurement difference is not zero, the pixel point with the distance measurement difference not zero is a crosstalk pixel point of the echo image.

In step 910, when the distance measurement difference is zero, the pixel point with the distance measurement difference being zero is not a crosstalk pixel point of the echo image.

As mentioned above, there are various methods for obtaining object pose information through calculation. When continuous wave type incoherent detection is adopted, a detection signal is emitted after being modulated by a carrier wave, and distance information is obtained by resolving a phase difference between an echo signal and the detection signal. The calculation method can be as follows:

as shown in fig. 10, the detection signal emitted by the lidar is represented as:

s(t)＝a₁+a₂ cos(2πft) (8)

the echo signal received by the lidar is represented as:

r(t)＝Acos(2πft-2πfτ)+B (9)

wherein, a₁Indicating the offset of the probe signal, a₂Represents the modulation amplitude of the detection signal, and f represents the modulation frequency of the detection signal; a denotes the amplitude of the echo signal, B denotes the offset of the echo signal due to background illumination, which may be illumination outside the transmitter itself, τ denotes the time of flight, i.e. the time difference between the probe signal and the echo signal, and Φ — 2 π f τ denotes the corresponding phase offset.

The cross-correlation function of the probe signal and echo signal power is expressed as:

let ψ 2 pi fx and Φ 2 pi f τ. Equation (10) can be expressed as:

the computer equipment can sample four points at equal intervals in one modulation period, and the correlation function value is psi₀＝0，ψ₁＝π/2，ψ₂＝π，ψ₃3 pi/2, the resulting correlation function is each C₀＝C(0，φ)，C₁＝C(π/2，φ)，C₂C (pi, phi) and C₃C (3 pi/2, phi). The echo signal offset B and the amplitude a are obtained by the four equations, and the calculation formula is as follows:

the calculation formula of the range value of the detected object is as follows:

where d represents the range value of the object and c represents the speed of light.

If pixel crosstalk does not occur at a pixel point, the pixel point only receives an echo signal reflected by an object, and the echo signal is a single echo condition at this time, the echo signal can be represented by formula (9), and a calculation formula of a ranging value is obtained according to formulas (10) to (15), as follows:

where c represents the speed of light and τ represents the time of flight.

In the single-echo case, the range value d is dependent only on the time of flight τ, and is independent of the modulation frequency of the echo signal.

If pixel crosstalk occurs in a pixel point, the pixel point receives echo signals reflected by at least two objects, at this time, a double echo or multi-echo situation is shown, taking the two echo signals received by the pixel point as an example, and formula (9) is modified as follows:

r(t)＝A₁ cos(2πft-2πfτ₁)+B₁+A₂ cos(2πft-2πfτ₂)+B₂ (17)

wherein A is₁Representing the amplitude, τ, of the first echo signal₁Representing the time difference between the first echo signal and the probe signal, B₁Indicating the offset of the first echo signal, A₂Representing the amplitude, tau, of the second echo signal₂Representing the time difference between the second echo signal and the probe signal, B₂Indicating the offset of the second echo signal.

The calculation formula of the ranging value is calculated according to the formulas (10) to (15), as follows:

at this time, the ranging value is related to the modulation frequency.

Two modulation frequencies are arbitrarily selected from the modulation frequency band of the laser radar, namely a first modulation frequency f1 and a second modulation frequency f 2. And when the working frequency of the laser radar is the first modulation frequency, acquiring a first ranging value of each pixel point in the echo image. And when the working frequency of the laser radar is the second modulation frequency, acquiring a second distance measurement value of each pixel point in the echo image. When the laser radar works, the two modulation frequencies are alternately switched, and after the first modulation frequency f1 works for a plurality of cycles, the first modulation frequency f2 is switched to the second modulation frequency f2 and works for a plurality of cycles.

Illustratively, as shown in fig. 11, the modulation frequency f1 is a first modulation frequency, and the modulation frequency f2 is a second modulation frequency, which is a second modulation frequency, for 3 periods. When the probe signal is modulated at the first modulation frequency f1, amplitudes of echo signals of four equally spaced sample points ψ of 0 °, ψ of 90 °, ψ of 180 °, ψ of 270 ° are obtained, which are C0, C1, C2, and C3, respectively. The first ranging value d1 is calculated according to equation (18) and C0-C3. During the operation of the first modulation frequency f1, the first distance measurement value of each pixel point can be calculated by the above method. When the probe signal is modulated at the second modulation frequency f2, the same four equally spaced sampling points ψ of 0 °, ψ of 90 °, ψ of 180 °, ψ of 270 ° are obtained, which are C0 ', C1', C2 ', and C3', respectively. And calculating a second ranging value d2 according to the formula (18) and C0 '-C3'. Similarly, during the operation period of the second modulation frequency f2, the second distance measurement value of each pixel point can be calculated by the above method.

As can be seen from the foregoing, in the case of a single echo, the ranging value d is related only to the flight time τ, and is independent of the modulation frequency of the echo signal; in the case of a double echo (or multiple echoes), the range value d is dependent on the modulation frequency. Therefore, different modulation frequencies are switched in the working process of the laser radar, and the pixel points with pixel crosstalk have different ranging values obtained by detecting the same object; the pixel points without pixel crosstalk have the same range value obtained by detecting the same object.

And subtracting the first distance measurement value and the second distance measurement value of each pixel point to obtain a distance measurement difference value. And taking any pixel point as an example for explanation, when the distance measurement difference value is zero, the first distance measurement value and the second distance measurement value are equal, the distance measurement values are not influenced by modulation frequency, and the pixel point is not a crosstalk pixel point in the echo image. When the distance measurement difference value is not zero, the first distance measurement value and the second distance measurement value are not equal, the distance measurement values change along with modulation frequency, and the pixel point is a crosstalk pixel point in the echo image. And judging the distance measurement difference values of all the pixel points one by one to obtain all crosstalk pixel points in the echo image.

In this embodiment, according to the resolving characteristic of continuous wave type incoherent detection, the laser radar includes at least two different modulation frequencies during operation, the computer device obtains a first ranging value of each pixel point during operation of the first modulation frequency and a second ranging value of each pixel point during operation of the second modulation frequency, and then the first ranging value and the second ranging value of each pixel point under different modulation frequencies are subtracted to obtain a ranging difference value. When pixel crosstalk occurs in a pixel point, echo signals reflected by at least two objects are received, namely a pixel point receives a plurality of echoes, and at the moment, a distance measurement value is related to modulation frequency; and judging whether the pixel points are crosstalk pixel points one by utilizing the characteristic. If the distance measurement difference value is zero, the first distance measurement value and the second distance measurement value are equal, the distance measurement values are not affected by modulation frequency, and the pixel point is not a crosstalk pixel point in the echo image; if the distance measurement difference value is not zero, the first distance measurement value and the second distance measurement value are not equal, the distance measurement values change along with modulation frequency, and the pixel point is a crosstalk pixel point in the echo image. Whether a pixel crosstalk point exists in an echo image is judged by the method, the modulation frequency of work can be switched as required by adjusting a transmitting end modulation circuit of the laser radar, a ranging value can be obtained during the normal work of the laser radar for judgment, and the detection process of the laser radar is not influenced; meanwhile, only the difference is needed to be made between the two ranging values of each pixel point, the calculation process and the judgment method are simple, the reliability of the calculation result is high, the calculation is simplified, the calculation rate is improved, and the calculation burden of the system is reduced.

There are various methods for correcting crosstalk pixel points to obtain a corrected echo image.

In one embodiment, step 206: correcting crosstalk pixel points, and obtaining a corrected echo image specifically comprises the following steps: selecting a pixel point adjacent to the crosstalk pixel point as a standby pixel point; and copying the information of the standby pixel points into the crosstalk pixel points to obtain the corrected echo image.

Specifically, the computer device selects a pixel point without pixel crosstalk as a standby pixel point from adjacent pixel points of the crosstalk pixel points; and copying the information of the standby pixel points into the crosstalk pixel points, and replacing the information of the crosstalk pixel points to obtain the corrected echo image. For example, as shown in fig. 7, if the P point is a crosstalk pixel point, since the pixel points 3 and 7 are both located on the dividing line perpendicular to the gradient direction of the P point, it is more likely that the pixel points are also crosstalk pixel points as described above; therefore, the pixels 1, 2 and 4 or the pixels 6, 8 and 9 adjacent to the point P can be selected as the standby pixels. If the pixel point 1 is selected as a standby pixel point, copying the information detected by the pixel point 1 into a P point, and replacing the original information of the P point.

In this embodiment, the computer device selects a pixel point adjacent to the crosstalk pixel point as a standby pixel point, copies information of the standby pixel point without pixel crosstalk into the crosstalk pixel point, and corrects the crosstalk pixel point; the method is adopted to correct all crosstalk pixel points to obtain a corrected echo image. Crosstalk pixel points are often positioned at the junction of two or more objects in an echo image, and error information in the crosstalk pixel points is replaced by information of standby pixel points adjacent to the crosstalk pixel points; because the spare pixel point does not have pixel crosstalk, and the positions of the crosstalk pixel point and the spare pixel point are adjacent, the object detected by the spare pixel point is one of a plurality of objects detected by the crosstalk pixel point. The adjacent non-pixel crosstalk pixel points are selected as the standby pixel points, namely, the boundary of a certain object detected by the standby pixel points is expanded by one pixel point, the boundaries of other objects return to one pixel point, the operation is simple and convenient, and the corrected echo image does not influence the normal detection of the object and the accuracy of the object detection. In addition, the correction method can reduce the error point rate of the echo image under the condition of not reducing the number of pixel points, thereby improving the calculation accuracy of the object pose information and the ranging accuracy of the laser radar.

In one embodiment, step 206: correcting crosstalk pixel points, and obtaining a corrected echo image specifically comprises the following steps:

and deleting the information of the crosstalk pixel points to obtain the corrected echo image.

After obtaining the crosstalk pixel points, the computer equipment directly deletes the information of the crosstalk pixel points, corrects the crosstalk pixel points, and corrects all the crosstalk pixel points by adopting the method to obtain corrected echo images. The influence of crosstalk pixel points on the accuracy of the echo image is effectively avoided, and the calculation accuracy of the object pose information is improved. Meanwhile, the information of the crosstalk pixel points is deleted, the method is simple and direct, other calculation operations are not needed, the burden of computer equipment is reduced, and the calculation rate is increased.

In one embodiment, step 206: correcting crosstalk pixel points, and obtaining a corrected echo image specifically comprises the following steps: dividing the crosstalk pixel point into a plurality of sub pixel points; and copying the information of the pixel points adjacent to the sub-pixel points into the sub-pixel points to obtain the corrected echo image.

Specifically, the computer device may partition the crosstalk pixel point into a plurality of sub-pixel points according to a preset partition manner; the preset segmentation mode can be that the pixel is segmented into NxN sub-pixels, and can also be that the pixel is segmented along the gradient direction and the segmentation line direction of the crosstalk pixel. Copying the information of the adjacent pixel points into the sub-pixel points when the adjacent pixel points of each sub-pixel point are not crosstalk pixel points; and splitting and replacing the information of the crosstalk pixel points to obtain a corrected echo image.

For example, as shown in fig. 7, if the point P is a crosstalk pixel point, the arrow direction is a gradient direction of the point P, and a dividing line is disposed perpendicular to the gradient direction through the point P. As shown in fig. 12, the crosstalk pixel is divided into 4 sub-pixels, P1, P2, P3, and P4, according to the gradient direction and the dividing line direction, that is, there are two sub-pixels on one side of the dividing line and two sub-pixels on the other side of the dividing line. The adjacent pixel of the sub-pixel P1 is the pixel 4, and the information of the pixel 4 is copied into the sub-pixel P1. By analogy, the information of the pixel 2 is copied into the sub-pixel P2, the information of the pixel 6 is copied into the sub-pixel P3, and the information of the pixel 8 is copied into the sub-pixel P4. As shown in fig. 13, the crosstalk pixel is divided into 4 sub-pixels of 2 × 2, which are P1, P2, P3, and P4, respectively. The information of the pixel 2 is copied into the sub-pixels P1 and P2, and the information of the pixel 8 is copied into the sub-pixels P3 and P4. Or, the information of the pixel 4 is copied into the sub-pixels P1 and P3, and the information of the pixel 6 is copied into the sub-pixels P2 and P4.

In this embodiment, the computer device further divides the crosstalk pixel into sub-pixels, copies information of adjacent pixels without pixel crosstalk into the sub-pixels, and corrects the crosstalk pixel; the method is adopted to correct all crosstalk pixel points to obtain a corrected echo image. As described above, the crosstalk pixel point is usually located at the boundary of a plurality of objects in the echo image, the pixel point without pixel crosstalk adjacent to the crosstalk pixel point is detected, and the detected object is one of the plurality of objects detected by the crosstalk pixel point. After the crosstalk pixel point is divided into a plurality of sub-pixel points, the information in each sub-pixel point is replaced by the information of the adjacent pixel point without pixel crosstalk, namely, the sub-pixel points are divided into pixel point sets corresponding to a plurality of objects according to the position relation, and the boundary between the sub-pixel points divided into different objects is the boundary of different objects. By means of disassembly, the crosstalk pixel point is divided into a plurality of sub-pixel points and then divided, and boundaries of different objects in the crosstalk pixel point are finer and more accurate; because the information content of each sub-pixel point is less, even if the information of the adjacent pixel points is copied, the influence on the whole echo image ranging result is not great.

The method for correcting the crosstalk pixel points to obtain the corrected echo image can be used independently or simultaneously. Illustratively, the echo image may be divided into regions. The detection accuracy of the central area of the echo image is higher than that of the peripheral area, the upper part of the echo image is mainly higher than the detection result of the laser radar installation position, and the resolution requirement is lower; therefore, the echo image can be divided into three regions from high to low according to the detection accuracy requirement, as shown in fig. 14, the three regions are region a, region B and region C, respectively, region a is the central region of the echo image, region C is the region above the central region, and the rest is region B. The crosstalk pixel points in the area A can be corrected by copying the information of the adjacent pixel points after being divided into sub pixel points; the pixel points in the area B can be corrected by copying the information of the standby pixel points; and the pixel points in the area C can be corrected by deleting the information of the crosstalk pixel points. By using multiple correction methods simultaneously and combining the correction methods, the accuracy of the echo image obtained after correction can be ensured, the calculation amount of the whole computer equipment can be reduced, the system pressure is reduced, and the calculation rate is improved.

It should be understood that although the various steps in the flowcharts of fig. 3-6 and fig. 9 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 3-6 and 9 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternating with other steps or at least some of the sub-steps or stages of other steps.

In one embodiment, as shown in fig. 15, there is provided a laser radar ranging apparatus including: an obtaining module 1502, a determining module 1504, a correcting module 1506, and a calculating module 1508, wherein:

an obtaining module 1502 is configured to obtain an echo image received by the laser radar.

The determining module 1504 is configured to determine whether a crosstalk pixel exists in the echo image.

The correcting module 1506 is configured to correct the crosstalk pixel point when the crosstalk pixel point exists in the echo image, so as to obtain a corrected echo image.

And a calculating module 1508, configured to calculate object pose information according to the corrected echo image.

In one embodiment, the determining module 1504 is further configured to obtain a first edge pixel point of the depth image; judging whether the first edge pixel point meets a preset crosstalk condition or not; and when the first edge pixel point meets the preset crosstalk condition, determining the first edge pixel point as a crosstalk pixel point of the echo image.

In one embodiment, the determining module 1504 is further configured to obtain a second edge pixel point of the grayscale image; searching a corresponding pixel point of the second edge pixel point in the depth image; and determining the corresponding pixel points as first edge pixel points of the depth image.

In one embodiment, the determining module 1504 is further configured to obtain an edge distance measurement value and a gradient direction of the first edge pixel; acquiring first average ranging values of the first edge pixel point and all surrounding pixel points on the same side of the gradient direction; acquiring second average ranging values of the first edge pixel point and all surrounding pixel points on the opposite side of the gradient direction; and when the absolute value of the difference value between the edge ranging value and the first average ranging value is greater than the preset distance threshold value and the absolute value of the difference value between the edge ranging value and the second average ranging value is greater than the preset distance threshold value, the first edge pixel point meets the preset crosstalk condition.

In one embodiment, the surrounding pixels include pixels adjacent to the first edge pixel.

In one embodiment, the determining module 1504 is further configured to obtain a first ranging value of each pixel point in the echo image when the working frequency of the laser radar is the first modulation frequency; when the working frequency of the laser radar is a second modulation frequency, obtaining a second distance measurement value of each pixel point in the echo image; the first distance measurement value and the second distance measurement value of each pixel point are subjected to difference to obtain a distance measurement difference value; and when the distance measurement difference value is not zero, the pixel point with the distance measurement difference value not zero is a crosstalk pixel point of the echo image.

In one embodiment, the calibration module 1506 is further configured to select a pixel adjacent to the crosstalk pixel as a spare pixel; and copying the information of the standby pixel points into the crosstalk pixel points to obtain the corrected echo image.

In one embodiment, the correction module 1506 is further configured to delete information of the crosstalk pixel to obtain a corrected echo image.

In one embodiment, the calibration module 1506 is further configured to divide the crosstalk pixel into a plurality of sub-pixels, and copy information of a pixel adjacent to the sub-pixel into the sub-pixel, so as to obtain a calibrated echo image.

For specific limitations of the lidar ranging apparatus, reference may be made to the above limitations of the lidar ranging method, which are not described herein again. All or part of each module in the laser radar ranging device can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, the internal structure of which may be as shown in fig. 16. The computer device includes a processor, a memory, a communication interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer readable instructions, and a database. The internal memory provides an environment for the operating system and execution of computer-readable instructions in the non-volatile storage medium. The database of the computer device is used for storing the echo images and the object pose information. The communication interface of the computer equipment is used for connecting and communicating with the laser radar. The computer readable instructions, when executed by a processor, implement a lidar ranging method.

Those skilled in the art will appreciate that the architecture shown in fig. 16 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

One or more non-transitory computer-readable storage media storing computer-readable instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of the various method embodiments described above.

It will be understood by those of ordinary skill in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware associated with computer readable instructions, which can be stored in a non-volatile computer readable storage medium, and when executed, can include processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.