阅读说明：本技术 目标障碍物位置的获取方法和系统 (Method and system for acquiring position of target obstacle ) 是由 陈安 江志浩 崔迪潇 周棉炜 龚伟林 于 2019-11-27 设计创作，主要内容包括：本发明提供一种目标障碍物位置的获取方法和系统,该获取方法包括以下步骤：获取双目相机的第一图像和第二图像；根据第一图像获取目标障碍物信息；根据目标障碍物信息确认目标障碍物在第二图像中的搜索区域；在该搜索区域中搜索目标障碍物确定目标障碍物的精确视差值；根据目标障碍物的精确视差值及第二图像中的矩行框中心点的坐标值获取目标障碍物的位置信息；根据当前目标障碍物的位置信息和精确视差值更新目标障碍物的位置信息。本发明对图像中的特定区域来计算目标障碍物的视差,由于无需全图计算,只考虑了感兴趣的特定区域,计算量小且在高分辨率图像上也能实时处理。(The invention provides a method and a system for acquiring the position of a target obstacle, wherein the method comprises the following steps: acquiring a first image and a second image of a binocular camera; acquiring target obstacle information according to the first image; confirming a search area of the target obstacle in the second image according to the target obstacle information; searching the target obstacle in the search area to determine an accurate parallax value of the target obstacle; acquiring position information of the target obstacle according to the accurate parallax value of the target obstacle and the coordinate value of the rectangular frame center point in the second image; and updating the position information of the target obstacle according to the position information of the current target obstacle and the accurate parallax value. The invention calculates the parallax of the target obstacle for the specific area in the image, only considers the interested specific area without the calculation of the whole image, has small calculation amount and can process in real time on the high-resolution image.)

1. A method of acquiring a position of a target obstacle, the method comprising the steps of:

acquiring a first image and a second image of a binocular camera;

acquiring target obstacle information according to the first image;

confirming a search area of the target obstacle in the second image according to the target obstacle information;

searching a target obstacle in the search area, and determining an accurate parallax value of the target obstacle;

acquiring position information of the target obstacle according to the accurate parallax value of the target obstacle and the coordinate value of the rectangular frame center point in the second image;

updating the position information of the target obstacle according to the position information of the current target obstacle and the accurate parallax value;

wherein the confirming of the search area of the target obstacle in the second image according to the target obstacle information includes the substeps of:

acquiring the maximum parallax value of the target barrier in the tracking list according to the target barrier information;

and confirming a search area of the target obstacle in the second image according to the maximum parallax value and the size of the rectangular frame of the target obstacle in the first image.

2. The acquisition method according to claim 1, characterized in that it further comprises the steps of: and smoothing the updated position information of the target obstacle.

3. The acquisition method according to claim 1, wherein the acquiring target obstacle information from the first image includes the substeps of:

calculating the similarity between the target obstacle in the tracking list and the detected target obstacle;

and associating the target obstacles by using a Hungarian matching algorithm, acquiring the associated target obstacles and tracking image domain information of the associated target obstacles.

4. The method according to claim 1, wherein the formula for confirming the search area of the target obstacle in the second image according to the maximum parallax value and the size of the rectangular frame of the target obstacle in the first image is as follows:

RIO_R＝(X_L-disparity,Y_L-disparity,W_L+disparity,H_L)

wherein (X)_L,Y_LIs the rectangular box coordinates of the target obstacle in the first image; w_L,H_LA size of the target obstacle in a rectangular box in the first image; RIO_RIs a search area of the target obstacle in the second image; disparity is the maximum disparity value of the target obstacle.

5. The acquisition method according to claim 1, wherein the determining of the precise disparity value of the target obstacle comprises the sub-steps of:

sampling a detection area in the first image and a search area of a target obstacle in the second image to construct an image pyramid;

calculating the similarity of each candidate position of the first image on the second image layer by layer to generate a similarity response image;

respectively obtaining the position information of the matching point of the target barrier in the first image and the second image according to the similarity response image of the 0 th layer of the image pyramid;

and determining the accurate parallax value of the target barrier according to the position information of the matching points in the first image and the second image.

6. The method of claim 5, wherein the exact disparity value of the target obstacle is set as disparity_cDetermining the accurate disparity value of the target obstacle_cThe method comprises the following substeps:

the image pyramid comprises P-layer images, wherein the 0-layer image can be represented as

the similarity response image of the 0 th layer is R₀Selecting the point p corresponding to the maximum response value on the similarity response image of the 0 th layer_tIf point p_tIf the response value is smaller than the preset response threshold tsselect, setting the parallax value of the target obstacle to-1; if point p_tIf the response value of (d) is not less than the preset response threshold tsselect, the point p is pointed to_tFitting is performed to obtain a maximum point, which is used as matching point position information (x) of the target obstacle in the first image_L,y_L)；

Matching point position information (x) of the target obstacle in the first image_L,y_L) Converting to the second image to obtain the position information (x) of the matching point of the target obstacle in the second image_R,y_R)；

By the formula: disparity is a measure of the distance between two objects_c＝x_L-x_RAn accurate disparity value for the target obstacle is generated.

7. An acquisition system of a target obstacle comprises a first acquisition module, a second acquisition module, a confirmation module, a search module, a third acquisition module and an updating module; wherein the content of the first and second substances,

the first acquisition module is used for acquiring a first image and a second image of the binocular camera;

the second acquisition module is used for acquiring target obstacle information according to the first image;

the confirming module is used for confirming a searching area of the target obstacle in the second image according to the target obstacle information;

the searching module is used for searching a target obstacle according to the searching area and determining an accurate parallax value of the target obstacle;

the third acquisition module is used for acquiring the position information of the target obstacle according to the accurate parallax value of the target obstacle and the coordinate value of the rectangular frame center point in the second image;

the updating module updates the position information of the target obstacle according to the position information of the current target obstacle and the accurate parallax value;

wherein the confirmation module performs the following:

acquiring the maximum parallax value of the target barrier in the tracking list according to the target barrier information;

and confirming a search area of the target obstacle in the second image according to the maximum parallax value and the size of the rectangular frame of the target obstacle in the first image.

8. The acquisition system of claim 7, further comprising an optimization module that smoothes the updated position information of the target obstacle.

9. The acquisition system according to claim 7, wherein the formula for confirming the search area of the target obstacle in the second image according to the maximum parallax value and the size of the rectangular frame of the target obstacle in the first image is:

RIO_R＝(X_L-disparity,Y_L-disparity,W_L+disparity,H_L)

wherein (X)_L,Y_LIs the rectangular box coordinates of the target obstacle in the first image; w_L,H_LA size of the target obstacle in a rectangular box in the first image; RIO_RIs a search area of the target obstacle in the second image; disparity is the maximum disparity value of the target obstacle.

10. The acquisition system according to any one of claims 7 to 9, characterized in that the search module performs the following operations:

sampling a detection area in the first image and a search area of a target obstacle in the second image to construct an image pyramid;

calculating the similarity of each candidate position of the first image on the second image layer by layer to generate a similarity response image;

respectively obtaining the position information of the matching point of the target barrier in the first image and the second image according to the similarity response image of the 0 th layer of the image pyramid;

and determining the accurate parallax value of the target barrier according to the position information of the matching points in the first image and the second image.

Technical Field

The invention belongs to the technical field of intelligent traffic, and particularly relates to a method and a system for acquiring the position of a target obstacle.

Background

For over a century recently, the appearance of automobiles replaces the traditional transportation mode, so that the life of people is more convenient. In recent years, with the development of science and technology, especially the rapid development of intelligent computing, the research of the automatic driving automobile technology becomes a focus of all industries. The '12 leading edge technologies for determining future economy' report issued by McKensin discusses the influence degree of the 12 leading edge technologies on the future economy and society, and analyzes and estimates the respective economic and social influence of the 12 technologies in 2025, wherein the automatic driving automobile technology is ranked at the 6 th position, and the influence of the automatic driving automobile technology in 2025 is estimated as follows: economic benefits are about $ 0.2-1.9 trillion per year, and social benefits can recover 3-15 million lives per year.

In general, systems for autonomous driving of a vehicle are generally divided into three modules: the sensing module is equivalent to eyes of people, and the peripheral environment state is collected in real time through sensors such as a camera, a millimeter wave radar and a laser radar; the decision module is equivalent to the brain of a person and calculates the optimal driving decision plan according to the environmental state; and the third is an execution module, which is equivalent to hands and feet of a person and is used for executing decision-making commands and carrying out corresponding driving operations such as an accelerator, a brake, steering and the like.

The sensing module is an important module of the automatic driving system, and a safe automatic driving system cannot be realized without a reliable sensing module. In the sensing module, a binocular camera is an important device, and the binocular camera not only has a monocular function, but also can provide distance information between an obstacle and a vehicle. Binocular cameras have gradually become an indispensable sensor in an automatic driving system. Real-time, reliable and stable distance information is an important guarantee of the sensing module, and the sensing module cannot stably output reliable information such as distance and speed of the obstacle without accurate and smooth distance information.

The binocular ranging method is a visual ranging algorithm that calculates a difference in position between a left image and a right image (the left and right images refer to two images photographed by a binocular camera at a certain time) acquired by a binocular camera.

The traditional binocular vision distance measurement method firstly needs to calculate a binocular disparity map, and then calculates the 3D coordinates of points on the image according to the binocular disparity map. The binocular disparity map comprises the following steps: and calculating cost, aggregating the cost, calculating a disparity map and refining the disparity map. The method is limited by the matching effect, and the processing effect on the shielded area is poor; in addition, when the binocular camera is applied to an image with high resolution, processing delay thereof is rapidly increased, and a disparity map cannot be generated in real time.

With the development of deep learning technology in recent years, the traditional binocular distance measurement algorithm is rapidly developed, and a plurality of vision algorithms based on deep learning are developed at present, but the current binocular distance measurement algorithm based on deep learning does not exceed the flow of the traditional binocular distance measurement algorithm, and only some steps are partially improved; because the truth value of the disparity map is difficult to obtain, accurate training data is lacked in a binocular distance measurement model for deep learning, many training models are easy to be over-fitted to a specific camera, and the generalization is weak; in addition, the network layer number of the binocular ranging model for deep learning is high, so that the complexity of the model is high, the real-time performance is poor, and the binocular ranging model cannot be applied to the actual process of confirming the position of the target obstacle.

In summary, the prior art method for acquiring the position of the target obstacle has the following technical problems:

1. the obtained parallax is not accurate enough;

2. the calculation method is complex and the calculation efficiency is low;

3. real-time ranging cannot be realized on a high-resolution image;

4. images processed by the prior art are processed single frame information, continuous frame information is not considered, and finally obtained target tracks are not smooth enough.

Disclosure of Invention

The invention provides a method and a system for acquiring a position of a target obstacle, which aim to solve at least one technical problem in the prior art.

In a first aspect, an embodiment of the present invention provides a method for acquiring a position of a target obstacle, where the method includes:

acquiring a first image and a second image of a binocular camera;

acquiring target obstacle information according to the first image;

confirming a search area of the target obstacle in the second image according to the target obstacle information;

searching a target obstacle in the search area, and determining an accurate parallax value of the target obstacle;

acquiring position information of the target obstacle according to the accurate parallax value of the target obstacle and the coordinate value of the rectangular frame center point in the second image;

updating the position information of the target obstacle according to the position information of the current target obstacle and the accurate parallax value;

wherein the confirming of the search area of the target obstacle in the second image according to the target obstacle information includes the substeps of:

acquiring the maximum parallax value of the target barrier in the tracking list according to the target barrier information;

and confirming a search area of the target obstacle in the second image according to the maximum parallax value and the size of the rectangular frame of the target obstacle in the first image.

Further, the acquiring method further comprises the following steps: and smoothing the updated position information of the target obstacle.

Further, the acquiring the target obstacle information according to the first image includes the following sub-steps:

calculating the similarity between the target obstacle in the tracking list and the detected target obstacle;

and associating the target obstacles by using a Hungarian matching algorithm, acquiring the associated target obstacles and tracking image domain information of the associated target obstacles.

Further, the formula for determining the search area of the target obstacle in the second image according to the maximum parallax value and the size of the rectangular frame of the target obstacle in the first image is as follows:

RIO_R＝(X_L-disparity,Y_L-disparity,W_L+disparity,H_L)

wherein (X)_L,Y_LIs the rectangular box coordinates of the target obstacle in the first image; w_L,H_LA size of the target obstacle in a rectangular box in the first image; RIO_RIs a search area of the target obstacle in the second image; disparity is the maximum disparity value of the target obstacle.

Further, the determining the accurate parallax value of the target obstacle comprises the following sub-steps:

sampling a detection area in the first image and a search area of a target obstacle in the second image to construct an image pyramid;

calculating the similarity of each candidate position of the first image on the second image layer by layer to generate a similarity response image;

respectively obtaining the position information of the matching point of the target barrier in the first image and the second image according to the similarity response image of the 0 th layer of the image pyramid;

and determining the accurate parallax value of the target barrier according to the position information of the matching points in the first image and the second image.

Further, the precise disparity value of the target obstacle is set as disparity_cDetermining the accurate disparity value of the target obstacle_cThe method comprises the following substeps:

the image pyramid comprises P-layer images, wherein the 0-layer image can be represented as

Wherein the content of the first and second substances,

is the first image of the image to be displayed,

is a second image, i is an obstacle;

the similarity response image of the 0 th layer is R₀Selecting the point p corresponding to the maximum response value on the similarity response image of the 0 th layer_tIf point p_tIf the response value is smaller than the preset response threshold tsselect, setting the parallax value of the target obstacle to-1; if point p_tIf the response value of (d) is not less than the preset response threshold tsselect, the point p is pointed to_tFitting is performed to obtain a maximum point, which is used as matching point position information (x) of the target obstacle in the first image_L,y_L)；

Matching point position information (x) of the target obstacle in the first image_L,y_L) Converting to the second image to obtain the position information (x) of the matching point of the target obstacle in the second image_R,y_R)；

By the formula: disparity is a measure of the distance between two objects_c＝x_L-x_RAn accurate disparity value for the target obstacle is generated.

In a second aspect, an embodiment of the present invention provides a system for acquiring a position of a target obstacle, where the system includes a first acquiring module, a second acquiring module, a confirming module, a searching module, a third acquiring module, and an updating module; wherein the content of the first and second substances,

the first acquisition module is used for acquiring a first image and a second image of the binocular camera;

the second acquisition module is used for acquiring target obstacle information according to the first image;

the confirming module is used for confirming a searching area of the target obstacle in the second image according to the target obstacle information;

the searching module is used for searching a target obstacle according to the searching area and determining an accurate parallax value of the target obstacle;

the third acquisition module is used for acquiring the position information of the target obstacle according to the accurate parallax value of the target obstacle and the coordinate value of the rectangular frame center point in the second image;

the updating module updates the position information of the target obstacle according to the position information of the current target obstacle and the accurate parallax value;

wherein the confirmation module performs the following:

acquiring the maximum parallax value of the target barrier in the tracking list according to the target barrier information;

and confirming a search area of the target obstacle in the second image according to the maximum parallax value and the size of the rectangular frame of the target obstacle in the first image.

Further, the acquiring system further comprises an optimizing module, and the optimizing module performs smoothing processing on the updated position information of the target obstacle. Further, the formula for determining the search area of the target obstacle in the second image according to the maximum parallax value and the size of the rectangular frame of the target obstacle in the first image is as follows:

RIO_R＝(X_L-disparity,Y_L-disparity,W_L+disparity,H_L)

wherein (X)_L,Y_LIs the rectangular box coordinates of the target obstacle in the first image; w_L,H_LA size of the target obstacle in a rectangular box in the first image; RIO_RIs a search area of the target obstacle in the second image; disparity is the maximum disparity value of the target obstacle.

Further, the search module performs the following operations:

sampling a detection area in the first image and a search area of a target obstacle in the second image to construct an image pyramid;

calculating the similarity of each candidate position of the first image on the second image layer by layer to generate a similarity response image;

respectively obtaining the position information of the matching point of the target barrier in the first image and the second image according to the similarity response image of the 0 th layer of the image pyramid;

and determining the accurate parallax value of the target barrier according to the position information of the matching points in the first image and the second image.

The parallax of the target obstacle is calculated only aiming at the specific area in the picture, and only the interested specific area (namely a rectangular frame) is considered without calculating the whole picture, so that the calculation amount is small, the real-time processing can be realized even on a high-resolution image, and the real-time distance measurement can be realized on the high-resolution image;

and simultaneously using the position information and the accurate parallax of the target obstacle, and smoothing the position information to provide a smooth and stable target track.

Drawings

Fig. 1 is a schematic flowchart of a method for acquiring a target obstacle position according to an embodiment of the present invention;

FIG. 2 is a schematic flow chart of determining an accurate disparity value of a target obstacle according to an embodiment of the present invention;

fig. 3 is a schematic flowchart of a process for acquiring location information of a target obstacle according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of a system for acquiring a position of a target obstacle according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The present invention is described in detail with reference to the embodiments shown in the drawings, but it should be understood that these embodiments are not intended to limit the present invention, and those skilled in the art should understand that functional, methodological, or structural equivalents or substitutions made by these embodiments are within the scope of the present invention.