阅读说明：本技术 一种测距方法、装置及系统 (Distance measurement method, device and system ) 是由 朱雨时 孙杰 于 2018-06-15 设计创作，主要内容包括：本申请提供一种测距方法、装置及系统,该方法包括：获取摄像机、雷达在同一时刻分别采集到的视频图像、雷达信号集合；在所述视频图像中确定各目标对象对应的位置信息；确定所述视频图像中匹配到雷达信号的目标对象,并利用所述匹配到雷达信号的目标对象对应的位置信息,以及其匹配的雷达信号确定测距算法；根据所述测距算法、所述视频图像中未匹配到雷达信号的目标对象对应的位置信息,计算所述视频图像中未匹配到雷达信号的目标对象与指定车辆之间的距离。应用该方法,可以实现更全面、更准确地测量出车辆与其前方障碍物之间的距离。(The application provides a distance measuring method, a distance measuring device and a distance measuring system, wherein the method comprises the following steps: acquiring video images and radar signal sets respectively acquired by a camera and a radar at the same time; determining position information corresponding to each target object in the video image; determining a target object matched with a radar signal in the video image, and determining a ranging algorithm by using the position information corresponding to the target object matched with the radar signal and the matched radar signal; and calculating the distance between the target object which is not matched with the radar signal in the video image and the appointed vehicle according to the ranging algorithm and the position information corresponding to the target object which is not matched with the radar signal in the video image. By applying the method, the distance between the vehicle and the front obstacle can be more comprehensively and accurately measured.)

1. A method of ranging, the method comprising:

acquiring video images and radar signal sets respectively acquired by a camera and a radar at the same time;

determining position information corresponding to each target object in the video image;

determining a target object matched with a radar signal in the video image, and determining a ranging algorithm by using the position information corresponding to the target object matched with the radar signal and the matched radar signal;

and calculating the distance between the target object which is not matched with the radar signal in the video image and the appointed vehicle according to the ranging algorithm and the position information corresponding to the target object which is not matched with the radar signal in the video image.

2. The method of claim 1, wherein the location information corresponding to the target object comprises: and Y-axis coordinate values of lower frames of the circumscribed rectangular frames of the target objects in the video images in an image coordinate system.

3. The method according to claim 1, wherein a video area of the target object in the video image intersects with a projection area of the radar target corresponding to the radar signal matched with the target object projected onto the video image.

4. The method of claim 1, wherein the determining a ranging algorithm using the position information corresponding to the target object matched to the radar signal and the matched radar signal comprises:

establishing a model parameter set by using the position information corresponding to the target object matched with the radar signal and the matched radar signal;

and determining a monocular distance measurement model by using the model parameter set and the preset configuration parameters of the camera, wherein the monocular distance measurement model takes the position information corresponding to the target object as input and takes the distance between the target object and the appointed vehicle as output.

5. The method of claim 4, wherein the establishing a model parameter set by using the position information corresponding to the target object matched to the radar signal and the matched radar signal comprises:

determining a confidence level of radar signals matched by each target object matched to radar signals in the video image;

and adding the position information corresponding to each target object matched to the radar signal, the distance information in the matched radar signal and the confidence coefficient of the matched radar signal into a model parameter set as a group of model parameters.

6. The method of claim 4, wherein determining a monocular distance measurement model using the set of model parameters and pre-calibrated configuration parameters of the camera comprises:

fitting a road pitch angle by using each group of model parameters in the model parameter set;

and determining a monocular distance measurement model by using the road pitch angle and the pre-calibrated configuration parameters of the camera.

7. The method of claim 4, wherein prior to said determining a monocular distance measurement model using said set of model parameters and pre-calibrated configuration parameters of said camera, said method further comprises:

judging whether the number of the groups of the model parameters in the model parameter set is smaller than a preset threshold value or not;

and if so, adding preset model parameters into the model parameter set.

8. A ranging apparatus, the apparatus comprising:

the acquisition module is used for acquiring video images and radar signal sets which are respectively acquired by a camera and a radar at the same time;

the position determining module is used for determining position information corresponding to each target object in the video image;

the algorithm determining module is used for determining a target object matched with the radar signal in the video image, and determining a ranging algorithm by using the position information corresponding to the target object matched with the radar signal and the matched radar signal;

and the calculation module is used for calculating the distance between the target object which is not matched with the radar signal in the video image and the appointed vehicle according to the ranging algorithm and the position information corresponding to the target object which is not matched with the radar signal in the video image.

9. The apparatus according to claim 8, wherein the position information corresponding to the target object is: and Y-axis coordinate values of lower frames of the circumscribed rectangular frames of the target objects in the video images in an image coordinate system.

10. The apparatus according to claim 8, wherein the video area of the target object in the video image intersects the projection area of the radar target corresponding to the matching radar signal projected onto the video image.

11. The apparatus of claim 8, wherein the algorithm determination module comprises:

the establishing submodule is used for establishing a model parameter set by utilizing the position information corresponding to the target object matched with the radar signal in the video image and the matched radar signal;

and the processing submodule is used for determining a monocular distance measurement model by utilizing the model parameter set and the pre-calibrated configuration parameters of the camera, and the monocular distance measurement model takes the position information corresponding to the target object as input and takes the distance between the target object and the specified vehicle as output.

12. The apparatus of claim 11, wherein the establishing sub-module comprises:

the confidence coefficient determining submodule is used for determining the confidence coefficient of the radar signal matched with each target object matched with the radar signal in the video image;

and the adding submodule is used for adding the position information corresponding to each target object matched with the radar signal, the distance information in the matched radar signal and the confidence coefficient of the matched radar signal into a model parameter set as a group of model parameters.

13. The apparatus of claim 11, wherein the processing submodule comprises:

the pitch angle fitting submodule is used for fitting a road pitch angle by utilizing each group of model parameters in the model parameter set;

and the model determining submodule is used for determining a monocular distance measuring model by utilizing the road pitch angle and the pre-calibrated configuration parameters of the camera.

14. The apparatus of claim 11, further comprising:

the judging module is used for judging whether the number of the groups of the model parameters in the model parameter set is smaller than a preset threshold value or not;

and the supplement module is used for adding preset model parameters into the model parameter set if the number of the groups of the model parameters in the model parameter set is less than the preset threshold value.

15. A ranging system, the system comprising: computer equipment, radar, cameras;

the radar is used for collecting a radar signal set;

the camera is used for acquiring video images;

the computer device comprises a processor and a memory;

the memory is used for storing a computer program;

the processor is configured to execute the computer program stored in the memory, and when the processor executes the computer program, the processor implements the steps of the method according to any one of claims 1 to 7.

16. The system of claim 15, the radar being disposed at a front bumper cover of a designated vehicle, the camera being disposed at a front windshield of the designated vehicle.

Technical Field

The present application relates to the field of machine vision technologies, and in particular, to a distance measuring method, device, and system.

Background

With the increase of the number of automobiles, road traffic accidents frequently occur, wherein collisions between vehicles and rear-end collisions are the most likely traffic accidents, and in order to effectively reduce the occurrence of such traffic accidents, the prior art provides a front vehicle detection system to detect the distance between a front vehicle and a vehicle, and when the distance is too short, an early warning signal is timely sent out to remind a driver to take avoidance measures.

Disclosure of Invention

In view of the above, the present application provides a distance measuring method, device and system to measure the distance between a vehicle and an obstacle ahead of the vehicle more comprehensively and accurately.

Specifically, the method is realized through the following technical scheme:

according to a first aspect of embodiments of the present application, there is provided a ranging method, including:

acquiring video images and radar signal sets respectively acquired by a camera and a radar at the same time;

determining position information corresponding to each target object in the video image;

determining a target object matched with a radar signal in the video image, and determining a ranging algorithm by using the position information corresponding to the target object matched with the radar signal and the matched radar signal;

and calculating the distance between the target object which is not matched with the radar signal in the video image and the appointed vehicle according to the ranging algorithm and the position information corresponding to the target object which is not matched with the radar signal in the video image.

Optionally, the position information corresponding to the target object is: and Y-axis coordinate values of lower frames of the circumscribed rectangular frames of the target objects in the video images in an image coordinate system.

Optionally, a video region of the target object in the video image intersects with a projection region of the radar target corresponding to the matched radar signal projected onto the video image.

Optionally, the determining a ranging algorithm by using the position information corresponding to the target object matched to the radar signal and the radar signal matched thereto includes:

establishing a model parameter set by using the position information corresponding to the target object matched with the radar signal in the video image and the matched radar signal;

and determining a monocular distance measurement model by using the model parameter set and the preset configuration parameters of the camera, wherein the monocular distance measurement model takes the position information corresponding to the target object as input and takes the distance between the target object and the appointed vehicle as output.

Optionally, the establishing a model parameter set by using the position information corresponding to the target object matched with the radar signal in the video image and the matched radar signal includes:

determining a confidence level of radar signals matched by each target object matched to radar signals in the video image;

and adding the position information corresponding to each target object matched to the radar signal, the distance information in the matched radar signal and the confidence coefficient of the matched radar signal into a model parameter set as a group of model parameters.

Optionally, the determining a monocular distance measurement model by using the model parameter set and the pre-calibrated configuration parameters of the camera includes:

fitting a road pitch angle by using each group of model parameters in the model parameter set;

and determining a monocular distance measurement model by using the road pitch angle and the pre-calibrated configuration parameters of the camera.

Optionally, before the determining a monocular distance measurement model by using the model parameter set and the pre-calibrated configuration parameters of the camera, the method further includes:

judging whether the number of the groups of the model parameters in the model parameter set is smaller than a preset threshold value or not;

and if so, adding preset model parameters into the model parameter set.

According to a second aspect of embodiments of the present application, there is provided a ranging apparatus, the apparatus comprising:

the acquisition module is used for acquiring video images and radar signals which are respectively acquired by a camera and a radar at the same time;

the position determining module is used for determining position information corresponding to each target object in the video image;

the algorithm determining module is used for determining a target object which is not matched with the radar signal in the video image, and determining a ranging algorithm by using the position information corresponding to the target object which is matched with the radar signal and the matched radar signal;

and the calculation module is used for calculating the distance between the target object which is not matched with the radar signal in the video image and the appointed vehicle according to the ranging algorithm and the position information corresponding to the target object which is not matched with the radar signal in the video image.

Optionally, the position information corresponding to the target object is: and Y-axis coordinate values of lower frames of the circumscribed rectangular frames of the target objects in the video images in an image coordinate system.

Optionally, a video region of the target object in the video image intersects with a projection region of the radar target corresponding to the matched radar signal projected onto the video image.

Optionally, the algorithm determining module includes:

the establishing submodule is used for establishing a model parameter set by utilizing the position information corresponding to the target object matched with the radar signal in the video image and the matched radar signal;

and the processing submodule is used for determining a monocular distance measurement model by utilizing the model parameter set and the pre-calibrated configuration parameters of the camera, and the monocular distance measurement model takes the position information corresponding to the target object as input and takes the distance between the target object and the specified vehicle as output.

Optionally, the establishing sub-module includes:

the confidence coefficient determining submodule is used for determining the confidence coefficient of the radar signal matched with each target object matched with the radar signal in the video image;

and the adding submodule is used for adding the position information corresponding to each target object matched with the radar signal, the distance information in the matched radar signal and the confidence coefficient of the matched radar signal into a model parameter set as a group of model parameters.

Optionally, the processing sub-module includes:

the pitch angle fitting submodule is used for fitting a road pitch angle by utilizing each group of model parameters in the model parameter set;

and the model determining submodule is used for determining a monocular distance measuring model by utilizing the road pitch angle and the pre-calibrated configuration parameters of the camera.

Optionally, the apparatus further comprises:

the judging module is used for judging whether the number of the groups of the model parameters in the model parameter set is smaller than a preset threshold value or not;

and the supplement module is used for adding preset model parameters into the model parameter set if the number of the groups of the model parameters in the model parameter set is less than the preset threshold value.

According to a third aspect of embodiments of the present application, there is provided a ranging system, the system comprising: computer equipment, radar, cameras;

the radar is used for collecting a radar signal set;

the camera is used for acquiring video images;

the computer device comprises a processor and a memory;

the memory is used for storing a computer program;

the processor is configured to execute the computer program stored in the memory, and when the processor executes the computer program, the processor implements the steps of any of the ranging methods provided in the embodiments of the present application.

According to a fourth aspect of embodiments of the present application, there is provided a computer device comprising a processor, a communication interface, a memory, and a communication bus;

the processor, the communication interface and the memory are communicated with each other through the communication bus;

the memory is used for storing a computer program;

the processor is configured to execute the computer program stored in the memory, and when the processor executes the computer program, the processor implements the steps of any of the ranging methods provided in the embodiments of the present application.

According to a fifth aspect of embodiments of the present application, there is provided a computer-readable storage medium having a computer program stored therein, where the computer program is executed by a processor to implement the steps of any of the ranging methods provided by the embodiments of the present application.

It can be seen from the above embodiments that, by obtaining the video image and the radar signal set respectively collected by the camera and the radar at the same time, determining the position information corresponding to each target object in the video image, and determining the ranging algorithm by using the position information corresponding to the target object matched with the radar signal in the video image and the radar signal matched therewith, then, for the target object not matched with the radar signal, the distance between the target object not matched with the radar signal and the designated vehicle can be calculated according to the ranging algorithm and the position information corresponding to the target object not matched with the radar signal, because the ranging algorithm is dynamically calculated by using the known position information corresponding to the radar signal and the target object matched with each other, the ranging algorithm can adapt to the change of the road and has higher ranging accuracy, then, for a target object which is not matched with the radar signal, the distance between the target object and a specified vehicle can be measured by using the distance measurement algorithm, so that the distance between an obstacle in front of the vehicle and the vehicle can be more comprehensively and accurately measured by the processing.

Drawings

Fig. 1 is a schematic view of an application scenario for implementing a distance measuring method according to an exemplary embodiment of the present application;

FIG. 2 is an example of a calibrated radar coordinate system;

FIG. 3 is an example of a camera coordinate system;

FIG. 4 is a flowchart illustrating an embodiment of a ranging method according to an exemplary embodiment of the present disclosure;

FIG. 5 is an example of a video image;

FIG. 6 is a flow chart illustrating an embodiment of another ranging method according to an exemplary embodiment of the present application;

FIG. 7 is a block diagram illustrating an exemplary embodiment of a ranging device according to the present disclosure;

fig. 8 is a hardware structure diagram of a computer device provided in the present application.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.

Please refer to fig. 1, which is a schematic diagram of an application scenario for implementing a ranging method according to an exemplary embodiment of the present application. As shown in fig. 1, the vehicle 110, the camera 120, the radar 130, the obstacle 1#, the obstacle 2#, and the obstacle 3#, wherein the camera 120 may be a monocular camera disposed at a front windshield of the vehicle 110, the radar 130 may be a millimeter wave radar disposed at a front bumper of the vehicle 110, and the obstacles 1# -3 # are located in front of the vehicle 110, and may be a vehicle or other obstacles that may hinder the vehicle 110 from traveling.

The radar 130 generates sufficient electromagnetic energy by its internal transmitter (not shown in fig. 1) and transmits the electromagnetic energy through a duplexer (not shown in fig. 1) to an antenna (not shown in fig. 1), which radiates the electromagnetic energy into the atmosphere and focuses the electromagnetic energy on a detection range (e.g., the range indicated by the sparse dashed line in fig. 1) to form a beam that propagates forward. When the electromagnetic waves encounter radar targets in the range of the directions, reflections are generated along all directions, and a part of electromagnetic energy is reflected back to the direction of the radar 130 and is acquired by the antenna. Electromagnetic energy acquired by the antenna is transmitted to a receiver (not shown in fig. 1) inside the radar 130 through a transmit-receive switch to form a radar signal, and further, a signal processor (not shown in fig. 1) inside the radar 130 performs amplification, denoising and other processing on the radar signal to extract information such as distance, direction, speed and the like of the radar target.

The camera 120 may capture a video image within a certain viewing angle range (as indicated by a dense dotted line in fig. 1), and perform a series of processes such as foreground analysis, feature extraction, and depth calculation on the video image through a machine vision technology, for example, a monocular distance measurement technology, so as to obtain distances between target objects in front of the vehicle 110, for example, the obstacles 1# to 3# illustrated in fig. 1 and the vehicle 110.

As can be seen from the above description, although ranging can be achieved only by the radar 130, since the detection range of the radar 130 is limited, missing detection of an obstacle in front of the vehicle 110 is easily caused, for example, the obstacle 2# illustrated in fig. 1 does not fall within the detection range of the radar 130, so that a radar signal of the obstacle 2# cannot be detected, and meanwhile, since the radar signal is easily interfered, misdetection of the distance between the vehicle 110 and the obstacle in front is easily caused; if the distance measurement is performed only by the camera 120, it is likely that the distance measurement accuracy is low because: the existing distance calculation model, such as the monocular distance measurement model, is mostly realized based on an assumed ideal situation, for example, road edges on two sides of a road are assumed to be parallel to each other, a vehicle body trend is assumed to be parallel to the road edges, no bump is assumed in the road flat vehicle driving process, no camera lens distortion is assumed to exist, and the like.

Based on this, the embodiment of the application provides a distance measuring method to achieve more comprehensive and more accurate measurement of the distance between a vehicle and an obstacle in front of the vehicle.

The method proposed in the examples of the present application is explained as follows:

first, it is explained that the method proposed in the embodiment of the present application may be applied to the application scenario illustrated in fig. 1, and based on the application scenario illustrated in fig. 1, in the method, the internal reference of the camera 120 may be calibrated based on a "calibration by zhangnyou" (also referred to as zhang calibration) method, including: the scale factor f of the camera 120 in the direction of the u-axis of the image coordinate system_xScale factor f in the direction of the v-axis_yCoordinate value c of the principal point of the camera 120 in the u-axis direction of the image coordinate system_xCoordinate value c in the direction of the v-axis_y

Meanwhile, in the method, the installation position of the radar 130 may also be calibrated by using a goniometer, and as a result of the calibration, the ZOX plane of the radar coordinate system is parallel to the road surface, and the ZOY plane is parallel to the longitudinal section of the vehicle 110, for example, as shown in fig. 2, the radar coordinate system illustrated in fig. 2 is an example of the calibrated radar coordinate system, and the radar coordinate system uses a mass point of the radar 130 as a coordinate origin, uses a direction toward the vehicle head as a positive Z-axis direction, uses a direction perpendicular to the ground as a positive Y-axis direction, and uses a direction toward the right side of the driver as a positive X-axis direction.

In the method, the external reference of the camera 120 may be calibrated, including: the radar 130 is at a reference pitch angle θ relative to the camera 120 in the camera coordinate system_rccDisplacement Y relative to the camera 120 in the Y-axis direction of the camera coordinate system_rccA displacement Z relative to the camera 120 in the Z-axis direction of the camera coordinate system_rccAnd the displacement Y of the road surface in the Y-axis direction of the camera coordinate system relative to the radar 130_prc。

The camera coordinate system is an example of a camera coordinate system, in which the focal point of the camera 120 is used as the origin of coordinates, the heading direction of the vehicle head is used as the positive Z-axis direction, the heading direction perpendicular to the ground is used as the positive Y-axis direction, and the heading direction to the right of the driver is used as the positive X-axis direction, as shown in fig. 3.

In the embodiment of the present application, for convenience of description, the internal reference and the external reference are collectively referred to as configuration parameters of the camera 120, and as for the specific process of obtaining the internal reference and the external reference of the camera 120 through the calibration, a person skilled in the art may refer to the related description in the related art, and details of the embodiment of the present application are not described here.

The following examples are provided to explain the ranging method proposed in the examples of the present application.