阅读说明：本技术 障碍物测距方法、装置、电子设备及可读介质 (Obstacle ranging method and device, electronic equipment and readable medium ) 是由 曹杰葳 赖健明 郝俊杰 钟辉强 徐现昭 郑顺航 于 2021-09-13 设计创作，主要内容包括：本发明实施例提供了一种障碍物测距方法、装置、电子设备及可读介质,包括：在车辆停放在标定位置时,获取车门处于不同开启位置时车门相机采集到的标定场地的第一图像,其中,标定场地设有预设数量的标记点,确定出第一图像中各标记点的像素坐标,获取各标记点的世界坐标,根据标记点的像素坐标和世界坐标,计算出车门处于不同开启位置时车门相机的外部参数。应用本发明实施例,得到车门处于不同开启位置时车门相机的外部参数之后,在泊车后或者临时停车开启车门过程中,基于车门所处的开启位置对应的车门相机的外部参数,从而获取正确的全景图和准确识别出车门相机拍摄物体的距离位置。(The embodiment of the invention provides a method and a device for obstacle ranging, electronic equipment and a readable medium, wherein the method comprises the following steps: when a vehicle is parked at a calibration position, acquiring a first image of a calibration field acquired by a door camera when the door is at different opening positions, wherein the calibration field is provided with a preset number of marking points, determining pixel coordinates of each marking point in the first image, acquiring world coordinates of each marking point, and calculating external parameters of the door camera when the door is at different opening positions according to the pixel coordinates and the world coordinates of the marking points. After the external parameters of the door camera when the door is at different opening positions are obtained, the accurate panoramic image is obtained and the distance position of the door camera shooting object is accurately identified based on the external parameters of the door camera corresponding to the opening position of the door after parking or in the process of temporarily parking and opening the door.)

1. An obstacle ranging method, comprising:

when a vehicle is parked at a calibration position, acquiring a first image of a calibration site acquired by a door camera when a door is at different opening positions; the calibration site is provided with a preset number of marking points;

determining the pixel coordinates of each marking point in the first image;

acquiring world coordinates of each marking point;

calculating external parameters of the vehicle door camera when the vehicle door is at different opening positions according to the pixel coordinates and the world coordinates of the mark points;

and when the vehicle door is at different opening positions, measuring the distance between the obstacle shot by the vehicle door camera and the vehicle door based on the external parameters.

2. The method of claim 1, wherein acquiring a first image of the calibration field acquired by the door camera when the door is in different open positions while the vehicle is parked in the calibration position comprises:

when the vehicle door is positioned at each opening position, the vehicle door camera acquires first images of a plurality of calibration sites based on a preset sampling frequency;

determining a first image meeting a preset condition from the plurality of first images, and taking the first image as a first image when the vehicle door is at the opening position; the preset condition is that the first image comprises a preset number of mark points, and the definition of the first image is greater than a preset threshold value.

3. The method of claim 1, wherein said determining pixel coordinates of each of said marked points in said first image comprises:

calibrating each marking point in the first image to obtain a pixel coordinate of each marking point;

acquiring internal parameters of the vehicle door camera;

and carrying out distortion removal processing on the pixel coordinates of each mark point through the internal parameters to obtain normalized pixel coordinates.

4. The method of claim 3, further comprising, prior to said obtaining internal parameters of said door camera:

and calibrating the vehicle door camera through a calibration plate to obtain the internal parameters of the vehicle door camera.

5. The method of claim 4, wherein measuring the distance from the door to an obstacle captured by the door camera based on the external parameters while the door is in different open positions comprises:

determining a corresponding distance lookup table when the vehicle door is at different opening positions based on the internal parameters and the external parameters;

when a door of a vehicle is opened, determining pixel coordinates of a grounding point of an obstacle shot by a door camera;

acquiring a target world coordinate corresponding to the pixel coordinate of the grounding point of the obstacle;

obtaining a distance lookup table corresponding to the current opening position of the vehicle door; the current opening position of the vehicle door is the current world coordinate of the grounding point of the vehicle door;

and searching the distance between the target world coordinate and the current world coordinate of the door grounding point from the distance lookup table to be used as the distance between the obstacle grounding point and the door grounding point.

6. The method of claim 5, wherein the calibration site is provided with a plurality of grid points, and the determining the corresponding distance lookup table for the vehicle door at different opening positions based on the internal parameters and the external parameters comprises:

when the vehicle is parked at the calibration position, acquiring a second image of the calibration site acquired by the vehicle door camera when the vehicle door is at different opening positions;

performing projection transformation on the grid points in the second image through the internal parameters and the external parameters to obtain pixel coordinates of the grid points;

acquiring world coordinates of the grid points;

associating the pixel coordinates and the world coordinates corresponding to the same grid point;

and when the vehicle door is at different opening positions, calculating the distance between the world coordinate of the grid point and the world coordinate of the grounding point of the vehicle door, and storing the distance as a corresponding distance lookup table when the vehicle door is at different opening positions.

7. An obstacle ranging apparatus, comprising:

the image acquisition module is used for acquiring a first image of a calibration site acquired by a door camera when a door of a vehicle is at different opening positions when the vehicle is parked at the calibration position; the calibration site is provided with a preset number of marking points;

the position determining module is used for determining the pixel coordinates of each marking point in the first image;

the position acquisition module is used for acquiring the world coordinates of each marking point;

the parameter calculation module is used for calculating external parameters of the vehicle door camera when the vehicle door is at different opening positions according to the pixel coordinates and the world coordinates of the mark points;

and the distance measuring module is used for measuring the distance between the obstacle shot by the vehicle door camera and the vehicle door based on the external parameters when the vehicle door is at different opening positions.

8. The apparatus of claim 7, wherein the image acquisition module comprises:

the image acquisition sub-module is used for acquiring first images of a plurality of calibration fields by the vehicle door camera based on a preset sampling frequency when the vehicle door is positioned at each opening position;

the image determining submodule is used for determining a first image meeting a preset condition from the plurality of first images to be used as a first image when the vehicle door is at the opening position; the preset condition is that the first image comprises a preset number of mark points, and the definition of the first image is greater than a preset threshold value.

9. An electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

the memory is used for storing a computer program;

the processor, when executing a program stored on the memory, implementing the method of any of claims 1-6.

10. One or more computer-readable media having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the method of any of claims 1-6.

Technical Field

The present invention relates to the field of vehicle technologies, and in particular, to a method for measuring a distance to an obstacle, an apparatus for measuring a distance to an obstacle, an electronic device, and a computer-readable medium.

Background

A plurality of cameras work jointly to form the all-round-looking camera in the vehicle, and the image information of the cameras is fused, so that the visual blind areas and the visual dead angles can be eliminated. The external reference calibration of each camera of the all-round looking camera is a key link, the accuracy of the external reference calibration of each camera in the all-round looking camera influences whether the splicing and fusion of the images of the multiple cameras are perfect, and whether a vehicle can be effectively positioned, range-measured or obtain a panoramic image and the like can be determined.

However, at present, calibration of external parameters of a door camera is completed in a state that a door is closed, and the external parameters of the door camera are changed after parking or during a process of temporarily parking and opening the door, so that problems of obtaining an incorrect panoramic image, being unable to accurately identify a distance position of an object shot by the door camera, and the like occur.

Disclosure of Invention

The embodiment of the invention provides a method and a device for measuring distance of an obstacle, electronic equipment and a computer readable storage medium, which are used for solving the problem that external parameters of a vehicle door camera determined in a closed state of a vehicle door are not suitable for the opening process of the vehicle door.

The embodiment of the invention discloses a method for measuring the distance of an obstacle, which comprises the following steps:

when a vehicle is parked at a calibration position, acquiring a first image of a calibration site acquired by a door camera when a door is at different opening positions; the calibration site is provided with a preset number of marking points;

determining the pixel coordinates of each marking point in the first image;

acquiring world coordinates of each marking point;

calculating external parameters of the vehicle door camera when the vehicle door is at different opening positions according to the pixel coordinates and the world coordinates of the mark points; wherein the external parameters include a translation matrix and a translation matrix;

and when the vehicle door is at different opening positions, measuring the distance between the obstacle shot by the vehicle door camera and the vehicle door based on the external parameters.

Optionally, the acquiring, when the vehicle is parked at the calibration position, a first image of the calibration site acquired by the door camera when the door is at different opening positions includes:

when the vehicle door is positioned at each opening position, the vehicle door camera acquires first images of a plurality of calibration sites based on a preset sampling frequency;

determining a first image meeting a preset condition from the plurality of first images, and taking the first image as a first image when the vehicle door is at the opening position; the preset condition is that the first image comprises a preset number of mark points, and the definition of the first image is greater than a preset threshold value.

Optionally, the determining the pixel coordinates of each of the marker points in the first image includes:

calibrating each marking point in the first image to obtain a pixel coordinate of each marking point;

acquiring internal parameters of the vehicle door camera;

and carrying out distortion removal processing on the pixel coordinates of each mark point through the internal parameters to obtain normalized pixel coordinates.

Optionally, before the acquiring the internal parameters of the door camera, the method further includes:

and calibrating the vehicle door camera through a calibration plate to obtain the internal parameters of the vehicle door camera.

Optionally, the measuring a distance between an obstacle photographed by the door camera and the door based on the external parameter when the door is at different opening positions includes:

determining a corresponding distance lookup table when the vehicle door is at different opening positions based on the internal parameters and the external parameters;

when a door of a vehicle is opened, determining pixel coordinates of a grounding point of an obstacle shot by a door camera;

acquiring a target world coordinate corresponding to the pixel coordinate of the grounding point of the obstacle;

obtaining a distance lookup table corresponding to the current opening position of the vehicle door; the current opening position of the vehicle door is the current world coordinate of the grounding point of the vehicle door;

and searching the distance between the target world coordinate and the current world coordinate of the door grounding point from the distance lookup table to be used as the distance between the obstacle grounding point and the door grounding point.

Optionally, the calibration site is provided with a plurality of grid points, and the determining a corresponding distance lookup table when the vehicle door is at different opening positions based on the internal parameter and the external parameter includes:

when the vehicle is parked at the calibration position, acquiring a second image of the calibration site acquired by the vehicle door camera when the vehicle door is at different opening positions;

performing projection transformation on the grid points in the second image through the internal parameters and the external parameters to obtain pixel coordinates of the grid points;

acquiring world coordinates of the grid points;

associating the pixel coordinates and the world coordinates corresponding to the same grid point;

and when the vehicle door is at different opening positions, calculating the distance between the world coordinate of the grid point and the world coordinate of the grounding point of the vehicle door, and storing the distance as a corresponding distance lookup table when the vehicle door is at different opening positions.

Optionally, the determining the pixel coordinate of the grounding point of the obstacle photographed by the door camera includes:

detecting an obstacle shot by a vehicle door camera by using an object detection algorithm based on a deep learning model to obtain the outline size of the obstacle and the central pixel coordinate of the grounding point of the obstacle; the center pixel coordinate of the grounding point of the obstacle corresponds to the center point coordinate of the grounding point of the obstacle in a world coordinate system;

calculating to obtain the pixel coordinate of the grounding point of the obstacle according to the central pixel coordinate of the grounding point of the obstacle and the size of the outline; the pixel coordinate of the obstacle grounding point corresponds to the coordinate of the world coordinate system with the closest distance between the obstacle grounding point and the door grounding point.

The embodiment of the invention also discloses an obstacle ranging device, which comprises:

the image acquisition module is used for acquiring a first image of a calibration site acquired by a door camera when a door of a vehicle is at different opening positions when the vehicle is parked at the calibration position; the calibration site is provided with a preset number of marking points;

the coordinate determination module is used for determining the pixel coordinates of each marking point in the first image;

the coordinate acquisition module is used for acquiring world coordinates of each marking point;

the parameter calculation module is used for calculating external parameters of the vehicle door camera when the vehicle door is at different opening positions according to the pixel coordinates and the world coordinates of the mark points; wherein the external parameters include a translation matrix and a translation matrix;

and the distance measuring module is used for measuring the distance between the obstacle shot by the vehicle door camera and the vehicle door based on the external parameters when the vehicle door is at different opening positions.

Optionally, the image acquisition module includes:

the image acquisition sub-module is used for acquiring first images of a plurality of calibration fields by the vehicle door camera based on a preset sampling frequency when the vehicle door is positioned at each opening position;

the image determining submodule is used for determining a first image meeting a preset condition from the plurality of first images to be used as a first image when the vehicle door is at the opening position; the preset condition is that the first image comprises a preset number of mark points, and the definition of the first image is greater than a preset threshold value.

Optionally, the position determining module includes:

the coordinate calibration submodule is used for calibrating the mark points in the first image to obtain pixel coordinates of the mark points;

the parameter acquisition submodule is used for acquiring internal parameters of the car door camera;

and the position processing submodule is used for carrying out distortion removal processing on the pixel coordinates of each mark point through the internal parameters to obtain normalized pixel coordinates.

Optionally, the method further comprises:

and the camera calibration module is used for calibrating the vehicle door camera through the calibration plate to obtain the internal parameters of the vehicle door camera.

Optionally, the distance measuring module includes:

the lookup table determining submodule is used for determining a corresponding distance lookup table when the vehicle door is at different opening positions based on the internal parameter and the external parameter;

the coordinate determination submodule is used for determining the pixel coordinate of the grounding point of the obstacle shot by the door camera when the door of the vehicle is opened;

the coordinate acquisition submodule is used for acquiring a target world coordinate corresponding to the pixel coordinate of the grounding point of the obstacle;

the lookup table acquisition sub-module is used for acquiring a distance lookup table corresponding to the current opening position of the vehicle door; the current opening position of the vehicle door is the current world coordinate of the grounding point of the vehicle door;

and the distance lookup submodule is used for looking up the distance between the target world coordinate and the current world coordinate of the vehicle door grounding point from the distance lookup table to be used as the distance between the obstacle grounding point and the vehicle door grounding point.

Optionally, the calibration site is provided with a plurality of grid points, and the lookup table determination sub-module includes:

the image acquisition unit is used for acquiring a second image of the calibration site acquired by the door camera when the door is at different opening positions when the vehicle is parked at the calibration position;

a coordinate transformation unit, configured to perform projection transformation on the grid points in the second image through the internal parameters and the external parameters to obtain pixel coordinates of the grid points;

the coordinate acquisition unit is also used for acquiring world coordinates of the grid points;

the coordinate association unit is used for associating the pixel coordinates corresponding to the same grid point with the world coordinates;

and the distance calculation unit is used for calculating the distance between the world coordinate of the grid point and the world coordinate of the grounding point of the vehicle door when the vehicle door is at different opening positions, and storing the distance as a corresponding distance lookup table when the vehicle door is at different opening positions.

Optionally, the coordinate determination sub-module includes:

the obstacle detection unit is used for detecting an obstacle shot by a vehicle door camera based on an object detection algorithm of a deep learning model to obtain the outline size of the obstacle and the central pixel coordinate of the grounding point of the obstacle; the center pixel coordinate of the grounding point of the obstacle corresponds to the center point coordinate of the grounding point of the obstacle in a world coordinate system;

the coordinate calculation unit is used for calculating the pixel coordinate of the grounding point of the obstacle according to the central pixel coordinate of the grounding point of the obstacle and the size of the outline; the pixel coordinate of the obstacle grounding point corresponds to the coordinate of the world coordinate system with the closest distance between the obstacle grounding point and the door grounding point.

The embodiment of the invention also discloses electronic equipment which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory finish mutual communication through the communication bus;

the memory is used for storing a computer program;

the processor is configured to implement the method according to the embodiment of the present invention when executing the program stored in the memory.

Also disclosed are one or more computer-readable media having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform a method according to an embodiment of the invention.

The embodiment of the invention has the following advantages: when a vehicle is parked at a calibration position, acquiring a first image of a calibration field acquired by a vehicle door camera when a vehicle door is at different opening positions, wherein the calibration field is provided with a preset number of marking points, and determining the pixel coordinates of each marking point in the first image; the method comprises the steps of obtaining world coordinates of all mark points, and calculating external parameters of a vehicle door camera when a vehicle door is at different opening positions according to pixel coordinates and the world coordinates; when the vehicle door is at different opening positions, the distance between the obstacle shot by the vehicle door camera and the vehicle door is measured based on the external parameters. By applying the embodiment of the invention, the external parameters of the door camera are calibrated in the motion process of the door, the external parameters of the door camera relative to the world coordinate system when the door is at different opening positions are obtained, the external parameters of the door camera are dynamically updated according to the opening position of the door in the process of opening the door after parking or temporary parking, and the accurate panoramic image is obtained and the distance between the barrier shot by the door camera and the door is accurately measured in the state that the door of the vehicle is opened based on the updated external parameters.

Drawings

Fig. 1 is a flowchart illustrating steps of a method for measuring distance of an obstacle according to an embodiment of the present invention;

FIG. 2 is a flow chart illustrating steps of another method for measuring distance to an obstacle according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a calibration site image captured by a right door camera according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a calibration site image captured by a left door camera according to an embodiment of the present invention;

fig. 5 is a block diagram of an obstacle distance measuring device according to an embodiment of the present invention;

fig. 6 is a block diagram of an electronic device provided in an embodiment of the invention;

fig. 7 is a schematic diagram of a computer-readable medium provided in an embodiment of the invention.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.

The embodiment of the invention provides a method, a device, electronic equipment and a readable medium for measuring the distance of an obstacle, wherein the method comprises the steps of calibrating external parameters of a door camera in the process of moving a door to obtain the external parameters of the door camera relative to a world coordinate system when the door is at different opening positions, and dynamically updating the external parameters of the door camera according to the opening position of the door after parking or in the process of opening the door after temporary parking, so that the problems that the external parameters of the door camera determined in the state that the door is closed are not suitable for the opening process of the door, a wrong panoramic image is obtained, the distance between the obstacle shot by the door camera and the door cannot be accurately measured, and the like are solved.

Referring to fig. 1, a flowchart illustrating steps of a method for measuring distance of an obstacle provided in an embodiment of the present invention is shown, which may specifically include the following steps:

step 101: when a vehicle is parked at a calibration position, acquiring a first image of a calibration site acquired by a door camera when a door is at different opening positions; the calibration site is provided with a preset number of mark points.

The vehicle door camera is an AVM (around View monitoring) camera, and integrates the surrounding conditions of the vehicle through four cameras, namely a front camera, a rear camera, a left camera and a right camera, which are mounted on the vehicle, so as to assist a driver to safely park; a preset number of marking points are arranged on the calibration site, such as 5, 7 and the like; when external parameters of the door camera are calibrated, the vehicle needs to be parked at a specified calibration position, and a certain distance exists between the calibration position and a calibration field, so that a shooting area of the door camera can cover all the marker points in the calibration field; the opening position of the vehicle door is a coordinate of a vehicle door grounding point on a world coordinate system; the world coordinate system is a coordinate system established by using any position of the vehicle as an origin, for example, the world coordinate system is established at an intersection point of the center of a rear axle of the vehicle, which is vertically downward, and the ground, the front of the vehicle is a Z-axis, the right side of the advancing direction is an X-axis, and the vertical downward direction is a Y-axis.

Specifically, the vehicle is parked at a designated calibration position, a first image of a calibration site acquired by a door camera when the door is at different opening positions is acquired, for example, in the process of opening the door to the maximum angle from closing, the first image of the calibration site is acquired by the door camera at a certain sampling frequency, and the first image of the calibration site corresponding to the door at different opening positions is acquired.

Step 102: and determining the pixel coordinates of each marking point in the first image.

The image collected by the door camera can be converted into a digital image in an M multiplied by N array form in a computer, and the column number and the row number (u, v) of each point in the image in the array form are coordinates of the point in a pixel coordinate system.

Specifically, after a first image of a calibration field acquired by a vehicle door camera when a vehicle door is at different opening positions is acquired, the first image comprises all mark points on the calibration field, and the mark points in the first image are sequentially marked by using a marking tool to obtain the number of columns and the number of rows of each mark point in an array, so that the pixel coordinates of each mark point are obtained.

Step 103: and acquiring the world coordinates of each marking point.

Specifically, since the positions of the vehicle and the calibration site are relatively fixed, the coordinate M ═ X, Y, Z at the intersection of each special mark point with the ground vertically downward relative to the center of the rear axle of the vehicle, that is, the world coordinate of each mark point, can be calculated.

Step 104: calculating external parameters of the vehicle door camera when the vehicle door is at different opening positions according to the pixel coordinates and the world coordinates of the mark points; wherein the external parameters include a translation matrix and a translation matrix.

Specifically, after the pixel coordinates and the world coordinates of each mark Point are obtained when the door is at an opening position, based on the pixel coordinates and the world coordinates of each mark Point, a rotation matrix R and a translation matrix T of the door camera relative to a world coordinate system, that is, external parameters of the door camera at the opening position, can be calculated by using a PnP (passive-n-Point) algorithm, and the external parameters are repeatedly obtained when the door is at different opening positions.

The working principle of the PnP algorithm is to estimate the pose (external parameters) of the camera when n three-dimensional space points (world coordinates) and two-dimensional projection positions (pixel coordinates) thereof are known. Besides calculating the rotation matrix R and the translation matrix T of the car door camera by using the PnP algorithm, the external parameters of the car door camera can be calculated by using algorithms such as EPnP, RANSAC and the like, and thus, the embodiment of the invention is not limited.

Step 105: and when the vehicle door is at different opening positions, measuring the distance between the obstacle shot by the vehicle door camera and the vehicle door based on the external parameters.

In the current technical scheme on the market, since the external parameters of the door camera are calibrated in the closed state of the door, the external parameters of the door camera are inaccurate when the door is in the open state, and the distance between the obstacle shot by the door camera and the door cannot be accurately measured.

Specifically, according to the opening position of the vehicle door, the external parameters of the vehicle door camera are dynamically updated, and the distance between the obstacle shot by the vehicle door camera and the vehicle door is measured based on the updated external parameters.

In the embodiment of the invention, in the process of moving the vehicle door, the external parameters of the vehicle door camera are calibrated to obtain the external parameters of the vehicle door camera relative to the world coordinate system when the vehicle door is at different opening positions, the external parameters of the vehicle door camera are dynamically updated according to the opening position of the vehicle door after parking or in the process of opening the vehicle door after temporary parking, and the accurate panoramic view is obtained and the distance between the obstacle shot by the vehicle door camera and the vehicle door is accurately measured under the condition that the vehicle door of the vehicle is opened based on the updated external parameters.

Referring to fig. 2, a flowchart illustrating steps of another obstacle distance measuring method provided in the embodiment of the present invention is shown, which may specifically include the following steps:

step 201: when a vehicle is parked at a calibration position, acquiring a first image of a calibration site acquired by a door camera when a door is at different opening positions; the calibration site is provided with a preset number of mark points.

In an embodiment of the present invention, the step 201 includes: when the vehicle door is positioned at each opening position, the vehicle door camera acquires first images of a plurality of calibration sites based on a preset sampling frequency; determining a first image meeting a preset condition from the plurality of first images, and taking the first image as a first image when the vehicle door is at the opening position; the preset condition is that the first image comprises a preset number of mark points, and the definition of the first image is greater than a preset threshold value.

Specifically, in the process of acquiring a first image of a calibration site by a vehicle door camera, acquiring a plurality of first images of the calibration site when a vehicle door is at each opening position, filtering the plurality of first images based on a preset condition to obtain a first image containing a preset number of mark points, taking the first image with the definition of the first image being greater than a preset threshold value as the first image when the vehicle door is at the opening position, and randomly selecting one first image from the plurality of first images meeting the preset condition as the first image when the vehicle door is at the opening position if the plurality of first images meet the preset condition.

Referring to fig. 3, a schematic diagram of an image of a calibration site taken by a right door camera according to an embodiment of the present invention is shown, and as can be seen, 5 marker points (1, 2, 3, 4, 5) are provided in the diagram, and the 5 marker points are distributed in a certain rule. Referring to fig. 4, a schematic diagram of an image of a calibration site captured by a left door camera according to an embodiment of the present invention is shown.

Step 202: and calibrating each marking point in the first image to obtain the pixel coordinate of each marking point.

Specifically, after a first image of a calibration field acquired by a vehicle door camera when a vehicle door is at different opening positions is acquired, the first image comprises all mark points on the calibration field, and the mark points in the image are sequentially marked by using a marking tool to obtain the number of columns and the number of rows of each mark point in an array, so that the pixel coordinate of each mark point is obtained.

Step 203: and acquiring internal parameters of the vehicle door camera.

Step 204: and carrying out distortion removal processing on each pixel coordinate through the internal parameters to obtain a normalized pixel coordinate.

The internal parameters of the car door camera are parameters related to the characteristics of the camera, such as the focal length, the pixel size and the like of the camera; the image distortion is a phenomenon that an original image is distorted due to manufacturing accuracy of a lens used for a camera and deviation of an assembly process, and further the image is distorted.

Specifically, the pixel coordinates of each mark point are subjected to distortion removal through internal parameters of the camera, so that normalized pixel coordinates are obtained.

In an embodiment of the present invention, before the acquiring the internal parameters of the door camera, the method further includes: and calibrating the vehicle door camera through a calibration plate to obtain the internal parameters of the vehicle door camera.

Step 205: and acquiring the world coordinates of each marking point.

Step 206: calculating external parameters of the vehicle door camera when the vehicle door is at different opening positions according to the pixel coordinates and the world coordinates of the mark points; wherein the external parameters include a translation matrix and a translation matrix.

Step 207: and when the vehicle door is at different opening positions, measuring the distance between the obstacle shot by the vehicle door camera and the vehicle door based on the external parameters.

In an embodiment of the present invention, the step 207 includes:

determining a corresponding distance lookup table when the vehicle door is at different opening positions based on the internal parameters and the external parameters; when a door of a vehicle is opened, determining pixel coordinates of a grounding point of an obstacle shot by a door camera; acquiring a target world coordinate corresponding to the pixel coordinate of the grounding point of the obstacle; obtaining a distance lookup table corresponding to the current opening position of the vehicle door; the current opening position of the vehicle door is the current world coordinate of the grounding point of the vehicle door; and searching the distance between the target world coordinate and the current world coordinate of the door grounding point from the distance lookup table to be used as the distance between the obstacle grounding point and the door grounding point.

Specifically, the distance lookup tables corresponding to the vehicle door at different opening positions are determined through internal parameters and external parameters, and the distance lookup tables store the distances between the world coordinates corresponding to the shooting area of the camera and the world coordinates of the grounding point of the vehicle door when the distance lookup tables correspond to the opening positions of the vehicle door.

In the process of opening a vehicle door after a vehicle is temporarily parked or parked, determining the pixel coordinate of a grounding point of an obstacle shot by a vehicle door camera in a pixel coordinate system based on internal parameters and external parameters of the vehicle door camera, acquiring target world coordinates corresponding to the pixel coordinate of the grounding point of the obstacle, and acquiring a distance lookup table corresponding to the current opening position of the vehicle door, wherein the distance lookup table stores the distance between each world coordinate corresponding to an area shot by the camera and the world coordinate of the current grounding point of the vehicle door; and searching the distance between the target world coordinate and the world coordinate of the current door grounding point from the distance lookup table to be used as the distance between the obstacle grounding point and the door grounding point.

In the embodiment of the invention, the distance between each world coordinate corresponding to the area shot by the camera and the world coordinate of the grounding point of the vehicle door when the distance lookup table respectively stores the distance corresponding to the opening position of the vehicle door, and the distance between the grounding point of the obstacle shot by the vehicle door camera and the grounding point of the vehicle door is determined from the distance lookup table according to the current opening position of the vehicle door and the target world coordinate corresponding to the pixel coordinate of the grounding point of the obstacle in the pixel coordinate system in the process of opening the vehicle door after the vehicle is temporarily parked or parked, and whether the vehicle door can touch the obstacle is judged, so that the condition that the vehicle door touches the obstacle in the opening process is avoided.

In an embodiment of the present invention, the determining a distance lookup table corresponding to the door at different opening positions based on the internal parameter and the external parameter includes:

when the vehicle is parked at the calibration position, acquiring a second image of the calibration site acquired by the vehicle door camera when the vehicle door is at different opening positions; performing projection transformation on the grid points in the second image through the internal parameters and the external parameters to obtain pixel coordinates of the grid points; acquiring world coordinates of the grid points; associating the pixel coordinates and the world coordinates corresponding to the same grid point; and when the vehicle door is at different opening positions, calculating the distance between the world coordinate of the grid point and the world coordinate of the grounding point of the vehicle door, and storing the distance as a corresponding distance lookup table when the vehicle door is at different opening positions.

Specifically, the calibration site is full of a plurality of grid points besides preset marking points, after a second image of the calibration site acquired by the door camera when the door is at different opening positions is acquired, the second image is full of the grid points on the calibration site, and at the moment, the grid points in the second image are subjected to projection transformation through internal parameters and external parameters of the door camera to obtain pixel coordinates of the grid points in a pixel coordinate system;

meanwhile, because the positions of the vehicle and the calibration site are relatively fixed, the coordinates of each grid point at the intersection point of the vertical downward direction relative to the center of the rear axle of the vehicle and the ground, namely the world coordinates of the grid points, can be calculated.

And associating the coordinates in the pixel coordinate system corresponding to the same grid point with the coordinates in the world coordinate system, calculating the distance between the world coordinate of the grid point and the world coordinate of the grounding point of the vehicle door when the vehicle door is at different opening positions, and storing the distance as a corresponding distance lookup table when the vehicle door is at different opening positions.

In an embodiment of the present invention, the determining the pixel coordinate of the grounding point of the obstacle photographed by the door camera includes: detecting an obstacle shot by a vehicle door camera by using an object detection algorithm based on a deep learning model to obtain the outline size of the obstacle and the central pixel coordinate of the grounding point of the obstacle; the center pixel coordinate of the grounding point of the obstacle corresponds to the center point coordinate of the grounding point of the obstacle in a world coordinate system; calculating to obtain the pixel coordinate of the grounding point of the obstacle according to the central pixel coordinate of the grounding point of the obstacle and the size of the outline; the pixel coordinate of the obstacle grounding point corresponds to the coordinate of the world coordinate system with the closest distance between the obstacle grounding point and the door grounding point.

Wherein the outline size of the obstacle is the length and width of the obstacle; the obstacle occupies a plurality of coordinates of the pixel coordinate system, and therefore, in order to prevent the collision of the door with the obstacle, it is necessary to determine one grounding point of the obstacle closest to the grounding point of the door, and to determine one grounding point of the obstacle occupying the closest to the grounding point of the door among the plurality of coordinates of the pixel coordinate system.

Specifically, it is required to obtain internal parameters of a door camera and external parameters of the door camera at the current position of the door, detect an obstacle present in an image captured by the door camera through an object detection algorithm of a deep learning model based on the internal parameters and the external parameters, determine a contour size of the obstacle and a center pixel coordinate of a center point of the obstacle in a pixel coordinate system, for example, B ═ x, y, w, h, where x and y are the center coordinates of a contour frame, and w and h are the length and width of the contour frame.

The method includes the steps of calculating a grounding point position closest to (closest to) a door grounding point in an obstacle according to a central pixel coordinate and an outline size B, (U, V) determining a coordinate of the grounding point position in a pixel coordinate system, and specifically calculating the grounding point position closest to the door grounding point according to the central pixel coordinate and the outline size of the obstacle.

In the embodiment of the invention, the grounding point position closest to the grounding point of the vehicle door in the barrier is calculated through the pixel coordinate and the outline size of the barrier, so that the accuracy of the distance between the grounding point of the vehicle door and the grounding point of the barrier can be increased.

In the embodiment of the invention, in the process of moving the vehicle door, the external parameters of the vehicle door camera are calibrated to obtain the external parameters of the vehicle door camera relative to the world coordinate system when the vehicle door is at different opening positions, the external parameters of the vehicle door camera are dynamically updated according to the opening position of the vehicle door after parking or in the process of opening the vehicle door after temporary parking, and the accurate panoramic view is obtained and the distance position of the object shot by the vehicle door camera is accurately identified in the state that the vehicle door of the vehicle is opened based on the updated external parameters.

When the distance lookup tables respectively store the distances between the world coordinates corresponding to the shooting areas of the cameras and the world coordinates of the grounding points of the doors when the doors correspond to the opening positions of the doors, the distances between the grounding points of the obstacles shot by the cameras of the doors and the grounding points of the doors are determined from the distance lookup tables according to the current opening positions of the doors and the target world coordinates corresponding to the pixel coordinates of the obstacles in the process of opening the doors after the vehicles are temporarily parked or parked, and whether the doors touch the obstacles or not is judged so as to avoid the situation that the doors touch the obstacles in the opening process.

It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.

Referring to fig. 5, a block diagram of a structure of an obstacle distance measuring device provided in the embodiment of the present invention is shown, and specifically, the structure may include the following modules:

the image acquisition module 501 is configured to acquire, when a vehicle is parked at a calibration position, a first image of a calibration site acquired by a door camera when a door is at different opening positions; the calibration site is provided with a preset number of marking points;

a coordinate determining module 502, configured to determine pixel coordinates of each of the marked points in the first image;

a coordinate obtaining module 503, configured to obtain world coordinates of each of the mark points;

a parameter calculation module 504, configured to calculate external parameters of the door camera when the door is at different opening positions according to the pixel coordinates and the world coordinates of each of the mark points; wherein the external parameters include a translation matrix and a translation matrix;

and a distance measuring module 505, configured to measure a distance between the obstacle photographed by the door camera and the door based on the external parameter when the door is at different opening positions.

In an embodiment of the present invention, the image obtaining module 501 includes:

the image acquisition sub-module is used for acquiring first images of a plurality of calibration fields by the vehicle door camera based on a preset sampling frequency when the vehicle door is positioned at each opening position;

the image determining submodule is used for determining a first image meeting a preset condition from the plurality of first images to be used as a first image when the vehicle door is at the opening position; the preset condition is that the first image comprises a preset number of mark points, and the definition of the first image is greater than a preset threshold value.

In an embodiment of the present invention, the coordinate determining module 503 includes:

the position calibration submodule is used for calibrating the marking points in the first image to obtain pixel coordinates of the marking points;

the parameter acquisition submodule is used for acquiring internal parameters of the car door camera;

and the position processing submodule is used for carrying out distortion removal processing on the pixel coordinates of each mark point through the internal parameters to obtain normalized pixel coordinates.

In an embodiment of the present invention, the method further includes:

and the camera calibration module is used for calibrating the vehicle door camera through the calibration plate to obtain the internal parameters of the vehicle door camera.

Optionally, the open position of the door is the coordinate of the door grounding point in the world coordinate system; the distance measuring module 505 includes:

the lookup table determining submodule is used for determining a corresponding distance lookup table when the vehicle door is at different opening positions based on the internal parameter and the external parameter;

the coordinate determination submodule is used for determining the pixel coordinate of the grounding point of the obstacle shot by the door camera when the door of the vehicle is opened;

the coordinate acquisition submodule is used for acquiring a target world coordinate corresponding to the pixel coordinate of the grounding point of the obstacle;

the lookup table acquisition sub-module is used for acquiring a distance lookup table corresponding to the current opening position of the vehicle door;

and the distance lookup submodule is used for looking up the distance between the target world coordinate and the current world coordinate of the vehicle door grounding point from the distance lookup table to be used as the distance between the obstacle grounding point and the vehicle door grounding point.

In an embodiment of the present invention, the calibration site is provided with a plurality of grid points, and the lookup table determination sub-module includes:

the image acquisition unit is used for acquiring a second image of the calibration site acquired by the door camera when the door is at different opening positions when the vehicle is parked at the calibration position;

a coordinate transformation unit, configured to perform projection transformation on the grid points in the second image through the internal parameters and the external parameters to obtain pixel coordinates of the grid points;

the coordinate acquisition unit is also used for acquiring world coordinates of the grid points;

the coordinate association unit is used for associating the pixel coordinates corresponding to the same grid point with the world coordinates;

and the distance calculation unit is used for calculating the distance between the world coordinate of the grid point and the world coordinate of the grounding point of the vehicle door when the vehicle door is at different opening positions, and storing the distance as a corresponding distance lookup table when the vehicle door is at different opening positions.

In an embodiment of the present invention, the coordinate determination sub-module includes:

the obstacle detection unit is used for detecting an obstacle shot by a vehicle door camera based on an object detection algorithm of a deep learning model to obtain the outline size of the obstacle and the central pixel coordinate of the grounding point of the obstacle; the center pixel coordinate of the grounding point of the obstacle corresponds to the center point coordinate of the grounding point of the obstacle in a world coordinate system;

the coordinate calculation unit is used for calculating the pixel coordinate of the grounding point of the obstacle according to the central pixel coordinate of the grounding point of the obstacle and the size of the outline; the pixel coordinate of the obstacle grounding point corresponds to the coordinate of the world coordinate system with the closest distance between the obstacle grounding point and the door grounding point.

For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.

In addition, an electronic device is further provided in an embodiment of the present invention, as shown in fig. 6, and includes a processor 601, a communication interface 602, a memory 603, and a communication bus 604, where the processor 601, the communication interface 602, and the memory 503 complete mutual communication through the communication bus 604,

a memory 603 for storing a computer program;

the processor 601 is configured to implement the obstacle distance measuring method in the foregoing embodiment when executing the program stored in the memory 603.

The communication bus mentioned in the above terminal may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the terminal and other equipment.

The Memory may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.

In yet another embodiment of the present invention, as shown in fig. 7, a computer-readable storage medium 701 is further provided, which stores instructions that, when executed on a computer, cause the computer to perform an obstacle ranging method as described in the above embodiments.

In a further embodiment provided by the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the obstacle ranging method described in the above embodiment.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.