阅读说明：本技术 一种车用开门预警防撞系统及控制方法 (Vehicle door opening early warning anti-collision system and control method ) 是由 欧阳颖 陈振斌 杨峥 于 2021-09-22 设计创作，主要内容包括：本发明涉及一种车用开门预警防撞系统及控制方法,应用于车辆的车载系统终端。方法包括：获取目标区域在当前时刻的第一雷达回波信号和第一图像。根据第一雷达回波信号和第一图像确定若干目标物对应在像素坐标系中的第一坐标。利用区域生长法对第一图像进行分割,获取第二图像。利用遗传算法对第一图像进行图像识别,确定若干目标物在目标区域中的位姿信息,遗传算法的输入包括第二图像。根据目标物的位姿信息进行防撞预警。本发明通过将毫米波雷达和摄像头采集到的数据融合,弥补单个传感器存在的不足,具备高精度、鲁棒性强、运算速度快等特点。通过本申请的技术方案可以进行不同等级的预警以达到警示车内外人员、保障人员安全和财产安全的目的。(The invention relates to a vehicle door opening early warning anti-collision system and a control method, which are applied to a vehicle-mounted system terminal of a vehicle. The method comprises the following steps: a first radar echo signal and a first image of a target area at the current moment are acquired. And determining first coordinates of the plurality of target objects in the pixel coordinate system according to the first radar echo signals and the first image. And segmenting the first image by using a region growing method to obtain a second image. And performing image recognition on the first image by using a genetic algorithm, and determining pose information of a plurality of target objects in the target area, wherein the input of the genetic algorithm comprises a second image. And performing anti-collision early warning according to the pose information of the target object. The invention makes up the defects of a single sensor by fusing the data acquired by the millimeter wave radar and the camera, and has the characteristics of high precision, strong robustness, high operation speed and the like. Through the technical scheme of this application can carry out the early warning of different grades in order to reach the purpose of warning the interior personnel of car, guarantee personnel's safety and property safety.)

1. The vehicle door opening anti-collision early warning control method is characterized by being applied to a vehicle-mounted system terminal of a vehicle and comprising the following steps:

acquiring a first radar echo signal of a target area at the current moment and a first image of the target area at the current moment;

determining first coordinates of a plurality of targets in a pixel coordinate system according to the first radar echo signal and the first image, wherein the first coordinates correspond to the plurality of targets in the pixel coordinate system, and the first coordinates comprise: performing wavelet transformation on the first radar echo signal to obtain a radar wavelet; performing wavelet inverse transformation operation on the radar wavelet to reconstruct a signal to obtain a second radar signal; determining first coordinates of a plurality of targets in a pixel coordinate system according to the second radar signals and the first image;

segmenting the first image by using a region growing method to obtain a second image; wherein, the seed point of the region growing method is the first coordinate;

performing image recognition on the first image by using a genetic algorithm, and determining pose information of the plurality of target objects in the target area; wherein the input to the genetic algorithm comprises the second image;

and performing anti-collision early warning according to the pose information of the plurality of target objects.

2. The control method according to claim 1, wherein the first radar echo signal is a beat signal, and the beat signal includes a signal obtained by mixing a radar echo signal with a radar transmission signal;

the determining, according to the first radar echo signal and the first image, first coordinates of a plurality of targets in a pixel coordinate system specifically includes:

acquiring a first radar beat signal of a target area at the current moment, and performing wavelet transformation on the beat signal to obtain a beat wavelet;

selecting a wavelet function to perform multi-scale decomposition on a first wavelet coefficient of the beat wavelet, and performing threshold quantization processing on each wavelet coefficient obtained by decomposition to obtain a second wavelet coefficient;

performing inverse wavelet transform operation on the second wavelet coefficient to reconstruct a signal to obtain a second radar beat signal;

and determining first coordinates of a plurality of targets in a pixel coordinate system according to the second radar beat signals and the first image.

3. The control method according to claim 1, wherein the image recognition of the first image by using a genetic algorithm, and the determining pose information of the plurality of objects in the target region comprises:

graying the first image to obtain a third image;

taking the second image and the third image as input of a genetic algorithm, and segmenting the third image by utilizing the genetic algorithm to obtain a fourth image; the fitness function of the genetic algorithm is determined through a Kmeans clustering algorithm;

and performing image recognition on the fourth image, and determining a plurality of target objects and pose information of target areas where the target objects are located.

4. The control method according to claim 1, wherein the pose information of the plurality of objects includes position information and attitude information of the objects, the position information including relative distances of the objects with respect to the own vehicle, and the attitude information including speed information of the objects with respect to the own vehicle and direction information of the objects;

the performing anti-collision early warning according to the pose information of the plurality of target objects comprises:

when the position with the target object is in the target area, judging that the time of the target object reaching the door of the vehicle is not more than a safety threshold value according to the pose information of the target object, and performing primary anti-collision early warning;

or the position with the target object is in the target area, the time of the target object reaching the door of the vehicle is judged to be larger than a safety threshold value according to the pose information of the target object, and secondary anti-collision early warning is carried out.

5. The utility model provides a vehicular early warning collision avoidance system that opens door which characterized in that includes:

the radar signal acquisition module is used for acquiring a first radar echo signal of a target area at the current moment;

the camera data acquisition module is used for acquiring a first image of the target area at the current moment;

the system control module is used for determining first coordinates of a plurality of targets in a pixel coordinate system according to the first radar echo signal and the first image, and comprises: performing wavelet transformation on the first radar echo signal to obtain a radar wavelet; performing wavelet inverse transformation operation on the radar wavelet to reconstruct a signal to obtain a second radar signal; determining first coordinates of a plurality of targets in a pixel coordinate system according to the second radar signals and the first image; segmenting the first image by using a region growing method to obtain a second image; wherein, the seed point of the region growing method is the first coordinate; performing image recognition on the first image by using a genetic algorithm, and determining pose information of the plurality of target objects in the target area; wherein the input to the genetic algorithm comprises the second image;

and the early warning execution module is used for carrying out anti-collision early warning according to the pose information of the plurality of target objects.

6. The warning and collision avoidance system of claim 5, wherein the first radar echo signal is a beat signal, the beat signal comprising a signal obtained by mixing a radar echo signal with a radar transmission signal;

the determining, according to the first radar echo signal and the first image, first coordinates of a plurality of targets in a pixel coordinate system specifically includes:

acquiring a first radar beat signal of a target area at the current moment, and performing wavelet transformation on the beat signal to obtain a beat wavelet;

selecting a wavelet function to perform multi-scale decomposition on a first wavelet coefficient of the beat wavelet, and performing threshold quantization processing on each wavelet coefficient obtained by decomposition to obtain a second wavelet coefficient;

performing inverse wavelet transform operation on the second wavelet coefficient to reconstruct a signal to obtain a second radar beat signal;

and determining first coordinates of a plurality of targets in a pixel coordinate system according to the second radar beat signals and the first image.

7. The warning collision avoidance system of claim 5, wherein the image recognition of the first image using a genetic algorithm to determine pose information of the plurality of targets in the target region comprises:

graying the first image to obtain a third image;

taking the second image and the third image as input of a genetic algorithm, and segmenting the third image by utilizing the genetic algorithm to obtain a fourth image; the fitness function of the genetic algorithm is determined through a Kmeans clustering algorithm;

and performing image recognition on the fourth image, and determining a plurality of target objects and pose information of target areas where the target objects are located.

8. The early warning collision avoidance system of claim 1, wherein the pose information of the plurality of objects comprises position information and attitude information of the objects, the position information comprises relative distances of the objects relative to the host vehicle, and the attitude information comprises speed information of the objects relative to the host vehicle and direction information of the objects;

the performing anti-collision early warning according to the pose information of the plurality of target objects comprises:

when the position with the target object is in the target area, judging that the time of the target object reaching the door of the vehicle is not more than a safety threshold value according to the pose information of the target object, and performing primary anti-collision early warning;

or the position with the target object is in the target area, the time of the target object reaching the door of the vehicle is judged to be larger than a safety threshold value according to the pose information of the target object, and secondary anti-collision early warning is carried out.

Technical Field

The application relates to the field of artificial intelligence, in particular to a vehicle door opening early warning and collision avoidance system and a control method.

Background

The development of economy enables people to continuously pursue good life, and the automobile becomes a travel tool for common people. The rapid increase of the automobile holding capacity causes road traffic to be more congested, so that simple vehicles such as electric bicycles and the like are selected by a plurality of residents in the aspect of short-distance travel such as leisure and entertainment, the complexity of road traffic participants is increased, and different types of road traffic accidents are increased. Among them, the door-opening collision accidents of automobiles are the most common. According to the statistical data of the national statistical bureau, the electric motorcycle has about 16.6 thousands of traffic accidents in total in the period of 2015 to 2018, wherein in the collision accident of the automobile and the two-wheel vehicle, the proportion of the automobile parking door opening accident is large, and the occurrence frequency is extremely high particularly in the period of holidays. At present, the vehicle types of the whole vehicle configuration door opening warning system are fewer, the related technology is not mature enough, most common vehicle types prompt other people to pay attention in a mode of starting vehicle front-rear left-right light flashing when a vehicle door is opened, but a driver cannot be reminded of taking corresponding measures in time, and therefore the probability of traffic accidents cannot be obviously reduced.

In the prior art, the vehicle door opening early warning technology is mainly realized by the following three ways: firstly, the detection of a target in a rear area of a vehicle is realized through vision, then data are transmitted to an internal computer through data transmission equipment, the internal computer processes and analyzes the running speed, the distance and the motion state of pedestrians and other vehicles, and the staying time of the target in an early warning area is calculated, so that whether a reminding and self-locking device is started or not is determined, and the detection distance is limited and the precision is poor in the mode, so that the state information of the moving target is insufficient due to the influence of various environmental factors; and secondly, an ultrasonic sensor technology is adopted, the technology is a system consisting of a single ultrasonic sensor or a laser sensor and the combination of the ultrasonic sensor or the laser sensor, and signals received by the sensors in unit time are converted into vehicle related parameters so as to judge and alarm. Although the precision is high and the cost is low, the detection distance is too close, the long-distance early warning and response speed required by design are difficult to meet, and the early warning time is greatly reduced, so that the area required to be monitored cannot be effectively covered; thirdly, detecting objects in a fixed area behind the vehicle through a frequency modulation radar technology to obtain parameters such as speed, distance and angle of a moving target, and judging whether to start the alarm device or not by comparing set early warning threshold values.

Therefore, it is necessary to research and develop a low-cost and high-efficiency vehicle door opening early warning system.

Disclosure of Invention

In order to solve the problems, the application provides a vehicle door opening early warning anti-collision system and a control method.

In a first aspect, the application provides a vehicle door opening anti-collision early warning control method, which is applied to a vehicle-mounted system terminal of a vehicle, and comprises:

acquiring a first radar echo signal of a target area at the current moment and a first image of the target area at the current moment;

determining first coordinates of a plurality of targets in a pixel coordinate system according to the first radar echo signal and the first image, wherein the first coordinates correspond to the plurality of targets in the pixel coordinate system, and the first coordinates comprise: performing wavelet transformation on the first radar echo signal to obtain a radar wavelet; performing wavelet inverse transformation operation on the radar wavelet to reconstruct a signal to obtain a second radar signal; determining first coordinates of a plurality of targets in a pixel coordinate system according to the second radar signals and the first image;

segmenting the first image by using a region growing method to obtain a second image; wherein, the seed point of the region growing method is the first coordinate;

performing image recognition on the first image by using a genetic algorithm, and determining pose information of the plurality of target objects in the target area; wherein the input to the genetic algorithm comprises the second image;

and performing anti-collision early warning according to the pose information of the plurality of target objects.

Preferably, the first radar echo signal is a beat signal, and the beat signal includes a signal obtained by mixing a radar echo signal with a radar emission signal;

the determining, according to the first radar echo signal and the first image, first coordinates of a plurality of targets in a pixel coordinate system specifically includes:

acquiring a first radar beat signal of a target area at the current moment, and performing wavelet transformation on the beat signal to obtain a beat wavelet;

selecting a wavelet function to perform multi-scale decomposition on a first wavelet coefficient of the beat wavelet, and performing threshold quantization processing on each wavelet coefficient obtained by decomposition to obtain a second wavelet coefficient;

performing inverse wavelet transform operation on the second wavelet coefficient to reconstruct a signal to obtain a second radar beat signal;

and determining first coordinates of a plurality of targets in a pixel coordinate system according to the second radar beat signals and the first image.

Preferably, the image recognition of the first image by using a genetic algorithm and the determination of the pose information of the plurality of targets in the target region comprise:

graying the first image to obtain a third image;

taking the second image and the third image as input of a genetic algorithm, and segmenting the third image by utilizing the genetic algorithm to obtain a fourth image; the fitness function of the genetic algorithm is determined through a Kmeans clustering algorithm;

and performing image recognition on the fourth image, and determining a plurality of target objects and pose information of target areas where the target objects are located.

Preferably, the pose information of the plurality of objects comprises position information and pose information of the objects, the position information comprises relative distances of the objects relative to the self-vehicle, and the pose information comprises speed information of the objects relative to the self-vehicle and direction information of the objects;

the performing anti-collision early warning according to the pose information of the plurality of target objects comprises:

when the position with the target object is in the target area, judging that the time of the target object reaching the door of the vehicle is not more than a safety threshold value according to the pose information of the target object, and performing primary anti-collision early warning;

or the position with the target object is in the target area, the time of the target object reaching the door of the vehicle is judged to be larger than a safety threshold value according to the pose information of the target object, and secondary anti-collision early warning is carried out.

In a second aspect, the present application provides an automobile-used early warning collision avoidance system that opens door, its characterized in that includes:

the radar signal acquisition module is used for acquiring a first radar echo signal of a target area at the current moment;

the camera data acquisition module is used for acquiring a first image of the target area at the current moment;

the system control module is used for determining first coordinates of a plurality of targets in a pixel coordinate system according to the first radar echo signal and the first image, and comprises: performing wavelet transformation on the first radar echo signal to obtain a radar wavelet; performing wavelet inverse transformation operation on the radar wavelet to reconstruct a signal to obtain a second radar signal; determining first coordinates of a plurality of targets in a pixel coordinate system according to the second radar signals and the first image; segmenting the first image by using a region growing method to obtain a second image; wherein, the seed point of the region growing method is the first coordinate; performing image recognition on the first image by using a genetic algorithm, and determining pose information of the plurality of target objects in the target area; wherein the input to the genetic algorithm comprises the second image;

and the early warning execution module is used for carrying out anti-collision early warning according to the pose information of the plurality of target objects.

Preferably, the first radar echo signal is a beat signal, and the beat signal includes a signal obtained by mixing a radar echo signal with a radar emission signal;

the determining, according to the first radar echo signal and the first image, first coordinates of a plurality of targets in a pixel coordinate system specifically includes:

acquiring a first radar beat signal of a target area at the current moment, and performing wavelet transformation on the beat signal to obtain a beat wavelet;

selecting a wavelet function to perform multi-scale decomposition on a first wavelet coefficient of the beat wavelet, and performing threshold quantization processing on each wavelet coefficient obtained by decomposition to obtain a second wavelet coefficient;

performing inverse wavelet transform operation on the second wavelet coefficient to reconstruct a signal to obtain a second radar beat signal;

and determining first coordinates of a plurality of targets in a pixel coordinate system according to the second radar beat signals and the first image.

Preferably, the image recognition of the first image by using a genetic algorithm and the determination of the pose information of the plurality of targets in the target region comprise:

graying the first image to obtain a third image;

taking the second image and the third image as input of a genetic algorithm, and segmenting the third image by utilizing the genetic algorithm to obtain a fourth image; the fitness function of the genetic algorithm is determined through a Kmeans clustering algorithm;

and performing image recognition on the fourth image, and determining a plurality of target objects and pose information of target areas where the target objects are located.

Preferably, the pose information of the plurality of objects comprises position information and pose information of the objects, the position information comprises relative distances of the objects relative to the self-vehicle, and the pose information comprises speed information of the objects relative to the self-vehicle and direction information of the objects;

the performing anti-collision early warning according to the pose information of the plurality of target objects comprises:

when the position with the target object is in the target area, judging that the time of the target object reaching the door of the vehicle is not more than a safety threshold value according to the pose information of the target object, and performing primary anti-collision early warning;

or the position with the target object is in the target area, the time of the target object reaching the door of the vehicle is judged to be larger than a safety threshold value according to the pose information of the target object, and secondary anti-collision early warning is carried out. According to the technical scheme, the data collected by the millimeter wave radar and the camera sensor are fused, so that the defects of a single sensor are overcome, and the method has the characteristics of high precision, strong robustness, high operation speed and the like. Through the technical scheme of this application can carry out the early warning of different grades in order to reach the purpose of warning the interior personnel of car, guarantee personnel's safety and property safety.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic application diagram of the technical solution provided in the embodiment of the present application;

fig. 2 is a flowchart of a method of early warning control provided in an embodiment of the present application;

fig. 3 is a schematic diagram of a system for early warning control provided in an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

For the convenience of understanding of the embodiments of the present invention, the following description will be further explained with reference to specific embodiments, which are not to be construed as limiting the embodiments of the present invention.

Fig. 1 is an application schematic diagram of a technical scheme provided in an embodiment of the present application. Referring to fig. 1, the anti-collision early warning control method is applied to a vehicle-mounted system terminal of a vehicle, a radar detection area is a shadow part in the figure, and a camera detection area includes shadow parts at the front and rear sides of the vehicle. The anti-collision warning control is executed according to different states of a target object (a vehicle, a pedestrian or a stationary object) in the radar detection area.

Fig. 2 is a flowchart of a method for early warning control provided in the embodiment of the present application. As shown in fig. 2, the anti-collision warning control method provided by the present application includes:

s201: a first radar echo signal of a target area at the current moment and a first image of the target area at the current moment are obtained.

In one possible implementation mode, when the vehicle speed sensor detects that the vehicle speed of the vehicle is continuously lower than 5km/h and the continuous braking action of the vehicle occurs in the time period, the anti-collision early warning system in the vehicle-mounted system terminal of the vehicle is started, and the vehicle enters an anti-collision early warning state. The reason for starting the anti-collision early warning system when the vehicle speed is continuously lower than 5km/h is that the time required for completing the door opening action is very short, so that the system is started in advance to complete the initialization of the system, and the system is prevented from missing the anti-collision early warning time.

Referring to fig. 1, the anti-collision warning system is started, the system sends out a data acquisition instruction, sampling is started by the millimeter wave radar and the camera at the same time, sampling areas are right behind the vehicle and a shaded part shown in fig. 1, a, b and c represent longitudinal distances, d represents a transverse distance, wherein the longitudinal distance a is 15m as an example, the longitudinal distance b is 5m as an example, the longitudinal distance c is 30m as an example, and the transverse distance d is 1.3 times of the length of the door when the door of the vehicle is completely opened.

When the millimeter wave radar works, the millimeter wave radar sends linear frequency modulation continuous waves through the receiving and transmitting antenna to detect a plurality of possible target objects and receives a first radar echo signal returned by the target objects.

S202: and determining first coordinates of a plurality of targets in a pixel coordinate system according to the first radar echo signals and the first image.

The first radar echo signals acquire a plurality of target objects, the acquired target objects are recorded in a pixel coordinate system, and a first coordinate of the target objects in the pixel coordinate system is determined.

Specifically, wavelet transformation is carried out on a first radar echo signal to obtain a radar wavelet, wavelet inverse transformation operation is carried out on the radar wavelet to reconstruct signals to obtain a second radar signal, and first coordinates of a plurality of target objects in a pixel coordinate system are determined according to the second radar signal and the first image.

Due to noise and the like, the millimeter wave radar detects that the object contains target object signals which may threaten the vehicle, and a large amount of false target signals may exist. The existence of false target signals can increase the burden of system signal processing, so that the anti-collision early warning system cannot give early warning in time, driving safety is endangered, and accidents are caused. Therefore, for targets which are missed or mistakenly detected by the millimeter wave radar, effective detection and elimination are carried out by combining continuous multi-frame data.

In one possible embodiment, the sampling frequency of the millimeter wave radar and the sampling frequency of the camera are different. The sampling frequency of the millimeter wave radar is assumed to be A frame/second, the sampling frequency of the camera is B frame/second, wherein A < B, the data collected by the millimeter wave radar with lower sampling frequency is selected as the basis, the data collected by the camera is subjected to frame extraction processing, the data collected by the two different sensors in space and time are fused, the same frame of target image is detected and identified by the two sensors in the same time and space, and the precision guarantee is provided for the follow-up realization of matching the same target object and the acquisition of the accurate position and running state information of the target object.

Specifically, a first image of the target area at the current moment is shot through a camera, and the camera is a monocular camera.

In a more specific embodiment, the first radar echo signal is a beat signal, and the beat signal includes a signal obtained by mixing the radar echo signal and a radar transmission signal.

In a more specific embodiment, the radar-emitting signal is a radar signal from a millimeter wave radar mounted on the underside of the rear view mirror. Specifically, referring to fig. 1, reference numerals 1 and 2 denote millimeter wave radars installed on the lower side of the rearview mirror, and the radar emission signals are radar signals emitted by the millimeter wave radars corresponding to reference numerals 1 and 2 shown in the figure.

The acquiring a first radar echo signal of a target area at the current moment, and determining first coordinates of a plurality of target objects in a pixel coordinate system according to the first radar echo signal comprises:

s2021: the method comprises the steps of obtaining a first radar beat signal of a target area at the current moment, and performing wavelet transformation on the beat signal to obtain a beat wavelet.

Specifically, the input beat signal is wavelet transformed by a formula, which is expressed as:

wherein S (t) represents beat signal, K represents frequency modulation slope, exp represents exponential function with natural constant e as base, j represents complex exponential signal, v represents relative speed, c represents light speed, f₀The method comprises the following steps of characterizing the frequency of an object when the object is static, τ (T) characterizing the delay of an echo signal relative to a transmitted signal, B characterizing the frequency sweep bandwidth, T characterizing the modulation period, and e (T) white noise in the echo signal.

S2022: and selecting a wavelet function to perform multi-scale decomposition on the first wavelet coefficient of the beat wavelet, and performing threshold quantization processing on each wavelet coefficient obtained by decomposition to obtain a second wavelet coefficient.

Specifically, the Symmlet wavelet basis function pair s (t) is selected for multi-scale decomposition. Since high-frequency signals are represented as noise in the wavelet domain, and the corresponding wavelet coefficients are generally lower than those of the signals, the wavelet coefficients of the high-frequency part at each scale are subjected to threshold quantization processing. Because the difference between the echo signal of the ideal millimeter wave radar and the noise function is only a specific numerical value, the hard threshold is directly adopted as the threshold contraction function, and the value calculation formula of the threshold function adopts a uniform threshold method.

The threshold is calculated by the following formula:

in the formula, delta represents a threshold value, sigma represents the standard deviation of noise, and N represents the number of sampling points of the radar echo signal.

Further, after the threshold is calculated by the above formula, the wavelet coefficients of each scale are subjected to hard threshold processing, that is, the wavelet coefficients in the range of (- δ, δ) are set to 0, and the wavelet coefficients above the threshold are retained to retain the edge contour information of the image.

S2023: and performing inverse wavelet transform operation on the second wavelet coefficient to reconstruct a signal to obtain a second radar beat signal.

Since the second wavelet coefficient is a denoised wavelet coefficient, performing inverse wavelet transform on the second wavelet coefficient can obtain a denoised radar beat signal, that is, the second radar beat signal is a denoised radar beat signal.

S2024: and determining first coordinates of the plurality of target objects in the pixel coordinate system according to the second radar beat signals.

And characterizing a millimeter wave radar coordinate system by using a coordinate system Or-XrYrZr, characterizing a world coordinate system by using a coordinate system Ow-XwYwZw, and characterizing a camera coordinate system by using a coordinate system Oc-XcYcZc. In one possible embodiment, determining the first coordinates of the plurality of objects in the pixel coordinate system according to the second radar beat signal includes:

and determining coordinates of the plurality of target objects in the millimeter wave radar coordinate system, and converting the millimeter wave radar coordinate system into a world coordinate system.

Specifically, if the relation between a target object P detected by the millimeter wave radar with the reference number 1 and the current millimeter wave radar is denoted as P (R, α), where R denotes a straight line distance at which the millimeter wave radar reaches the target object P, α denotes an included angle between a vertical line of the millimeter wave radar and a connecting line of the target object P, and a horizontal distance and a vertical distance of the target object P in a world coordinate system can be obtained by performing triangular transformation through R and α. The coordinates of the target object P in the world coordinate system are:

in the formula, Xr and Zr represent the coordinate of the target object P in a radar coordinate system, Z_HThe distance between Zr and Zw is represented, H represents the distance between Yr and Yw, and X represents_HCharacterize the distance of Xr from Xw.

Further, the world coordinate system is converted into a camera coordinate system.

Specifically, the coordinates of the target object P in the world coordinate system are denoted as P_wP in the camera coordinate system_cThe following relationship exists between the two:

P_c＝T_cwP_w

in the formula, T_cwRepresenting a transformation matrix between the world coordinate system and the camera coordinate system in the form ofWherein R represents a rotation matrix, t represents a translation matrix, O^TThe transposed matrix is characterized.

Further, the camera coordinate system is converted into an image coordinate system.

Specifically, the coordinates of the target object P on the camera coordinate system are [ Xc, Yc, Zc [ ]]The coordinates on the image coordinate system are [ X, Y ]]The coordinates on the pixel coordinate system are [ u, v ]]The formula for converting the camera coordinate system into the image coordinate system is as follows:wherein f represents the focal length.

Further, the image coordinate system is converted into a pixel coordinate system, and the formula is as follows:wherein fx represents a focal length in the X-axis direction, fy represents a focal length in the Y-axis direction, and c_xc_yCharacterizing the offset of the camera optical axis in the pixel coordinate system. It is understood that the physical imaging plane is substantially composed of pixel units one by one, and in the case where the pixel units are not square, the values of the focal length fx in the X-axis direction and the focal length fy in the Y-axis direction are different, and in the case where the pixel units are square, the values of the focal length fx in the X-axis direction and the focal length fy in the Y-axis direction are the same.

S203: segmenting the first image by using a region growing method to obtain a second image; wherein the seed point of the region growing method is the first coordinate.

Wherein, the process of acquiring the second image comprises:

s2031: and acquiring the first image gray level histogram.

S2032: and acquiring the optimal threshold T of image segmentation by using the maximum inter-class variance method.

Specifically, let T be the threshold of image segmentation, regard the pixel not greater than threshold T as foreground point, regard the pixel greater than threshold T as background point. The formula of the variance method between the maximum classes is as follows:

g＝w₀×(u₀-u)²+w₁×(u₁-u)²

where g denotes the variance between the foreground and background images, u denotes the mean gray value of the image₀Mean gray value, u, characterizing the foreground map₁Mean gray value, w, characterizing the background map₀Representing the proportion of foreground points to the pixels of the whole image, w₁And representing the proportion of the background points in the pixel points of the whole image.

Through continuous iteration traversing the whole image, the optimal threshold value T is found so that g obtains the maximum value. In a more specific embodiment, the initial threshold T is set to 127.

S2033: and segmenting the first image by using a region growing method to obtain a second image.

Specifically, a target object returned by the millimeter wave radar is recorded in a first coordinate of a pixel coordinate system to be used as a growing point of the region growing method.

In view of the strong similarity between the vehicle and the boundary region of the background image, the boundary region is a region where the gradient indicates a large change in the image. Therefore, it is not enough to judge whether the difference between the seed point and the pixel in the field is smaller than a certain threshold, and the technical scheme not only judges whether the pixel in the field of the seed point is smaller than the threshold T obtained by the maximum inter-class variance method, but also needs to judge whether the gradient difference between the seed point and the field is smaller than the threshold T. When both are satisfied, it is determined that both are similar. The gradient difference calculation formula is expressed as follows:

in the formula I_iCharacterizing the gray value of the pixel to be detected, I_aCharacterization I_iThe field pixel points of (1) are,and characterizing a binary function to calculate a partial derivative. In a more specific example, the threshold t for the gradient difference between a seed point and its domain point is set at 44.

And when the eight fields of the growing point do not meet the growing criterion, stopping growing and obtaining a second image.

Part of the codes for the region growing method are as follows:

s204: performing image recognition on the first image by using a genetic algorithm, and determining pose information of the plurality of target objects in the target area; wherein the input to the genetic algorithm comprises the second image.

In one possible embodiment, the image recognition of the first image by using a genetic algorithm, and the determining of the pose information of the plurality of targets in the target region comprises:

s2041: and graying the first image to obtain a third image.

Specifically, the first image is grayed according to a graying formula Gray of 0.114B +0.587G +0.299R to obtain a grayscale histogram of the image, wherein Gray represents the obtained grayscale value, and RGB represents the red, green and blue color value of the original image. And carrying out filtering and denoising treatment on the grayed image to obtain a third image.

S2042: taking the second image and the third image as input of a genetic algorithm, and segmenting the third image by utilizing the genetic algorithm to obtain a fourth image; the fitness function of the genetic algorithm is determined by a Kmeans clustering algorithm

Specifically, the third image is a grayed image, and the population information is initialized by using the third image as an input of the genetic algorithm.

And taking the second image and the third image as input of a genetic algorithm to initialize population information. In a more specific example, the number of initialization populations is 10, the number of chromosomes is 8, and the maximum number of iterations is 200. Further, the fitness function is determined by means of a Kmeans clustering algorithm. And selecting the K value as the number of seed points, namely selecting a plurality of random points of the seed points as a clustering center. And calculating Euclidean distances from each data point in a data set consisting of pixel points of the third image to K central points, associating the Euclidean distances with the central point closest to the data point, and gathering all points associated with the same central point into one class. And calculating the mean value of each group, and moving the cluster center associated with the group to the position of the mean value. This step is repeated until the cluster center does not change.

The objective function of the KMeans clustering algorithm is to minimize the sum of squared distances from an object to the centroid of the cluster to which the object belongs, and the formula is:

in the formula, dist represents the Euclidean distance function, x represents the pixel point in the image, c_iCharacterizing the center of mass, g_aAn objective function is characterized.

When a genetic algorithm is applied to clustering and segmentation, each individual is represented by the formula as an objective function, if the square sum is smaller, the adaptive value of the individual is higher, and therefore, the fitness function is set to be 1/g_a。

Further, individuals with high fitness are selected using roulette methods. Roulette is a playback-type random sampling method in which the probability of each individual entering the next generation is equal to the ratio of its fitness value to the sum of the fitness values of the individuals in the population as a whole.

Further, two individuals are selected as parents, partial structures of the two parents are replaced, and recombination is carried out to generate new individuals, and the process is called crossing in genetic algorithm. In the initial stage of crossing, in order to prevent the excessive kinds of groups eliminated by the roulette method from falling into local optimum, the crossing probability is high, the algorithm convergence is accelerated in the later stage, and the crossing probability can be properly reduced. The relationship between the cross probability and the number of iterations is expressed as:

in the formula, P_cThe cross probability is characterized and gen the number of iterations.

Further, the gene values at certain loci in the individual's chromosomal code string are replaced with other alleles at that locus to form new individuals, a process referred to as mutation in the genetic algorithm. The relationship between the mutation probability and the iteration number is expressed as:

in the formula, P_mThe variation probability is characterized and gen the number of iterations.

Further, a population with the optimal fitness is output. If the iteration number is equal to 200, outputting the individual with the maximum fitness in the evolution process, and terminating the program. And decoding the population with the optimal fitness to obtain a fourth image.

In the technical scheme, the K clustering center is set as the number of seed points. The traditional KMeans method adopts random initialization of the clustering center, but the selection of the clustering center can influence the clustering result of the image, so that the clustering center is set as the centroid of each region, the defects of the traditional KMeans can be eliminated, and the algorithm speed can be improved.

S2043: and performing image recognition on the fourth image, and determining a plurality of target objects and pose information of target areas where the target objects are located.

Specifically, the fourth image is subjected to image recognition, the target object is determined to be a pedestrian, a vehicle or a static target, and the pose information of the target object is determined.

Specifically, the Canny operator is used for enhancing the edge of the fourth image and performing image morphological filtering, so that a region of interest (ROI) of the target is generated quickly, and the convergence speed is accelerated. And carrying out image recognition on the target object by using the trained model, and outputting a recognition result.

Because the radar has the characteristic that the electromagnetic wave emitted in one direction can only receive the echo signal in the direction, the relative distance and angle of the target object relative to the self-vehicle and the speed and coordinate information of the target object, namely the pose information of the target object, can be positioned and acquired. The pose information of the target object may include position information and pose information of the target object, the position information including a relative distance of the target object with respect to the host vehicle, and the pose information including speed information and direction information of the target object with respect to the host vehicle. According to the relative distance information of the target object relative to the vehicle and the speed information of the target object, the time information of the target object reaching the door of the vehicle can be obtained.

Due to the fact that the road environment is complex, a moving target is possibly shielded and the like, the phenomenon that the target is momentarily lost can be caused, and therefore after the millimeter wave radar and the camera complete sampling work, continuous tracking and positioning of the target are needed. The method and the device adopt unscented Kalman filtering to track the target object.

S205: and performing anti-collision early warning according to the pose information of the plurality of target objects.

In a possible implementation manner, the performing anti-collision early warning according to the pose information of the plurality of targets includes:

and when the position with the target object is in the target area, judging that the time for the target object to reach the door of the vehicle is not more than a safety threshold value according to the pose information of the target object, and performing primary anti-collision early warning.

Or the position with the target object is in the target area, the time of the target object reaching the door of the vehicle is judged to be larger than a safety threshold value according to the pose information of the target object, and secondary anti-collision early warning is carried out.

Table 1 is an early warning mechanism table, and table 2 is an early warning level division table. Specifically, referring to fig. 1, table 1 and table 2, since the walking speed of the pedestrian generally does not exceed 3m/s, a + b + 20m is defined as a pedestrian warning area, c + 30m is defined as a traffic warning area. The safety time threshold is set to be Ts, defined as the sum of the human reaction time and the time for opening the vehicle door, the human reaction time is 0.5s, the time for opening the vehicle door is 2.5s, and therefore Ts is set to be 3.

The execution of the early warning instruction comprises a buzzer, a control vehicle door lock and different color light alarms.

The light warning indicator, LED lamp promptly installs additional in the bumper department of tail side about the car, according to the early warning instruction of difference, makes it flicker with certain frequency to remind the outer personnel of car and passing vehicle. Meanwhile, dividing the frequency of the buzzer into three categories according to the danger level, and if the three-level danger indicates that a static object is identified, setting the frequency of the buzzer as 400 Hz; if the second-level danger represents that the pedestrian or the automobile with the possible danger is identified, setting the frequency of the buzzer to be 600Hz, and simultaneously warning the pedestrian or driving by the LED lamp; in the case of a first-order danger indicating an automobile with a short distance or a high traveling speed, the possibility of an accident caused by opening the door is high, and therefore, the buzzer frequency is set to 800Hz, the LED lamp is blinked, and the door lock is opened.

And when the target object is judged to be a pedestrian, judging whether the transverse distance x is not more than 1.3 times of the total opening time of the doors of the automobile, if so, judging whether the time for the target pedestrian to reach the doors of the automobile is not more than a safety time threshold Ts, and if so, sending a secondary early warning instruction.

And when the target object is judged to be a vehicle, judging whether the transverse distance x is not more than 1.3 times of the total opening of the doors of the vehicle, if so, judging whether the longitudinal distance k of the target vehicle is in a danger alarm area or a driving early warning area.

When the target vehicle is judged to be in a danger alarm area, judging whether the time for the target vehicle to reach the door of the vehicle is not more than a safety time threshold value Ts, if so, sending a primary early warning instruction; if not, a secondary early warning instruction is sent out.

When the target vehicle is judged to be in the driving early warning area, judging whether the time of the target vehicle reaching the door of the vehicle is not more than a safety time threshold value Ts, if so, sending a primary early warning instruction; if not, a three-level early warning instruction is sent.

When the target object is judged to be a static target, only whether the transverse distance of the static target is not more than 1.3 times of the total opening of the doors of the bicycle is needed to be judged, and if yes, a three-stage early warning instruction is sent.

TABLE 1

TABLE 2

Fig. 3 is a schematic diagram of a system for early warning control provided in an embodiment of the present application. As shown in fig. 3, an early warning collision avoidance system includes:

the radar signal acquiring module 301 is configured to acquire a first radar echo signal of the target area at the current time.

A camera data obtaining module 302, configured to obtain a first image of the target area at the current time.

The system control module 303 is configured to determine, according to the first radar echo signal and the first image, first coordinates of a plurality of targets in a pixel coordinate system, where the first coordinates correspond to: performing wavelet transformation on the first radar echo signal to obtain a radar wavelet; performing wavelet inverse transformation operation on the radar wavelet to reconstruct a signal to obtain a second radar signal; determining first coordinates of a plurality of targets in a pixel coordinate system according to the second radar signals and the first image; segmenting the first image by using a region growing method to obtain a second image; wherein, the seed point of the region growing method is the first coordinate; performing image recognition on the first image by using a genetic algorithm, and determining pose information of the plurality of target objects in the target area; wherein the input to the genetic algorithm comprises the second image. In one possible embodiment of the method according to the invention,

the first radar echo signal is a beat signal, and the beat signal comprises a signal obtained by mixing a radar echo signal and a radar transmitting signal;

the determining, according to the first radar echo signal and the first image, first coordinates of a plurality of targets in a pixel coordinate system specifically includes:

acquiring a first radar beat signal of a target area at the current moment, and performing wavelet transformation on the beat signal to obtain a beat wavelet;

selecting a wavelet function to perform multi-scale decomposition on a first wavelet coefficient of the beat wavelet, and performing threshold quantization processing on each wavelet coefficient obtained by decomposition to obtain a second wavelet coefficient;

performing inverse wavelet transform operation on the second wavelet coefficient to reconstruct a signal to obtain a second radar beat signal;

and determining first coordinates of a plurality of targets in a pixel coordinate system according to the second radar beat signals and the first image.

In a possible embodiment, the radar emission signal is an emission signal emitted by a millimeter wave radar mounted on the underside of the rear view mirror.

In one possible embodiment of the method according to the invention,

the image recognition of the first image by using a genetic algorithm, and the determination of the pose information of the plurality of target objects in the target area comprises:

graying the first image to obtain a third image;

taking the second image and the third image as input of a genetic algorithm, and segmenting the third image by utilizing the genetic algorithm to obtain a fourth image; the fitness function of the genetic algorithm is determined through a Kmeans clustering algorithm;

and performing image recognition on the fourth image, and determining a plurality of target objects and pose information of target areas where the target objects are located.

And the early warning execution module 304 is configured to perform anti-collision early warning according to the pose information of the plurality of targets.

In one possible implementation, the pose information of the plurality of targets includes position information and attitude information of the targets, the position information includes relative distances of the targets relative to the host vehicle, and the attitude information includes speed information of the targets relative to the host vehicle and direction information of the targets.

The performing anti-collision early warning according to the pose information of the plurality of target objects comprises:

and when the target object is judged to be a pedestrian, judging whether the transverse distance x is not more than 1.3 times of the total opening time of the doors of the automobile, if so, judging whether the time for the target pedestrian to reach the doors of the automobile is not more than a safety time threshold Ts, and if so, sending a secondary early warning instruction.

And when the target object is judged to be a vehicle, judging whether the transverse distance x is not more than 1.3 times of the total opening of the doors of the vehicle, if so, judging whether the longitudinal distance k of the target vehicle is in a danger alarm area or a driving early warning area.

When the target vehicle is judged to be in a danger alarm area, judging whether the time for the target vehicle to reach the door of the vehicle is not more than a safety time threshold value Ts, if so, sending a primary early warning instruction; if not, a secondary early warning instruction is sent out.

When the target vehicle is judged to be in the driving early warning area, judging whether the time of the target vehicle reaching the door of the vehicle is not more than a safety time threshold value Ts, if so, sending a primary early warning instruction; if not, a three-level early warning instruction is sent.

When the target object is judged to be a static target, only whether the transverse distance of the static target is not more than 1.3 times of the total opening of the doors of the bicycle is needed to be judged, and if yes, a three-stage early warning instruction is sent.

It should be noted that, in an emergency, the system may have misjudgment, so that the actuator has misoperations, which affects the door opening process and delays the time for people in the vehicle to get off and escape. To prevent this, the door actuating module is also provided with an emergency switch. The emergency switch can control the control module to be turned on or off. The emergency switch is in a normally closed state when working normally, and the control module can supply power normally; when an emergency occurs, personnel in the vehicle triggers the emergency switch to enable the emergency switch to be in a disconnected state, and the power supply of the control module is cut off to enable the control module to be invalid. The emergency switch can prevent the control module from being operated by mistake to delay the escape time of people in the vehicle when an emergency occurs.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are merely exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.