阅读说明：本技术 行车盲区透视视频的生成方法及行车盲区视野透视系统 (Driving blind area perspective video generation method and driving blind area view perspective system ) 是由 徐孟南 邱乙耕 杨朝雄 于 2019-08-27 设计创作，主要内容包括：本发明公开了一种行车盲区透视视频的生成方法及行车盲区视野透视系统,生成方法实现盲区透视化显示的原理是基于计算机视觉的多视图几何,多个相机间通过射影变换的关系确定位姿,盲区通过固定相机获取图像,再将该图像传输到移动的用户相机上,利用相机间的视角变换关系,对盲区图像进行变换,使其在用户相机上以正确的位置和视角进行融合,从而得到辅助驾驶员驾驶车辆的行车盲区透视视频,进而能够大幅减少交通事故的发生。(The invention discloses a method for generating driving blind area perspective videos and a driving blind area visual field perspective system, wherein the principle of the generation method for realizing the blind area perspective display is based on the multi-view geometry of computer vision, the position and the posture of a plurality of cameras are determined through the projective transformation relationship, the blind area obtains images through a fixed camera, then the images are transmitted to a mobile user camera, the blind area images are transformed by utilizing the visual angle transformation relationship among the cameras, so that the blind area images are fused at the correct position and visual angle on the user camera, the driving blind area perspective videos for assisting a driver in driving a vehicle are obtained, and the occurrence of traffic accidents can be greatly reduced.)

1, method for generating driving blind area perspective video, comprising:

shooting a driving video stream by a vehicle-mounted user camera;

shooting a blind area video stream and an image containing an identification plate by a fixed camera arranged in a shooting blind area of the user camera, wherein the identification plate is arranged in a public view field of the user camera and the fixed camera;

determining an initial value image according to the image containing the identification plate;

aiming at each frame of driving image and blind area image corresponding to the same moment in the driving video stream and the blind area video stream, executing the following steps:

matching characteristic points of the driving image and the initial value image to obtain a homography matrix corresponding to the driving image;

carrying out image conversion on the blind area image based on the homography matrix;

carrying out image splicing and fusion on the driving image and the image-converted blind area image to obtain a driving blind area perspective image at the moment; and (4) splicing the driving blind area perspective images at all times into driving blind area perspective videos.

2. The generation method according to claim 1, wherein a plurality of signboard boards are provided in the public view, the dead zone is provided with an th fixed camera and a second fixed camera,

determining an initial value image according to the image containing the identification plate, wherein the method comprises the following steps:

and for each identification plate, determining an initial value image corresponding to the identification plate according to the images containing the identification plate, which are shot by the th fixed camera and the second fixed camera.

3. The method of claim 2, wherein determining an initial image from the image containing the identification plate comprises:

and for each identification plate, splicing and fusing images which are shot by the th fixed camera and the second fixed camera and contain the identification plate to determine an initial value image corresponding to the identification plate.

4. The generation method according to claim 3,

for each frame of driving image, th blind area image and second blind area image corresponding to moments in the driving video stream, the th blind area video stream shot by the th fixed camera and the second blind area video stream shot by the second fixed camera, the following steps are executed:

respectively matching the driving image with the initial value images corresponding to the identification plates by characteristic points to obtain a homography matrix corresponding to each identification plate;

determining homography matrixes according to the homography matrixes corresponding to all the identification plates;

splicing and fusing the th blind area image and the second blind area image to determine a blind area image;

performing image conversion on the determined blind area image based on the determined homography matrix;

carrying out image splicing and fusion on the driving image and the image-converted blind area image to obtain a driving blind area perspective image at the moment; and (4) splicing the driving blind area perspective images at all times into driving blind area perspective videos.

5. The method according to claim 4, wherein determining homography matrices according to the homography matrices corresponding to all the identification boards comprises:

and solving the mean value of the homography matrixes corresponding to all the identification plates, and taking the mean value result as the determined homography matrix.

6, kind of driving blind area field of vision perspective system, characterized by, include:

an onboard user camera configured to capture a driving video stream;

a fixed camera disposed in a dead zone of the user camera and configured to capture a dead zone video stream and an image containing an identification plate;

an identification plate disposed in a common field of view of the user camera and the fixed camera; and

a controller configured to determine an initial value image from the image containing the identification plate, and to execute the following steps for each frame of the driving image and the blind area image corresponding to the same time in the driving video stream and the blind area video stream:

matching characteristic points of the driving image and the initial value image to obtain a homography matrix corresponding to the driving image;

carrying out image conversion on the blind area image based on the homography matrix;

carrying out image splicing and fusion on the driving image and the image-converted blind area image to obtain a driving blind area perspective image at the moment; and (4) splicing the driving blind area perspective images at all times into driving blind area perspective videos.

7. The system of claim 6, comprising:

a plurality of sign boards disposed in the public view; and

an th fixed camera and a second fixed camera which are arranged in the dead zone,

the controller is specifically configured to determine, for each identification plate, an initial value image corresponding to the identification plate according to the images containing the identification plate, which are captured by the th fixed camera and the second fixed camera.

8. The vehicle blind spot vision perspective system of claim 7, wherein the controller is specifically configured to:

and for each identification plate, splicing and fusing images which are shot by the th fixed camera and the second fixed camera and contain the identification plate to determine an initial value image corresponding to the identification plate.

9. The system of claim 8, wherein the controller is specifically configured to perform the following steps for each of the driving image, the blind area image and the second blind area image corresponding to time instant of the driving video stream, the blind area video stream captured by the th fixed camera and the second blind area video stream captured by the second fixed camera:

respectively matching the driving image with the initial value images corresponding to the identification plates by characteristic points to obtain a homography matrix corresponding to each identification plate;

determining homography matrixes according to the homography matrixes corresponding to all the identification plates;

splicing and fusing the th blind area image and the second blind area image to determine a blind area image;

performing image conversion on the determined blind area image based on the determined homography matrix;

carrying out image splicing and fusion on the driving image and the image-converted blind area image to obtain a driving blind area perspective image at the moment; and (4) splicing the driving blind area perspective images at all times into driving blind area perspective videos.

10. The vehicle blind spot vision perspective system of claim 9, wherein the controller is specifically configured to: and solving the mean value of the homography matrixes corresponding to all the identification plates, and taking the mean value result as the determined homography matrix.

Technical Field

The invention relates to the technical field of vehicle driving safety, in particular to a generation method of driving blind area perspective videos and a driving blind area view perspective system.

Background

In the driving process of the vehicle, except the range and the visual range of the automobile reflector device, most of the positions are difficult to see, the automobile reflector device belongs to a driving blind area of a driver, and driving danger easily occurs in the blind area. Blind areas are dead angles and unconscious places of sight for the driver. Generally, the driving blind area outside the vehicle is a driving blind area caused by blocking the sight of a driver by buildings, plants, vehicles and the like beside the road due to the shape of the road when the vehicle runs on the road.

As the development of the 5G technology and the 5G era are coming, regional real-time mass data transmission becomes possible, and driving blind area visual field perspective systems are urgently needed in the technical background.

Disclosure of Invention

The technical problem to be solved by the invention is as follows: in the prior art, traffic accidents are caused by blind areas in the driving process of vehicles.

The invention provides driving blind area visual field perspective system, which is designed to be used at the crossing of the city or the traffic area blocked by the building, and improves the safety of the city traffic by making the visual field blind area generated by the blocking in the view angle of the vehicle user perspective.

The technical scheme of the invention is as follows:

A method for generating a driving blind area perspective video, which comprises the following steps:

shooting a driving video stream by a vehicle-mounted user camera;

shooting a blind area video stream and an image containing an identification plate by a fixed camera arranged in a shooting blind area of the user camera, wherein the identification plate is arranged in a public view field of the user camera and the fixed camera;

determining an initial value image according to the image containing the identification plate;

aiming at each frame of driving image and blind area image corresponding to the same moment in the driving video stream and the blind area video stream, executing the following steps:

matching characteristic points of the driving image and the initial value image to obtain a homography matrix corresponding to the driving image;

carrying out image conversion on the blind area image based on the homography matrix;

carrying out image splicing and fusion on the driving image and the image-converted blind area image to obtain a driving blind area perspective image at the moment;

and (4) splicing the driving blind area perspective images at all times into driving blind area perspective videos.

Preferably, a plurality of identification plates are arranged in the public view, the dead zone is provided with an th fixed camera and a second fixed camera,

and determining an initial value image according to the image containing the identification plate, wherein for each identification plate, the initial value image corresponding to the identification plate is determined according to the images containing the identification plate, which are shot by the th fixed camera and the second fixed camera.

Preferably, determining an initial value image according to the image containing the identification plate includes:

and for each identification plate, splicing and fusing images which are shot by the th fixed camera and the second fixed camera and contain the identification plate to determine an initial value image corresponding to the identification plate.

Preferably, for each frame of driving image, th blind area image and second blind area image corresponding to time in the driving video stream, the th blind area video stream shot by the th fixed camera and the second blind area video stream shot by the second fixed camera, the following steps are executed:

respectively matching the driving image with the initial value images corresponding to the identification plates by characteristic points to obtain a homography matrix corresponding to each identification plate;

determining homography matrixes according to the homography matrixes corresponding to all the identification plates;

splicing and fusing the th blind area image and the second blind area image to determine a blind area image;

performing image conversion on the determined blind area image based on the determined homography matrix;

carrying out image splicing and fusion on the driving image and the image-converted blind area image to obtain a driving blind area perspective image at the moment;

and (4) splicing the driving blind area perspective images at all times into driving blind area perspective videos.

Preferably, homography matrices are determined according to the homography matrices corresponding to all the identification plates, including:

and solving the mean value of the homography matrixes corresponding to all the identification plates, and taking the mean value result as the determined homography matrix.

A blind area vision perspective system for vehicle driving, comprising:

an onboard user camera configured to capture a driving video stream;

a fixed camera disposed in a dead zone of the user camera and configured to capture a dead zone video stream and an image containing an identification plate;

an identification plate disposed in a common field of view of the user camera and the fixed camera; and

a controller configured to determine an initial value image from the image containing the identification plate, and to execute the following steps for each frame of the driving image and the blind area image corresponding to the same time in the driving video stream and the blind area video stream:

matching characteristic points of the driving image and the initial value image to obtain a homography matrix corresponding to the driving image;

carrying out image conversion on the blind area image based on the homography matrix;

carrying out image splicing and fusion on the driving image and the image-converted blind area image to obtain a driving blind area perspective image at the moment;

and (4) splicing the driving blind area perspective images at all times into driving blind area perspective videos.

Preferably, the driving blind area visual field perspective system includes:

a plurality of sign boards disposed in the public view; and

an th fixed camera and a second fixed camera which are arranged in the dead zone,

the controller is specifically configured to determine, for each identification plate, an initial value image corresponding to the identification plate according to the images containing the identification plate, which are captured by the th fixed camera and the second fixed camera.

Preferably, the controller is specifically configured to:

and for each identification plate, splicing and fusing images which are shot by the th fixed camera and the second fixed camera and contain the identification plate to determine an initial value image corresponding to the identification plate.

Preferably, the controller is specifically configured to execute the following steps for each of the driving image, the th blind area image and the second blind area image corresponding to moments in the driving video stream, the th blind area video stream captured by the th fixed camera and the second blind area video stream captured by the second fixed camera:

respectively matching the driving image with the initial value images corresponding to the identification plates by characteristic points to obtain a homography matrix corresponding to each identification plate;

determining homography matrixes according to the homography matrixes corresponding to all the identification plates;

splicing and fusing the th blind area image and the second blind area image to determine a blind area image;

performing image conversion on the determined blind area image based on the determined homography matrix;

carrying out image splicing and fusion on the driving image and the image-converted blind area image to obtain a driving blind area perspective image at the moment;

and (4) splicing the driving blind area perspective images at all times into driving blind area perspective videos.

Preferably, the controller is specifically configured to: and solving the mean value of the homography matrixes corresponding to all the identification plates, and taking the mean value result as the determined homography matrix.

Compared with the prior art, or more embodiments in the above scheme can have the following advantages or beneficial effects:

the principle of the method for generating the driving blind area perspective video for realizing the blind area perspective display is that based on the multi-view geometry of computer vision, the pose is determined by the relation of projective transformation among a plurality of cameras, the blind area obtains images through a fixed camera, then the images are transmitted to a mobile user camera, and the blind area images are transformed by utilizing the visual angle transformation relation among the cameras, so that the blind area images are fused at the correct position and visual angle on the user camera, thereby obtaining the driving blind area perspective video for assisting a driver in driving a vehicle, and further greatly reducing the occurrence of traffic accidents.

Drawings

The scope of the present disclosure may be better understood by reading the following detailed description of exemplary embodiments in conjunction with the accompanying drawings. Wherein the included drawings are:

FIG. 1 shows a schematic view of a blind spot field of view perspective system for driving vehicles of embodiment of the present invention;

fig. 2 is a schematic flow chart illustrating a method for generating a driving blind area perspective video in according to an embodiment of the present invention;

FIG. 3 shows a schematic diagram of a weighted smoothing algorithm;

FIG. 4 is a schematic view of a blind spot vision perspective system for driving vehicles according to a second embodiment of the present invention;

fig. 5 is a flowchart illustrating a method for generating a driving blind area perspective video according to a second embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the following will describe in detail an implementation method of the present invention with reference to the accompanying drawings and embodiments, so that how to apply technical means to solve the technical problems and achieve the technical effects can be fully understood and implemented.

In the prior art, for a user camera with vision dead zones on one side or two sides, traffic accidents often occur due to the existence of the dead zones in the driving process of a vehicle, regional real-time mass data transmission becomes possible due to the development of the 5G technology and the coming of the 5G era, and driving dead zone vision perspective systems are urgently needed in the technical background.

In order to solve the above technical problems, the embodiment of the present invention provides kinds of driving blind area view perspective systems.