阅读说明：本技术 一种基于深度学习的适用于道路能见度的监测方法与系统 (Monitoring method and system suitable for road visibility based on deep learning ) 是由 冯海霞 白博文 王�琦 咸化彩 李炜 张丽彩 孟祥鲁 王帅 于 2019-11-22 设计创作，主要内容包括：目前,对道路能见度的监测的各类方法很难实现全道路监测,因此本发明提出一种基于深度学习的适用于道路能见度的监测方法与系统,方法步骤包括：数据收集,利用深度学习方法构建能见度模型,构建道路所有摄影头数据的能见度监测系统,将能见度模型推广至系统中所有摄像头图像数据的能见度判定,能见度显示、分级预警并通过显示屏预警提示,该系统包括道路摄像头,能见度模型,数据处理中心和预警信息展示器。本发明发挥深度学习算法的优势,利用现有的道路摄影头的视频图像、能见度等数据,低成本地实现了对道路全程能见度的监控。(At present, various methods for monitoring road visibility are difficult to realize all-road monitoring, so the invention provides a monitoring method and a system based on deep learning and suitable for road visibility, and the method comprises the following steps: the method comprises the steps of collecting data, constructing a visibility model by using a deep learning method, constructing a visibility monitoring system for all road camera data, popularizing the visibility model to visibility judgment of all camera image data in the system, displaying visibility, carrying out grading early warning and carrying out early warning prompt through a display screen. The invention exerts the advantages of the deep learning algorithm, utilizes the data of the video image, the visibility and the like of the existing road camera, and realizes the monitoring of the whole-course visibility of the road with low cost.)

1. A monitoring method suitable for road visibility based on deep learning is characterized by comprising the following steps:

s1, data collection: selecting different typical road conditions and different weather conditions on a monitored road, capturing images by using a camera, and acquiring corresponding weather data and environment condition data;

s2, constructing a visibility model by using a deep learning method:

data preprocessing, namely acquiring video data of a camera on a monitored road section, preprocessing the video data, extracting an image every t seconds, and forming a dynamic time sequence image by the extracted camera image along with time, wherein t is more than or equal to 1 and less than or equal to 10;

calculating the contrast, edge characteristics and POLC indexes of the image;

the data correspond to each other one by one, and deep learning is carried out;

training a network by using the training data pair to construct a visibility model, optimizing the network and constructing the visibility model;

s3, constructing a visibility monitoring system for all road camera data:

establishing a database for all cameras on a road by using a GIS technology, and constructing a monitoring system for road visibility, so as to realize the processing and map display of all camera images along the road;

s4, popularizing the visibility model to the visibility judgment of all camera image data in the system;

and S5, visibility display, grading early warning and early warning prompt through a display screen.

2. The method of claim 1, wherein the step of calculating the contrast ratio of step S2 is:

wherein C is_x,xi(f) The contrast value of a pixel point corresponding to the row direction, f (x) is the gray value of the corresponding pixel point, the right pixel value of the adjacent position of the pixel, G is the maximum gray value of the image, and min { f (x), f (xi) } is the minimum value of the adjacent pixel;

in the same way, the column contrast can be calculated;

the calculation steps of the edge features are as follows:

the gradient value calculation is carried out on gray level change in a certain field by inspecting each pixel point of the image, the edge detection operator is used for calculating the local gradient value by utilizing the change rule of the first-order or second-order directional derivative near the edge, the 3x3sobel operator (first-order) is adopted, and the calculated local gradient value can be used as the gradient amplitude value

Wherein F (G)_mean) Is the global average gradient of the image, M, N are the image resolution size, G_s(i, j) is the gradient value of the corresponding pixel, the edge response is very strong when the visibility is high, otherwise, the edge response is very weak;

the calculation steps of the POLC index are as follows:

dividing an image into m multiplied by n areas, calculating the contrast of each area and the L channel value of an LAB color space of the image as brightness, and finally obtaining the POLC by taking the average value of each area as a quotient

Where Ω (x) is the size of the divided region, M × N is the total number of windows, M ≦ M, N ≦ N, C (y) is the contrast, and L (y) is the lightness.

3. The method of claim 1, wherein said deep learning comprises the steps of:

selecting a depth residual error network ResNet model, and selecting ResNet50 as a network design;

network architecture adjustment and determination: except for the image data of the camera, the visibility data is fitted by an auxiliary network through weather data, environmental condition data, calculated contrast, edge characteristics and a POLC index, so that the last line is removed, three layers of lines are added again, the auxiliary information is fused with the image information extracted by the layer 4 in front of the network when the first layer of line is used, the first layer of line is fitted, and the neuron of the last layer of line is 1 to represent the visibility;

and (3) network training:

s01), dividing the training set into a training set validation set according to the proportion of 8:2, subdividing the divided training set, and carrying out 4-fold cross validation;

s02), data amplification is carried out on the image, and then the image is converted into a 4-D sensor for normalization processing. The required assist features are then calculated. Performing code conversion and normalization on the auxiliary weather data and the environmental condition data;

s03), normalizing the visibility data to be used as label data;

s04), setting related hyper-parameters including maximum iteration times, a learning rate, an L2 regular coefficient and a learning rate attenuation coefficient, and using a learning rate hot start + cosine annealing learning rate strategy;

s05), using Adam as an optimizer, and adding L2 regular as a penalty term for preventing network overfitting;

s06), using MSE Loss as a Loss function to supervise network learning;

s07), and carrying out iterative training, wherein each epoch is trained, the precision calculation is carried out by using the verification set and the test set, and the best epoch in the training process is saved.

4. A system for monitoring road visibility based on deep learning, comprising:

the road camera is used for collecting data;

a visibility model;

the data processing center is respectively in wireless transmission connection with the road camera and the visibility model and is used for constructing a visibility monitoring system of all camera data of a road and popularizing the visibility model to visibility judgment of all camera image data in the system;

and the early warning information displayer is in wireless transmission connection with the data processing center, is used for visibility display and grading early warning, and gives an early warning prompt through a display screen.

Technical Field

The invention relates to the field of traffic safety, in particular to the field of a monitoring method and a monitoring system based on deep learning and suitable for road visibility.

Background

Along with the rapid increase of the number of motor vehicles and the improvement of the driving speed of the vehicles, higher requirements are put forward on the safety guarantee of transportation, and the influence of various natural disasters on the transportation safety is more and more emphasized, wherein the meteorological disasters seriously influence the transportation safety problem, particularly on a highway; according to statistics, in the influence factors of traffic accidents, the proportion of the traffic accidents caused by the road condition factors caused by the weather condition factors accounts for about 45%, the visibility is an important index for reflecting the weather conditions, when the weather processes such as rainfall, snow, haze and sand storm occur, the visibility is reduced, the reduced visibility can bring great hidden dangers to traffic safety, and the real-time monitoring and forecasting of the visibility have important significance for reducing and preventing the traffic accidents, formulating traffic safety measures in real time and guaranteeing the traffic safety.

Currently, monitoring can be roughly divided into three categories: one type of visual inspection, one type of visibility meter based monitoring and the other type of image based monitoring. The visibility observation by visual observation method mainly takes manual visual observation; the visibility meter is divided into three types of instruments for measuring atmospheric transmittance, forward scattering and atmospheric extinction coefficient; visibility monitoring based on image processing mainly obtains visibility values through image analysis processing. The normative and objectivity of visibility by visual measurement are relatively poor; visibility meters are the most common visibility monitoring methods at present, but the price of the visibility meters is high; due to different conditions along roads, especially in mountainous areas, the visibility monitoring method based on image processing at present is difficult to adapt to visibility monitoring of the whole road. Due to the characteristics of roads, such as long length and narrow width, the conditions of passing areas are complex, and particularly for roads in mountainous areas, the whole-course monitoring of road visibility is almost impossible by a visual measurement method; the arrangement of the visibility meter has to be small in distance, but the price is high and the cost is too high; the method becomes the most economical and rapid visibility monitoring method based on image processing due to rapid reduction of the cost of a camera, and becomes a priority scheme for predicting the visibility of a road due to a plurality of cameras on the road, but the visibility monitoring method based on image processing is difficult to adapt to the visibility monitoring of the whole road under complex road conditions at present; the rapid development of the deep learning method provides a new method and thought for visibility prediction based on images.

Disclosure of Invention

Because the visibility meter is high in cost and the camera is low in cost, most of the visibility monitoring methods for roads at present are image processing-based methods, but the method is difficult to adapt to the whole road with different road conditions. If the terrain of a region is complex, particularly in mountainous regions, it is very difficult to realize the whole-course monitoring of the road visibility. To solve the above problems, the present invention proposes the following solutions.

A monitoring method suitable for road visibility based on deep learning comprises the following steps:

s1, data collection: selecting different typical road conditions and different weather conditions on a monitored road, capturing images by using a camera, and acquiring corresponding weather data and environment condition data;

s2, constructing a visibility model by using a deep learning method:

data preprocessing, namely acquiring video data of a camera on a monitored road section, preprocessing the video data, extracting an image every t seconds, and forming a dynamic time sequence image by the extracted camera image along with time, wherein t is more than or equal to 1 and less than or equal to 10;

calculating the contrast, edge characteristics and POLC indexes of the image;

the data correspond to each other one by one, and deep learning is carried out;

training a network by using the training data pair to construct a visibility model, optimizing the network and constructing the visibility model;

s3, constructing a visibility monitoring system for all road camera data:

establishing a database for all cameras on a road by using a GIS technology, and constructing a monitoring system for road visibility, so as to realize the processing and map display of all camera images along the road;

s4, popularizing the visibility model to the visibility judgment of all camera image data in the system;

and S5, visibility display, grading early warning and early warning prompt through a display screen.

In step S2, the contrast ratio calculation step is:

wherein C is_x,xi(f) The contrast value of a pixel point corresponding to the row direction, f (x) is the gray value of the corresponding pixel point, the right pixel value of the adjacent position of the pixel, G is the maximum gray value of the image, and min { f (x), f (xi) } is the minimum value of the adjacent pixel;

in the same way, the column contrast can be calculated;

the calculation steps of the edge features are as follows:

calculating gradient value by inspecting each pixel point of the image according to gray level change in a certain field, calculating local gradient value by using an edge detection operator according to the change rule of a first-order or second-order directional derivative near the edge, and calculating the local gradient value by using a 3x3sobel operator (first-order), wherein the calculated local gradient value can be used as a gradient amplitude value

Wherein F (G)_mean) Is the global average gradient of the image, M, N are the image resolution size, G_s(i, j) is the gradient value of the corresponding pixel, the edge response is very strong when the visibility is high, otherwise, the edge response is very weak;

the calculation steps of the POLC index are as follows:

dividing an image into m multiplied by n areas, calculating the contrast of each area and the L channel value of an LAB color space of the image as brightness, and finally obtaining the POLC by taking the average value of each area as a quotient

Where Ω (x) is the size of the divided region, M × N is the total number of windows, M ≦ M, N ≦ N, C (y) is the contrast, and L (y) is the lightness.

The deep learning includes the steps of:

selecting a depth residual error network ResNet model, and selecting ResNet50 as a network design;

network architecture adjustment and determination: except for the image data of the camera, the visibility data is fitted by an auxiliary network through weather data, environmental condition data, calculated contrast, edge characteristics and a POLC index, so that the last line is removed, three layers of lines are added again, the auxiliary information is fused with the image information extracted by the layer 4 in front of the network when the first layer of line is used, the first layer of line is fitted, and the neuron of the last layer of line is 1 to represent the visibility;

and (3) network training:

s01), dividing the training set into a training set validation set according to the proportion of 8:2, subdividing the divided training set, and carrying out 4-fold cross validation;

s02), data amplification is carried out on the image, and then the image is converted into a 4-D sensor for normalization processing. The required assist features are then calculated. Performing code conversion and normalization on the auxiliary weather data and the environmental condition data;

s03), normalizing the visibility data to be used as label data;

s04), setting related hyper-parameters including maximum iteration times, a learning rate, an L2 regular coefficient and a learning rate attenuation coefficient, and using a learning rate hot start + cosine annealing learning rate strategy;

s05), using Adam as an optimizer, and adding L2 regular as a penalty term for preventing network overfitting;

s06), using MSE Loss as a Loss function to supervise network learning;

s07), and carrying out iterative training, wherein each epoch is trained, the precision calculation is carried out by using the verification set and the test set, and the best epoch in the training process is saved.

A system for monitoring road visibility based on deep learning, comprising:

the road camera is used for collecting data;

a visibility model;

the data processing center is respectively in wireless transmission connection with the road camera and the visibility model and is used for constructing a visibility monitoring system of all camera data of a road and popularizing the visibility model to visibility judgment of all camera image data in the system;

and the early warning information displayer is in wireless transmission connection with the data processing center, is used for visibility display and grading early warning, and gives an early warning prompt through a display screen.

The invention has the beneficial effects that:

aiming at the characteristics of long length and narrow width of the road and the defects of the current visibility monitoring method, the invention can provide visibility monitoring and early warning of different road sections and roads in the whole process;

the method makes full use of the existing free data such as meteorological data, environmental data and the like, not only can provide a data source for the visibility calculation of the road section where each camera is located, but also can provide accurate traffic safety information and traffic safety early warning measures for different road sections;

the invention only utilizes the sample data of the monitored road section, exerts the advantages of the deep learning algorithm, utilizes the video image, visibility and other data of the existing road camera to realize the monitoring of the whole-course visibility of the road, has low system cost, changes the current situations that a large amount of instruments and equipment are required to be added and the manufacturing cost is high in the visibility monitoring of the road at present, and has remarkable economic benefit.

Drawings

FIG. 1 is a schematic diagram of the system of the present invention;

FIG. 2 is a flow chart of the method of the present invention;

FIG. 3 is a flow chart of a visibility model using a deep learning method;

FIG. 4 is a diagram of residual error units in a ResNet network;

FIG. 5 is a diagram of a ResNet50 network architecture;

fig. 6 is a flow chart of network architecture adjustment and determination.

Detailed Description

A monitoring system for road visibility based on deep learning as shown in fig. 1, comprising:

the road camera is used for collecting data;

a visibility model;

the data processing center is respectively in wireless transmission connection with the road camera and the visibility model and is used for constructing a visibility monitoring system of all camera data of a road and popularizing the visibility model to visibility judgment of all camera image data in the system;

and the early warning information displayer is in wireless transmission connection with the data processing center, is used for visibility display and grading early warning, and gives an early warning prompt through a display screen.

As shown in fig. 2, the implementation method of the system includes the following steps:

s1, data collection: selecting different typical road conditions and different weather conditions on a monitored road, capturing images by using a camera, and acquiring corresponding weather data and environment condition data;

s2, constructing a visibility model by using a deep learning method, wherein the flow schematic diagram is as shown in FIG. 3:

data preprocessing, namely acquiring video data of a camera on a monitored road section, preprocessing the video data, extracting an image every t seconds, and forming a dynamic time sequence image by the extracted camera image along with time, wherein t is more than or equal to 1 and less than or equal to 10;

calculating the contrast of the image:

wherein C is_x,xi(f) Is the contrast value of the pixel point corresponding to the row direction, f (x) is the gray value of the corresponding pixel point, is the right pixel value of the adjacent position of the pixel, G is the maximum gray value of the imageMin { f (x), f (xi) } is the neighboring pixel minimum;

in the same way, the column contrast can be calculated;

calculating edge features:

calculating gradient value by inspecting each pixel point of the image according to gray level change in a certain field, calculating local gradient value by using an edge detection operator according to the change rule of a first-order or second-order directional derivative near the edge, and calculating the local gradient value by using a 3x3sobel operator (first-order), wherein the calculated local gradient value can be used as a gradient amplitude value

Wherein F (G)_mean) Is the global average gradient of the image, M, N are the image resolution size, G_s(i, j) is the gradient value of the corresponding pixel, the edge response is very strong when the visibility is high, otherwise, the edge response is very weak;

calculate POLC (weber contrast to lightness) index:

dividing an M multiplied by N image into M multiplied by N areas, calculating the contrast of each area and an L channel value of an LAB color space of the image as lightness, and obtaining the POLC by taking the mean value of each area as a quotient

Where Ω (x) is the size of the divided region, M × N is the total number of windows, M ≦ M, N ≦ N, C (y) is the contrast, and L (y) is the lightness.

The data are in one-to-one correspondence, and the data are processed according to the following steps of 8:2, dividing the training set and the verification set in proportion, and performing deep learning:

selecting a depth residual error network ResNet model, wherein the ResNet model comprises a plurality of residual error units shown in FIG. 4, selecting ResNet50 as a network design, and the schematic diagram of the network architecture is shown in FIG. 5;

the network architecture adjustment and determination process is shown in fig. 6: except for the image data of the camera, the visibility data is fitted by an auxiliary network through weather data, environmental condition data, calculated contrast, edge characteristics and a POLC index, so that the last line is removed, three layers of lines are added again, the auxiliary information is fused with the image information extracted by the layer 4 in front of the network when the first layer of line is used, the first layer of line is fitted, and the neuron of the last layer of line is 1 to represent the visibility;

and (3) network training:

s01), dividing the training set into a training set validation set according to the proportion of 8:2, subdividing the divided training set, and carrying out 4-fold cross validation;

s02), data amplification is carried out on the image, and then the image is converted into a 4-D sensor for normalization processing. The required assist features are then calculated. Performing code conversion and normalization on the auxiliary weather data and the environmental condition data;

s03), normalizing the visibility data to be used as label data;

s04), setting relevant hyper-parameters including maximum iteration times, a learning rate, an L2 regular coefficient and a learning rate attenuation coefficient, and using a learning rate hot start + cosine annealing learning rate strategy, wherein the learning rate hot start refers to setting a certain epoch hot start stage when the network just starts to train, so that the learning rate linearly increases from 0 to the set initial learning rate. Continuing the selected learning rate strategy at the epoch after the hot start stage;

s05), using Adam as an optimizer, and adding L2 regular as a penalty term for preventing network overfitting;

s06), using MSE Loss as a Loss function to supervise network learning;

s07), iterative training, wherein each time an epoch is trained, the verification set and the test set are used for carrying out precision calculation, and the best epoch in the training process is saved;

training a network by using the training data pair to construct a visibility model, optimizing the network and constructing the visibility model;

s3, constructing a visibility monitoring system for all road camera data:

establishing a database for all cameras on a road by using a GIS technology, and constructing a monitoring system for road visibility, so as to realize the processing and map display of all camera images along the road;

s4, popularizing the visibility model to the visibility judgment of all camera image data in the system;

and S5, visibility display, grading early warning and early warning prompt through a display screen.