阅读说明：本技术 基于计算机视觉的智慧城市道路路面检测方法及系统 (Intelligent urban road pavement detection method and system based on computer vision ) 是由 瞿夕凡 于 2021-09-17 设计创作，主要内容包括：本发明涉及道路路面检测领域,具体涉及一种基于计算机视觉的智慧城市道路路面检测方法及系统。获取待检测路面区域的基准图像和对比图像,并对其进行预处理；对基准图像进行边缘检测,得到基准图像的各个边缘区域；基于基准图像的各个边缘区域,对基准图像和对比图像的像素点进行匹配,根据像素点的匹配结果可以确定待检测路面的平整度情况。本发明实现了对道路路面平整度的实时检测,提高了道路检测的准确性,同时也缩短了道路路面检测所使用的时间。(The invention relates to the field of road surface detection, in particular to a method and a system for detecting road surfaces of smart cities based on computer vision. Acquiring a reference image and a contrast image of a road surface area to be detected, and preprocessing the reference image and the contrast image; carrying out edge detection on the reference image to obtain each edge area of the reference image; and matching pixel points of the reference image and the comparison image based on each edge region of the reference image, and determining the flatness condition of the pavement to be detected according to the matching result of the pixel points. The invention realizes the real-time detection of the road surface evenness, improves the accuracy of the road detection and shortens the time for the road surface detection.)

1. A smart urban road pavement detection method based on computer vision is characterized by comprising the following specific steps:

an image acquisition step: acquiring binocular images of a road surface area to be detected, taking one of the binocular images as a reference image, and taking the other image as a contrast image;

image edge detection: carrying out edge detection on the reference image to obtain each edge area;

pixel point matching: acquiring a current pixel point in a reference image and a parallax value between the current pixel point and a matched pixel point in a comparison image;

judging whether a next pixel point of a current pixel point in the reference image is located in any edge region, and determining the size of a matching sliding window corresponding to the next pixel point and a parallax gradient range according to a judgment result;

determining the matching search range of the next pixel point in the comparison image according to the current pixel point in the reference image, the parallax value of the current pixel point between the matching pixel points in the comparison image and the parallax gradient range corresponding to the next pixel point;

matching a matching pixel point corresponding to the next pixel point in the comparison image according to the size of a matching sliding window corresponding to the next pixel point and the matching search range of the next pixel point in the comparison image;

calculating a parallax value between a next pixel point in the reference image and a matching pixel point in the comparison image, repeating the pixel point matching step according to the next pixel point in the reference image and the parallax value between the next pixel point and the matching pixel point in the comparison image, obtaining the matching pixel point of the next pixel point in the reference image in the comparison image and the parallax value between the next pixel point in the reference image and the matching pixel point in the comparison image, and further obtaining the matching pixel point of any one pixel point in the reference image in the comparison image;

a road surface detection step: and calculating a parallax value between any one pixel point in the reference image and a matching pixel point in the comparison image, and determining the road surface smoothness of the road surface area to be detected according to the parallax values corresponding to all the pixel points in the reference image.

2. The method of claim 1, wherein the step of determining a parallax gradient range corresponding to a pixel next to a current pixel according to the determination result comprises:

when the next pixel point of the current pixel point in the reference image is located in any edge region, the value range of the parallax gradient corresponding to the next pixel point is；

When the next pixel point of the current pixel point in the reference image is not located in any edge region, the value range of the parallax gradient corresponding to the next pixel point is；

Wherein the content of the first and second substances,is a threshold value for the gradient of the parallax,and the parallax gradient corresponding to the next pixel point of the current pixel point.

3. The method of claim 2, wherein the step of determining the matching search range of the next pixel point in the comparison image comprises:

when the next pixel point of the current pixel point in the reference image is located in any edge area, the matching search range of the next pixel point in the comparison image is；

When the next pixel point of the current pixel point in the reference image is not positioned in any edge region, the matching search range of the next pixel point in the comparison image is；

Wherein the content of the first and second substances,for the abscissa in the reference image of the current pixel,is the disparity value between the current pixel point in the reference image and the matching pixel point in the comparison image,for the abscissa of the next pixel point in the comparison image,in order to set the maximum parallax,to set the minimum parallax.

4. The computer vision-based intelligent urban road pavement detection method according to any one of claims 1-3, wherein the step of determining the size of the matching sliding window corresponding to the next pixel point according to the judgment result comprises:

when the next pixel point of the current pixel point in the reference image is positioned in any edge region, reducing the size of an original matching sliding window corresponding to the next pixel point;

and when the next pixel point of the current pixel point in the reference image is not positioned in any edge region, increasing the size of the original matching sliding window corresponding to the next pixel point.

5. The method of claim 4, wherein the step of matching the matching pixel point corresponding to the next pixel point in the comparison image according to the size of the matching sliding window corresponding to the next pixel point and the matching search range of the next pixel point in the comparison image comprises:

according to the size of a matching sliding window corresponding to the next pixel point, sliding the matching sliding window within the matching search range in the comparison image of the next pixel point, and in the sliding process of the matching sliding window, matching the next pixel point of the current pixel point in the reference image with each pixel point in the matching sliding window, so as to find each pixel point which is originally designed in the comparison image;

and respectively taking the next pixel point of the current pixel point in the reference image and the preliminarily-designed pixel point in the comparison image as the centers of the two matching sliding windows, carrying out one-to-one corresponding matching on other pixel points of the two matching sliding windows, screening out the best matching pixel point from all the preliminarily-designed pixel points according to the matching result, and taking the best matching pixel point as the matching pixel point of the next pixel point of the current pixel point in the reference image.

6. The method for detecting the road surface of a smart city road based on computer vision as claimed in any one of claims 1-3, wherein the step of determining the road surface smoothness of the road surface area to be detected according to the parallax values corresponding to all pixel points in the reference image comprises:

calculating the variance of the parallax values corresponding to all the pixel points in the reference image, carrying out normalization processing, and taking the variance obtained after the normalization processing as the final variance;

when the final variance is smaller than a first variance threshold value, judging that the road surface to be detected is of a first flatness grade;

when the final variance is more than or equal to the first variance threshold and less than or equal to the second variance threshold, judging that the road surface to be detected is of a second flatness grade;

and when the final variance is larger than a second variance threshold value, judging that the road surface to be detected is a third flatness grade, and sequentially reducing the road surface flatness corresponding to the first flatness grade, the second flatness grade and the third flatness grade.

7. A method for intelligent city road pavement detection based on computer vision according to any one of claims 1-3, wherein the image acquisition step further comprises preprocessing the reference image and the comparison image, the preprocessing comprising:

denoising the reference image and the comparison image respectively, and sharpening the denoised reference image and the comparison image respectively;

and respectively carrying out filtering processing on the sharpened reference image and the sharpened comparison image by adopting a pyramid layer-by-layer filtering mode so as to reduce the resolution of the images.

8. The utility model provides a wisdom urban road surface detecting system based on computer vision which characterized in that includes:

an image acquisition module to: acquiring binocular images of a road surface area to be detected, taking one of the binocular images as a reference image, and taking the other image as a contrast image;

an image edge detection module to: carrying out edge detection on the reference image to obtain each edge area;

the pixel point matching module is used for acquiring an initial pixel point in the reference image and a parallax value between the initial pixel point and a matching point of the initial pixel point in the comparison image;

judging whether a next pixel point of a current pixel point in the reference image is located in any edge region, and determining the size of a matching sliding window corresponding to the next pixel point and a parallax gradient range according to a judgment result;

determining the matching search range of the next pixel point in the comparison image according to the current pixel point in the reference image, the parallax value of the current pixel point between the matching points in the comparison image and the parallax gradient range corresponding to the next pixel point;

matching a matching pixel point corresponding to the next pixel point in the comparison image according to the size of a matching sliding window corresponding to the next pixel point and the matching search range of the next pixel point in the comparison image;

calculating a parallax value between a next pixel point in the reference image and a matching pixel point in the comparison image, repeating the steps in the pixel point matching module according to the next pixel point in the reference image and the parallax value between the next pixel point and the matching pixel point in the comparison image, obtaining the matching pixel point of the next pixel point in the reference image in the comparison image and the parallax value between the next pixel point in the reference image and the matching pixel point in the comparison image, and further obtaining the matching pixel point of any one pixel point in the reference image in the comparison image;

and the road surface detection module is used for calculating a parallax value between any one pixel point in the reference image and a matching pixel point in the comparison image, and determining the road surface leveling condition of the road surface area to be detected according to the parallax values corresponding to all the pixel points in the reference image.

9. The system of claim 8, wherein the step of determining the corresponding parallax gradient range of the next pixel point according to the determination result comprises:

when the next pixel point of the current pixel point in the reference image is located in any edge region, the value range of the parallax gradient corresponding to the next pixel point is；

When the next pixel point of the current pixel point in the reference image is not located in any edge region, the value range of the parallax gradient corresponding to the next pixel point is；

Wherein the content of the first and second substances,is a threshold value for the gradient of the parallax,in order to be a parallax gradient, the parallax gradient,is a parallax gradient range.

10. The computer vision based intelligent urban road pavement detection system according to claim 9, wherein the step of determining the matching search range of the next pixel point in the comparison image comprises:

when the next pixel point of the current pixel point in the reference image is located in any edge area, the next pixel point is locatedThe search range for matches in the comparison image is；

When the next pixel point of the current pixel point in the reference image is not positioned in any edge region, the matching search range of the next pixel point in the comparison image is；

Wherein the content of the first and second substances,for the abscissa in the reference image of the current pixel,is the disparity value between the current pixel point in the reference image and the matching pixel point in the comparison image,the abscissa of the next pixel in the comparison image and,in order to set the maximum parallax,to set the minimum parallax.

Technical Field

The application relates to the field of road pavement detection, in particular to a method and a system for detecting road surfaces of smart cities based on computer vision.

Background

With the rapid development of society, vehicles in cities are increased, but road safety is becoming a concern. The influence of the road flatness on the vehicle running is relatively serious, and particularly, when an automobile runs on an uneven road at night, the automobile brings great trouble to a driver, and even traffic accidents can be caused. At present, most of small urban roads have the phenomenon of unevenness, so that the phenomenon of bumping and the like in the driving process of a vehicle can be caused, the chassis, the bumper and the like of the vehicle can be damaged in serious cases, and a large amount of time and money are wasted.

At present, road detection is usually manual detection or sensor detection, the limitation is large, the accuracy is low, manual detection wastes a large amount of manpower and material resources, phenomena such as false detection and missing detection are easy to occur, and the real-time performance is low; hardware equipment such as sensors is high in cost and complex in external environment, and the hardware equipment such as the sensors is prone to failure under the influence of the external environment, so that the conditions that detection is not timely or detection results are inaccurate are caused.

Disclosure of Invention

In order to solve the above technical problems, the present invention aims to provide a method and a system for detecting a road surface of a smart city based on computer vision, wherein the technical scheme adopted is as follows:

the invention provides a computer vision-based intelligent urban road surface detection method, which comprises the following specific steps:

an image acquisition step: acquiring binocular images of a road surface area to be detected, taking one of the binocular images as a reference image, and taking the other image as a contrast image;

image edge detection: carrying out edge detection on the reference image to obtain each edge area;

pixel point matching: acquiring a current pixel point in a reference image and a parallax value between the current pixel point and a matched pixel point in a comparison image;

judging whether a next pixel point of a current pixel point in the reference image is located in any edge region, and determining the size of a matching sliding window corresponding to the next pixel point and a parallax gradient range according to a judgment result;

determining the matching search range of the next pixel point in the comparison image according to the current pixel point in the reference image, the parallax value of the current pixel point between the matching pixel points in the comparison image and the parallax gradient range corresponding to the next pixel point;

matching a matching pixel point corresponding to the next pixel point in the comparison image according to the size of a matching sliding window corresponding to the next pixel point and the matching search range of the next pixel point in the comparison image;

calculating a parallax value between a next pixel point in the reference image and a matching pixel point in the comparison image, repeating the pixel point matching step according to the next pixel point in the reference image and the parallax value between the next pixel point and the matching pixel point in the comparison image, obtaining the matching pixel point of the next pixel point in the reference image in the comparison image and the parallax value between the next pixel point in the reference image and the matching pixel point in the comparison image, and further obtaining the matching pixel point of any one pixel point in the reference image in the comparison image;

a road surface detection step: and calculating a parallax value between any one pixel point in the reference image and a matching pixel point in the comparison image, and determining the road surface smoothness of the road surface area to be detected according to the parallax values corresponding to all the pixel points in the reference image.

Further, the step of determining the parallax gradient range corresponding to the next pixel point according to the judgment result includes:

when the next pixel point of the current pixel point in the reference image is located in any edge region, the value range of the parallax gradient corresponding to the next pixel point is；

When the next pixel point of the current pixel point in the reference image is not located in any edge region, the value range of the parallax gradient corresponding to the next pixel point is；

Wherein the content of the first and second substances,is a threshold value for the gradient of the parallax,the corresponding parallax gradient for the next pixel point.

Further, the step of determining the matching search range of the next pixel point in the comparison image includes:

when the next pixel point of the current pixel point in the reference image is located in any edge area, the matching search range of the next pixel point in the comparison image is；

When the next pixel point of the current pixel point in the reference image is not positioned in any edge region, the matching search range of the next pixel point in the comparison image is；

Wherein the content of the first and second substances,for the abscissa in the reference image of the current pixel,is the disparity value between the current pixel point in the reference image and the matching pixel point in the comparison image,for the abscissa of the next pixel point in the comparison image,in order to set the maximum parallax,to set the minimum parallax.

Further, the step of determining the size of the matching sliding window corresponding to the next pixel point according to the judgment result includes:

when the next pixel point of the current pixel point in the reference image is positioned in any edge region, reducing the size of an original matching sliding window corresponding to the next pixel point;

and when the next pixel point of the current pixel point in the reference image is not positioned in any edge region, increasing the size of the original matching sliding window corresponding to the next pixel point.

Further, according to the size of the matching sliding window corresponding to the next pixel point and the matching search range of the next pixel point in the comparison image, the step of matching the matching pixel point corresponding to the next pixel point in the comparison image is as follows:

sliding the matching sliding window within the matching search range of the next pixel point in the comparison image according to the size of the matching sliding window corresponding to the next pixel point, and matching the next pixel point of the current pixel point in the reference image with each pixel point in the matching sliding window in the sliding process of the matching sliding window, so as to find each pixel point which is originally designed in the comparison image;

and respectively taking the next pixel point of the current pixel point in the reference image and the preliminarily-designed pixel point in the comparison image as the centers of the two matching sliding windows, carrying out one-to-one corresponding matching on other pixel points of the two matching sliding windows, screening out the best matching pixel point from all the preliminarily-designed pixel points according to the matching result, and taking the best matching pixel point as the matching pixel point of the next pixel point of the current pixel point in the reference image.

Further, the step of determining the road surface smoothness of the road surface area to be detected according to the parallax values corresponding to all the pixel points in the reference image comprises the following steps:

calculating the variance of the parallax values corresponding to all the pixel points in the reference image, carrying out normalization processing, and taking the variance obtained after the normalization processing as the final variance;

when the final variance is smaller than a first variance threshold value, judging that the road surface to be detected is of a first flatness grade;

when the final variance is more than or equal to the first variance threshold and less than or equal to the second variance threshold, judging that the road surface to be detected is of a second flatness grade;

and when the final variance is larger than a second variance threshold value, judging that the road surface to be detected is a third flatness grade, and sequentially reducing the road surface flatness corresponding to the first flatness grade, the second flatness grade and the third flatness grade.

Further, the image acquiring step further includes preprocessing the reference image and the contrast image, and the preprocessing includes:

denoising the reference image and the comparison image respectively, and sharpening the denoised reference image and the comparison image respectively;

and respectively carrying out filtering processing on the sharpened reference image and the sharpened comparison image by adopting a pyramid layer-by-layer filtering mode so as to reduce the resolution of the images.

The invention also provides a computer vision-based intelligent urban road surface detection system, which comprises:

an image acquisition module to: acquiring binocular images of a road surface area to be detected, taking one of the binocular images as a reference image, and taking the other image as a contrast image;

an image edge detection module to: carrying out edge detection on the reference image to obtain each edge area;

the pixel point matching module is used for acquiring a current pixel point in the reference image and a parallax value between the current pixel point and a matching point of the current pixel point in the comparison image;

judging whether a next pixel point of a current pixel point in the reference image is located in any edge region, and determining the size of a matching sliding window corresponding to the next pixel point and a parallax gradient range according to a judgment result;

determining the matching search range of the next pixel point in the comparison image according to the current pixel point in the reference image, the parallax value of the current pixel point between the matching points in the comparison image and the parallax gradient range corresponding to the next pixel point;

matching a matching pixel point corresponding to the next pixel point in the comparison image according to the size of a matching sliding window corresponding to the next pixel point and the matching search range of the next pixel point in the comparison image;

calculating a parallax value between a next pixel point in the reference image and a matching pixel point in the comparison image, repeating the steps in the pixel point matching module according to the next pixel point in the reference image and the parallax value between the next pixel point and the matching pixel point in the comparison image, obtaining the matching pixel point of the next pixel point in the reference image in the comparison image and the parallax value between the next pixel point in the reference image and the matching pixel point in the comparison image, and further obtaining the matching pixel point of any one pixel point in the reference image in the comparison image;

and the road surface detection module is used for calculating a parallax value between any one pixel point in the reference image and a matching pixel point in the comparison image, and determining the road surface leveling condition of the road surface area to be detected according to the parallax values corresponding to all the pixel points in the reference image.

Further, the step of determining the parallax gradient range corresponding to the next pixel point according to the judgment result includes:

when the next pixel point of the current pixel point in the reference image is located in any edge region, the value range of the parallax gradient corresponding to the next pixel point is；

When the next pixel point of the current pixel point in the reference image is not located in any edge region, the value range of the parallax gradient corresponding to the next pixel point is；

Wherein the content of the first and second substances,is a threshold value for the gradient of the parallax,in order to be a parallax gradient, the parallax gradient,is a parallax gradient range.

Further, the step of determining the matching search range of the next pixel point in the comparison image includes:

when the next pixel point of the current pixel point in the reference image is located in any edge area, the matching search range of the next pixel point in the comparison image is；

When the next pixel point of the current pixel point in the reference image is not positioned in any edge region, the matching search range of the next pixel point in the comparison image is；

Wherein the content of the first and second substances,for the abscissa in the reference image of the current pixel,is the disparity value between the current pixel point in the reference image and the matching pixel point in the comparison image,for the abscissa of the next pixel point in the comparison image,in order to set the maximum parallax,to set the minimum parallax.

The invention has the following beneficial effects: the method comprises the steps of judging whether a pixel point to be matched, namely a next pixel point, is located in an edge area of a reference image, determining the size of a proper matching sliding window and a parallax gradient range according to a judgment result, determining the matching search range of the next pixel point in a comparison image according to the parallax gradient range, and finally finding the matching pixel point of the pixel point to be matched in the comparison image according to the size of the matching sliding window and the matching search range of the next pixel point in the comparison image. According to the invention, the size of the matching sliding window with proper size and the reasonable matching search range can be determined according to the position condition of the pixel point to be matched, so that the matching calculation amount can be effectively reduced while the matching pixel point of the pixel point to be matched is found, the matching accuracy and rapidity of the pixel point are improved, the detection and analysis of irrelevant matching pixel points are avoided, the overall progress of system detection is improved, and the detection time is shortened.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions and advantages of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.

FIG. 1 is a flow chart of a method for detecting road surfaces of smart cities based on computer vision according to the present invention;

FIG. 2 is a epipolar line diagram of an embodiment of the intelligent urban road pavement detection method and system based on computer vision according to the invention;

FIG. 3 is a detailed matching procedure of a pixel point matching module of the intelligent urban road pavement detection method and system based on computer vision according to the present invention;

wherein 01 is a three-dimensional space point; 02 is one of binocular images; 03 is the matching search range of the pixel points to be matched; 04 is a camera on one side of the road to be detected; 05 is the other of the binocular images; 06 is the matching search range of the pixel points to be matched; 07 is a camera on the other side of the road to be detected; 08 is the epipolar plane.

Detailed Description

To further illustrate the technical means and effects of the present invention adopted to achieve the predetermined objects, the following detailed description of the method for visually measuring the production information of a structural member according to the present invention, the specific implementation, structure, features and effects thereof will be given with reference to the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" refers to not necessarily the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

The following describes a specific scheme of the intelligent urban road pavement detection method based on computer vision in detail with reference to the accompanying drawings.

Referring to fig. 1, a flowchart illustrating steps of a smart urban road surface detection method based on computer vision according to an embodiment of the present invention is shown, the method including the following steps:

step (ii) of: and an image acquisition step, namely acquiring binocular images of the road surface area to be detected, taking one of the binocular images as a reference image, and taking the other image as a contrast image.

A plurality of cameras are arranged beside a road to acquire binocular images of the road surface to be detected, one of the binocular images is used as a reference image, and the other image is used as a contrast image. The arrangement of road camera is set for according to actual demand by oneself, but need guarantee that binocular image gathers the acquisition based on the camera under the different visual angles to the image of gathering is to same road surface region on the road, and the same road surface region that just so says that waits to detect needs to appear in binocular image simultaneously. In the embodiment, in order to acquire binocular images, a plurality of cameras are arranged on one side of a road at equal intervals, and one camera is correspondingly arranged on the right opposite side of the road of each camera. In order to obtain binocular images of all road surface areas of the whole road, the intervals of a plurality of cameras on one side of the road cannot be too large, and it is necessary that the shots of adjacent cameras at equal intervals have overlapping areas.

The image acquisition step further comprises preprocessing the reference image and the contrast image, wherein the preprocessing comprises the following steps: firstly, the image is denoised and sharpened, so that the influence of external noise and illumination in the image acquisition process is eliminated, and meanwhile, the texture information of the image edge can be enhanced, and the region with poor matching effect in the image matching process is conveniently eliminated in the follow-up process. In this embodiment, the image denoising processing method adopts a median filtering denoising algorithm, and the image sharpening processing method adopts a laplacian sharpening operator. Of course, as other embodiments, the image denoising algorithm may also adopt a gaussian filter algorithm, a mean value filter algorithm, and the like, and the image sharpening processing algorithm may also adopt a sobel operator, a gradient sharpening, a differential sharpening algorithm, and the like. It should be noted that, the processing methods for denoising and sharpening the image and the implementation process of the algorithm are all the prior art, and are not in the scope of the present invention, and no relevant description is provided here.

Then, in order to highlight the image texture information during image matching, the search range in the matching process is reduced, the retrieval speed is improved, and the low resolution processing is performed on the de-noised and sharpened original image. And (3) carrying out pyramid layer-by-layer filtering on the denoised and sharpened original image, wherein the original image needs to be filtered layer by layer, so that the purpose of reducing the resolution of the original image layer by layer is achieved, and finally, a multi-layer filtered low-resolution image is obtained. It is stated that how many filtering layers are selected can be set according to the quality of the image collected by the camera, and the filtering layers are set to be 3 layers in this embodiment. There are many image low-resolution processing methods, and when the pyramid layer-by-layer filtering method is adopted in this embodiment, the filtering algorithm therein adopts a gaussian filtering algorithm. Pyramid level-by-level filtering and gaussian filtering algorithms are prior art and are not within the scope of the present invention, and are not described here.

Step (ii) of: and an image edge detection step, wherein edge detection is carried out on the reference image to obtain each edge area. For the extraction of the edge region, the Sobel operator is adopted to extract the image edge, and all the edge regions in the reference image are obtained. The Sobel operator edge algorithm is a well-known technology, is not within the protection scope of the present invention, and is not further described herein.

Step (ii) of: a pixel point matching step, namely acquiring a current pixel point in the reference image and a parallax value between the current pixel point and a matching pixel point in the comparison image; judging whether a next pixel point of a current pixel point in the reference image is located in any edge region, and determining the size of a matching sliding window corresponding to the next pixel point and a parallax gradient range according to a judgment result; determining the matching search range of the next pixel point in the comparison image according to the current pixel point in the reference image, the parallax value of the current pixel point between the matching pixel points in the comparison image and the parallax gradient range corresponding to the next pixel point; matching a matching pixel point corresponding to the next pixel point in the comparison image according to the size of a matching sliding window corresponding to the next pixel point and the matching search range of the next pixel point in the comparison image; calculating the parallax value between the next pixel point in the reference image and the matching pixel point in the contrast image, and repeating the pixel point matching steps according to the next pixel point in the reference image and the parallax value between the next pixel point and the matching pixel point in the contrast image to obtain the matching pixel point of the next pixel point in the reference image and the matching pixel point in the contrast imageAnd obtaining the parallax value between the next pixel point in the reference image and the matching pixel point in the comparison image, thereby obtaining the matching pixel point of any one pixel point in the reference image in the comparison image.

As shown in fig. 3, the specific steps of the pixel matching step are as follows:

step (ii) of: firstly, acquiring the positions of initial pixel points in a reference image and matching pixel points of the initial pixel points in a comparison image in an image marking mode, and obtaining a parallax value between the initial pixel points and the matching pixel points in the comparison image according to the positions of the initial pixel points in the reference image and the matching pixel points in the comparison image. For convenience of the following description, the initial pixel point in the reference image is referred to as a current pixel point.

Step (ii) of: and judging whether the next pixel point of the current pixel point in the reference image is positioned in any edge region, and determining the size of a matching sliding window corresponding to the next pixel point of the current pixel point and the parallax gradient range according to the judgment result.

Step (ii) of: and judging whether the next pixel point of the current pixel point in the reference image is positioned in any edge area or not according to each edge area extracted in the image edge detection step.

Step (ii) of: according to the stepsDetermining that the next pixel point of the current pixel point corresponds to the determined resultThe matching sliding window size is as follows:

in the process of matching the road image pixel points, the matching of a single pixel point can be affected by illumination change and visual angle change, the robustness is poor, and in order to improve the matching precision of the road image pixel points, the embodiment adopts a sliding window mode for matching. When the fixed window is adopted for sliding window matching, a large number of wrong matching results can appear, and misjudgment of the pavement evenness is further caused, so that the matching of the pavement image pixel points adopts a sliding window matching mode.

The pixel points are located in different areas of the reference image, and the parallax variation of the pixel points is different. When the pixel points are in the edge area of the image, the parallax change is large, and the boundary is clear; when the pixel point is in the non-edge area of the image, the parallax change is relatively gentle. And dynamically adjusting the size of the sliding window according to the parallax change condition of the pixel points. For the edge area with large parallax change in the image, a sliding window with a small size is selected, so that the details of the edge area are richer; and selecting a sliding window with a larger size for an image area with smooth parallax change in the image, and increasing the texture features of the image contained in the window.

According to the stepsThe judging result determines whether the next pixel point is in the edge area, and then the size of the sliding matching window is dynamically adjusted in the next pixel point matching process, wherein the specific adjusting process is as follows:

when the next pixel point of the current pixel point in the reference image is located in the edge region, reducing the size of an original matching sliding window corresponding to the next pixel point;

and when the next pixel point of the current pixel point in the reference image is positioned in the non-edge region, increasing the size of the original matching sliding window corresponding to the next pixel point.

The specific implementation process is as follows: first, an original sliding window size is set toAnd the road surface image information can be set according to the road surface image information. Judging the region of the current pixel point in the image through an edge detection algorithm, and dynamically adjusting a sliding window on the basis:

wherein the content of the first and second substances,in order to be the original size of the sliding window,is a current pixel point in the reference imageThe size of the window after the adjustment is carried out,to adjust the factor, the present embodiment will adjust the factorSet to 4.

Step (ii) of: determining a parallax gradient range corresponding to a next pixel point of the current pixel point according to the judgment result of the edge area, and specifically comprising the following steps:

in this embodiment, a parallax gradient analysis model is constructed for detecting a parallax gradient of a pixel point next to a current pixel point in an image. Analyzing the search range of the matched pixel points according to the obtained parallax gradient range, and reducing the detection amount, wherein the determination process of the parallax gradient range is as follows:

firstly, two adjacent pixel points are selected from a three-dimensional scene、In the reference image and the contrast image respectively、And、the corresponding coordinate points are respectively、And、then, a parallax gradient calculation model is constructed as follows:

wherein the content of the first and second substances,is the parallax gradient of the next pixel point of the current pixel point, | | | | is a norm,is a pixel pointThe parallax between the reference image and the contrast image can be described as；Is a pixel pointThe parallax between the reference image and the contrast image can be described as. The following processing is carried out on the parallax gradient model:

wherein the content of the first and second substances,，，。

because two adjacent pixel points on the reference image、The coordinate relationship is as follows:，the final parallax gradient model is thus：

Wherein, it is known that two adjacent pixels on the reference image have sequence and are impossible to coincide with each other when matching pixels in the comparison image, soIs greater than zero. According to the final parallax gradient model, the following steps are carried outThe function value is increased and decreased, so the range of the parallax gradient in the model analysis is。

Step (ii) of: according to the parallax gradient rangeDetermining pixel points in a reference imageThe corresponding parallax gradient value range when the parallax gradient value range is located in the edge region or not located in the edge region includes the following specific contents:

when the pixel point in the reference imageWhen located in the region of any one of the edges,the parallax gradient value range of；

When the pixel point in the reference imageWhen it is not located in any one of the edge regions,the parallax gradient value range of。

Wherein the content of the first and second substances,is a threshold value for the gradient of the parallax,the parallax gradient of the next pixel point of the current pixel point. For parallax gradient thresholdThe setting of (c) can be selected according to the common parallax gradient, in this embodiment, the parallax gradient threshold is set to。

Step (ii) of: determining the matching search range of the next pixel point in the comparison image according to the current pixel point in the reference image, the parallax value of the current pixel point between the matching pixel points in the comparison image and the parallax gradient range corresponding to the next pixel point, wherein the determining step comprises the following steps:

firstly, the position of the opposite polar line of the pixel point in the binocular image is determined. In the matching process of the image pixel points, the projection points of one pixel point in the three-dimensional space in the images collected by the two cameras are always on the epipolar line, so that a fixed maximum matching search range can be determined on the epipolar line based on the projection points. The epipolar line is an intersection line of an epipolar plane and a drawing plane, the epipolar plane is a plane containing a base line, and the base line is a connecting line of light spots of the two cameras.

To facilitate the understanding of epipolar lines, fig. 2 shows that 01 is a pixel point in a three-dimensional space, and projection points in binocular images 05 and 02 formed by 07 cameras and 04 cameras are necessarily located on the epipolar lines, so that the matching search range of the pixel point to be matched can be 06 and 03.

Then, based on the position area of the epipolar line, the search range of the next pixel point in the contrast image is determined. Because the parallax changes among the pixel points are correlated, complete search and point-by-point matching analysis of the maximum matching search range are not needed in the matching search process. And determining the matching search range of the next pixel point of the current pixel point based on the parallax gradient range of the next pixel point of the current pixel point, the parallax value of the current pixel point and the current pixel point in the reference image. According to the stepsThe parallax gradient model ofAndthen, the search range prediction model is expressed as:

wherein the content of the first and second substances,for the next pixel point to be the matching pixel point on the comparison image,is the current pixel point on the reference image,is a pixel pointThe disparity between the reference image and the contrast image,the parallax gradient of the next pixel point of the current pixel point.

According to the current pixel point in the reference image, the value range of the parallax gradient of the next pixel point of the current pixel pointThe judgment of the position area of the next pixel point of the current pixel point and the parallax gradient of the next pixel point of the current pixel point determine the matching search range of the next pixel point in the comparison image, and the specific content comprises the following steps:

when the next pixel point of the current pixel point in the reference image is located in any edge area, the matching search range of the next pixel point in the comparison image is；

When the next pixel point of the initial pixel points in the reference image is not positioned in any edge region, the matching search range of the next pixel point in the comparison image is；

Wherein the content of the first and second substances,for the abscissa in the reference image of the current pixel,is the view between the current pixel point in the reference image and the matching pixel point in the contrast imageThe difference value is obtained by comparing the difference value,for the abscissa of the next pixel point in the comparison image,in order to set the maximum parallax,to set the minimum parallax.

Step 33: according to the size of a matching sliding window corresponding to the next pixel point and the matching search range of the next pixel point in the comparison image, matching the matching pixel point corresponding to the next pixel point in the comparison image, and the specific steps are as follows:

firstly, according to the size of a matching sliding window of a next pixel point in a comparison image, sliding the matching sliding window in a matching search range of the comparison image of the next pixel point. And in the sliding process of the matching sliding window, matching the next pixel point of the current pixel point in the reference image with each pixel point on the matching sliding window.

And then, screening according to the matching similarity between the next pixel point and each pixel point of the matching sliding window, determining each pixel point with proper matching similarity, and taking each pixel point as a primary pixel point. In the matching process, the distance measurement matching method is used as a determination method for detecting the matching similarity, that is, the absolute value of the gray difference between the next pixel point and each pixel point on the matching sliding window is used as a distance to determine, and each pixel point with the distance not exceeding the set similarity threshold is a tentative pixel point.

Of course, other embodiments may also employ: a normalized correlation coefficient matching method, a distance measurement method, a correlation coefficient matching method, a square error matching method, and the like. The method for measuring the similarity is a well-known technology and is not within the protection scope of the present invention, and the relevant description is not provided herein.

And finally, respectively taking the next pixel point in the reference image and one pixel point of the initial pixel point in the comparison image as the centers of the two matching sliding windows, and correspondingly matching other pixel points which do not comprise the center pixel points of the two matching sliding windows one by one.

And continuously repeating the matching steps of other pixel points, and screening out the optimal pixel point from the preliminary pixel points, wherein the optimal pixel point refers to that when the optimal pixel point is taken as the center of a matching sliding window, all pixel points in the matching sliding window are in one-to-one correspondence with all pixel points in the matching sliding window taking the next pixel point in the reference image as the center, and the matching similarity of each pair of one-to-one corresponding pixel points is greater than the similarity threshold. The best pixel point is the best matching pixel point of the next pixel point in the reference image in the comparison image.

Step (ii) of: and obtaining the matching pixel point of any one pixel point in the reference image in the comparison image according to the best matching pixel point of the next pixel point in the reference image.

And calculating the parallax value between the next pixel point in the reference image and the matching pixel point in the comparison image according to the optimal matching pixel point of the next pixel point in the reference image. And then, repeating the steps of the pixel point matching module according to the next pixel point in the reference image and the parallax value between the next pixel point and the matching pixel point in the comparison image, and obtaining the matching pixel point of the next pixel point in the reference image in the comparison image and the parallax value between the next pixel point in the reference image and the matching pixel point in the comparison image. And finally, determining that any pixel point in the reference image is matched with a pixel point in the comparison image and determining an integer parallax image of the road pavement to be detected.

The above stepsAnd 3, actually, repeating the steps of the pixel point matching module based on an iteration mode according to a matching pixel point of a certain pixel point in the reference image in the comparison image and a parallax value between the pixel point in the reference image and the comparison image, and obtaining a matching pixel point of a next pixel point of the certain pixel point in the reference image in the comparison image and a parallax value corresponding to a next pixel point of the certain pixel point in the reference image. Therefore, the matching pixel point and the parallax value of any pixel point in the reference image in the comparison image are determined.

Step (ii) of4: and a pavement detection step, namely determining the flatness condition of the pavement of the road to be detected according to the parallax value between any one pixel point in the reference image and the matching pixel point in the comparison image.

Determining the variance corresponding to the parallax value according to the calculated parallax values corresponding to all the pixel points in the reference imageAnd carrying out normalization processing, taking the variance obtained after the normalization processing as the final variance, and setting a first variance threshold value asThe second variance threshold isAt this time, there are:

when the final variance is less than the first variance thresholdJudging that the road surface to be detected is of a first flatness grade, and considering that the road flatness of the area is high and flat;

when the final variance is greater than or equal to the first variance thresholdAnd is less than or equal to the second variance thresholdJudging that the road surface to be detected is of a second flatness grade, considering that the road flatness of the area is general, and slightly fluctuating on the road surface;

when the final variance is greater than the second variance thresholdJudging that the road surface to be detected is of a third flatness grade, and considering that the road surface in the area has bulges or pits with larger amplitude;

the first flatness level, the second flatness level, and the third flatness level are threshold values set according to specific road flatness information, and the road flatness corresponding to each level is sequentially reduced. When the flatness grade of the road surface is detected to be higher, the system can timely give a corresponding alarm prompt to a driver or an urban road manager, avoid or overhaul the road as soon as possible, and prevent traffic accidents caused by the unevenness of the road surface.

This embodiment still provides a wisdom urban road surface detecting system based on computer vision, includes:

an image acquisition module to: acquiring binocular images of a road surface area to be detected, taking one of the binocular images as a reference image, and taking the other image as a contrast image;

an image edge detection module to: carrying out edge detection on the reference image to obtain each edge area;

the pixel point matching module is used for acquiring an initial pixel point in the reference image and a parallax value between the initial pixel point and a matching point of the initial pixel point in the comparison image;

judging whether a next pixel point of a current pixel point in the reference image is located in any edge region, and determining the size of a matching sliding window corresponding to the next pixel point and a parallax gradient range according to a judgment result;

determining the matching search range of the next pixel point in the comparison image according to the current pixel point in the reference image, the parallax value of the current pixel point between the matching points in the comparison image and the parallax gradient range corresponding to the next pixel point;

matching a matching pixel point corresponding to the next pixel point in the comparison image according to the size of a matching sliding window corresponding to the next pixel point and the matching search range of the next pixel point in the comparison image;

calculating a parallax value between a next pixel point in the reference image and a matching pixel point in the comparison image, repeating the steps in the pixel point matching module according to the next pixel point in the reference image and the parallax value between the next pixel point and the matching pixel point in the comparison image, obtaining the matching pixel point of the next pixel point in the reference image in the comparison image and the parallax value between the next pixel point in the reference image and the matching pixel point in the comparison image, and further obtaining the matching pixel point of any one pixel point in the reference image in the comparison image;

and the road surface detection module is used for calculating a parallax value between any one pixel point in the reference image and a matching pixel point in the comparison image, and determining the road surface leveling condition of the road surface area to be detected according to the parallax values corresponding to all the pixel points in the reference image.

All modules in the system are matched with each other, and the purpose is to realize a computer vision-based intelligent city road surface detection method.

It should be noted that: the precedence order of the above embodiments of the present invention is only for description, and does not represent the merits of the embodiments. And specific embodiments thereof have been described above. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.