阅读说明：本技术 一种自然覆冰在线监测运行导线图像覆冰程度评估方法 (Natural icing on-line monitoring running wire image icing degree evaluation method ) 是由 吴建蓉 文屹 曾华荣 何锦强 郝艳捧 梁苇 黄增浩 杨涛 范强 卢金科 曾伟 于 2021-08-09 设计创作，主要内容包括：本发明公开了一种自然覆冰在线监测运行导线图像覆冰程度评估方法,它包括：建立无覆冰导线标准知识库,根据图像自动配准方法将覆冰导线与对应同终端无覆冰导线进行匹配,进行导线分割；计算分割结果中导线Mask的面积差值,结合该终端在覆冰监测系统中的历史预警数据对导线Mask面积差值进行阈值设置,并将阈值设置结果作为覆冰程度预警判据。(The invention discloses a natural icing on-line monitoring running conductor image icing degree evaluation method, which comprises the following steps: establishing a standard knowledge base of the ice-free conductor, matching the ice-free conductor with the corresponding ice-free conductor of the same terminal according to an image automatic registration method, and performing conductor segmentation; and calculating the area difference of the lead Mask in the segmentation result, setting a threshold value of the area difference of the lead Mask by combining historical early warning data of the terminal in an icing monitoring system, and taking the threshold value setting result as an icing degree early warning criterion.)

1. A natural icing on-line monitoring running conductor image icing degree assessment method is characterized by comprising the following steps: establishing a standard knowledge base of the ice-free conductor, matching the ice-free conductor with the corresponding ice-free conductor of the same terminal according to an image automatic registration method, and performing conductor segmentation; and calculating the area difference of the lead Mask in the segmentation result, setting a threshold value of the area difference of the lead Mask by combining historical early warning data of the terminal in an icing monitoring system, and taking the threshold value setting result as an icing degree early warning criterion.

2. The method for evaluating the icing degree of the natural icing on-line monitoring running conductor image according to claim 1, characterized in that: the method for establishing the standard knowledge base of the ice-free conductor comprises the following steps: marking the non-icing conductor of each terminal, exporting the image of the on-line monitoring system according to the terminal number, selecting a non-icing image under each terminal to select and mark the conductor frame, and generating an RGB (red, green and blue) histogram of the non-icing image of the conductor so as to establish a non-icing conductor image knowledge base.

3. The method for evaluating the icing degree of the natural icing on-line monitoring running conductor image according to claim 1, characterized in that: the method for matching the ice-coated wire with the ice-coated wire without the corresponding same terminal according to the automatic image registration method and calculating the area difference of the wire Mask in the segmentation result comprises the following steps:

s2-1, image resolution investigation

Generating an RGB histogram of the picture to be detected, and putting the RGB histogram into an image knowledge base without an ice-coated wire for comparison; the comparison method comprises the following steps: comparing the resolution of the image to be detected with the knowledge base image, and excluding images with different resolutions; the resolution of the rest knowledge base candidate images is the same as that of the image to be detected;

s2-2, calculating image similarity

Putting histograms corresponding to the two images into a unified coordinate system, wherein each pixel point in the images represents a scale on a coordinate axis, and each abscissa in the coordinate system corresponds to two ordinates a and b and represents the number of pixels of the images under the brightness; calculating each horizontal coordinate scale, and calculating an average value;

s2-3, registering with terminal image

Performing similarity calculation traversal on the image to be detected and all images in the knowledge base, wherein the same terminal image has the highest similarity, so that a group of images with the smallest S is the same terminal image;

s2-4, strengthening of edge characteristics of lead

Converting the input of the identification network from a traditional original visible light image into an edge feature enhanced image by adopting an edge feature enhancement mode;

s2-5, ice-coated conductor region segmentation

Inputting a group of processed online monitoring images into a pre-trained Mask R-CNN wire recognition model for wire segmentation; mask R-CNN is input as an edge-enhanced image, and output as a wire region bbox (bounding box), a wire category (iced or iceless), and a wire region Mask (Mask).

4. The method for evaluating the icing degree of the natural icing on-line monitoring running conductor image according to claim 3, characterized by comprising the following steps of: the image similarity calculation formula is as follows:

and 1/S is similarity, N is the maximum value of the abscissa, and a and b are respectively the longitudinal coordinate values of the image to be detected and the knowledge base image corresponding to the same abscissa.

5. The method for evaluating the icing degree of the natural icing on-line monitoring running conductor image according to claim 3, characterized by comprising the following steps of: the method for strengthening the edge characteristics of the wire comprises the following steps: carrying out gray processing on an image to be detected, carrying out image edge identification by using a Canny operator, generating a binary image after carrying out Canny edge identification, and enabling the image to contain as many lead edges and as few background edges as possible through dual-threshold setting, wherein the binary image only contains edge information in the image; and superposing the binary image and the original image to obtain the edge feature enhanced image.

6. The method for evaluating the icing degree of the natural icing on-line monitoring running conductor image according to claim 3, characterized by comprising the following steps of: the method for calculating the area difference of the lead Mask in the segmentation result comprises the following steps: the Mask output in the Mask R-CNN network is represented by a binary image; the size of the binary image is the bbox size corresponding to the mask; obtaining the mask area by calculating the number of pixels with RGB values of (255,255 and 255) in the binary image; and (3) the area of the lead mask in the image to be detected after edge strengthening is S1, the area of the lead mask in the comparison image corresponding to the knowledge base is S2, and a difference value delta S is calculated, namely the delta S is S2-S1.

7. The method for evaluating the icing degree of the natural icing on-line monitoring running conductor image according to claim 3, characterized by comprising the following steps of: the method for early warning the icing degree comprises the following steps: exporting terminal icing images which are evaluated to have icing thickness larger than the designed ice thickness of the lead according to a tension calculation model in a historical icing database, calculating area difference values delta S3 in all the exported images, and taking the minimum value of each calculated delta S3 as an evaluation threshold T3 of which the terminal icing degree is severe; exporting terminal icing images which are evaluated to have icing thickness smaller than the designed ice thickness of the lead and larger than the early warning ice thickness according to a tension calculation model in a historical icing database, calculating area difference values delta S2 in all the exported images, and taking the minimum value of each calculated delta S2 as an evaluation threshold T2 of which the terminal icing degree is medium; exporting terminal icing images which are evaluated to have icing thickness smaller than the early warning icing thickness according to a tension calculation model in a historical icing database, calculating area difference values delta S1 in all the exported images, and taking the minimum value of each calculated delta S1 as an evaluation threshold T1 that the terminal icing degree is light; the degree of icing was evaluated according to the thresholds T1, T2 and T3.

8. The method for evaluating the icing degree of the natural icing on-line monitoring running conductor image according to claim 3, characterized by comprising the following steps of: the evaluation method of the icing degree according to the threshold values T1, T2 and T3 is as follows:

when Δ S < T1, the evaluation model output is "no ice coating";

when T1< Δ S < T2, the evaluation model output is "lighter";

the evaluation model output is "medium" when T2< Δ S < T3;

the evaluation model output is "severe" when Δ S > T3.

Technical Field

The invention belongs to the technical field of power grid icing monitoring, and particularly relates to a method for evaluating an icing degree of an image of an operation wire by on-line monitoring of natural icing.

Background

The ice coating seriously threatens the safety of the power transmission line, the terrain and weather in southern China are complex, and a plurality of long-distance extra/ultrahigh voltages pass through an ice coating area and are easily influenced by the ice coating, so that higher requirements on ice prevention and ice resistance are provided. Currently, there have been a number of research efforts in wire segmentation. However, most of the existing wire segmentation studies are based on mechanical models for analysis. However, the icing thickness calculation model has a large deviation from the actual value, the data availability of sensors such as microclimate and image sensors is not high in severe weather such as low temperature, rain and snow, and effective data quality assessment and improvement means are lacked, and most of the existing methods are to use computer graphics to perform traditional edge detection and deep learning based on visible light original images. The robustness of the edge detection algorithm is poor, and the expected effect of the algorithm cannot be realized in actual situations; the existing deep learning method based on the visible light original image is low in wire recall rate.

Disclosure of Invention

The technical problem to be solved by the invention is as follows: the method for evaluating the icing degree of the image of the running wire by on-line monitoring of the natural icing is provided, so that the problems that in the prior art, the icing degree of the wire is evaluated by adopting computer graphics to carry out traditional edge detection and deep learning based on a visible light original image are solved. The robustness of the edge detection algorithm is poor, and the expected effect of the algorithm cannot be realized in actual situations; the existing deep learning method based on the visible light original image has the technical problems of low wire recall rate and the like.

The technical scheme of the invention is as follows:

a natural icing on-line monitoring running conductor image icing degree assessment method comprises the following steps: establishing a standard knowledge base of the ice-free conductor, matching the ice-free conductor with the corresponding ice-free conductor of the same terminal according to an image automatic registration method, and performing conductor segmentation; and calculating the area difference of the lead Mask in the segmentation result, setting a threshold value of the area difference of the lead Mask by combining historical early warning data of the terminal in an icing monitoring system, and taking the threshold value setting result as an icing degree early warning criterion.

The method for establishing the standard knowledge base of the ice-free conductor comprises the following steps: marking the non-icing conductor of each terminal, exporting the image of the on-line monitoring system according to the terminal number, selecting a non-icing image under each terminal to select and mark the conductor frame, and generating an RGB (red, green and blue) histogram of the non-icing image of the conductor so as to establish a non-icing conductor image knowledge base.

The method for matching the ice-coated wire with the ice-coated wire without the corresponding same terminal according to the automatic image registration method and calculating the area difference of the wire Mask in the segmentation result comprises the following steps:

s2-1, image resolution investigation

Generating an RGB histogram of the picture to be detected, and putting the RGB histogram into an image knowledge base without an ice-coated wire for comparison; the comparison method comprises the following steps: comparing the resolution of the image to be detected with the knowledge base image, and excluding images with different resolutions; the resolution of the rest knowledge base candidate images is the same as that of the image to be detected;

s2-2, calculating image similarity

Putting histograms corresponding to the two images into a unified coordinate system, wherein each pixel point in the images represents a scale on a coordinate axis, and each abscissa in the coordinate system corresponds to two ordinates a and b and represents the number of pixels of the images under the brightness; calculating each horizontal coordinate scale, and calculating an average value;

s2-3, registering with terminal image

Performing similarity calculation traversal on the image to be detected and all images in the knowledge base, wherein the same terminal image has the highest similarity, so that a group of images with the smallest S is the same terminal image;

s2-4, strengthening of edge characteristics of lead

Converting the input of the identification network from a traditional original visible light image into an edge feature enhanced image by adopting an edge feature enhancement mode;

s2-5, ice-coated conductor region segmentation

Inputting a group of processed online monitoring images into a pre-trained Mask R-CNN wire recognition model for wire segmentation; mask R-CNN is input as an edge-enhanced image, and output as a wire region bbox (bounding box), a wire category (iced or iceless), and a wire region Mask (Mask).

The image similarity calculation formula is as follows:

and 1/S is similarity, N is the maximum value of the abscissa, and a and b are respectively the longitudinal coordinate values of the image to be detected and the knowledge base image corresponding to the same abscissa.

The method for strengthening the edge characteristics of the wire comprises the following steps: carrying out gray processing on an image to be detected, carrying out image edge identification by using a Canny operator, generating a binary image after carrying out Canny edge identification, and enabling the image to contain as many lead edges and as few background edges as possible through dual-threshold setting, wherein the binary image only contains edge information in the image; and superposing the binary image and the original image to obtain the edge feature enhanced image.

The method for calculating the area difference of the lead Mask in the segmentation result comprises the following steps: the Mask output in the Mask R-CNN network is represented by a binary image; the size of the binary image is the bbox size corresponding to the mask; obtaining the mask area by calculating the number of pixels with RGB values of (255,255 and 255) in the binary image; the area of the lead mask in the image to be detected after edge strengthening is S1, the area of the lead mask in the comparison image corresponding to the knowledge base is S2, and the difference value delta S is calculated, namely

ΔS＝S2-S1。

The method for early warning the icing degree comprises the following steps: exporting terminal icing images which are evaluated to have icing thickness larger than the designed ice thickness of the lead according to a tension calculation model in a historical icing database, calculating area difference values delta S3 in all the exported images, and taking the minimum value of each calculated delta S3 as an evaluation threshold T3 of which the terminal icing degree is severe; exporting terminal icing images which are evaluated to have icing thickness smaller than the designed ice thickness of the lead and larger than the early warning ice thickness according to a tension calculation model in a historical icing database, calculating area difference values delta S2 in all the exported images, and taking the minimum value of each calculated delta S2 as an evaluation threshold T2 of which the terminal icing degree is medium; exporting terminal icing images which are evaluated to have icing thickness smaller than the early warning icing thickness according to a tension calculation model in a historical icing database, calculating area difference values delta S1 in all the exported images, and taking the minimum value of each calculated delta S1 as an evaluation threshold T1 that the terminal icing degree is light; the degree of icing was evaluated according to the thresholds T1, T2 and T3.

The evaluation method of the icing degree according to the threshold values T1, T2 and T3 is as follows:

when Δ S < T1, the evaluation model output is "no ice coating";

when T1< Δ S < T2, the evaluation model output is "lighter";

the evaluation model output is "medium" when T2< Δ S < T3;

the evaluation model output is "severe" when Δ S > T3.

The invention has the beneficial effects that:

the method can effectively strengthen the expression of the edge characteristics of the wire in the image and reduce the interference of the background to the wire. Establishing an icing-free wire standard knowledge base, matching an icing-free wire with a corresponding terminal icing-free wire according to an image automatic registration method, calculating an area difference value of a wire Mask in a segmentation result, setting a threshold value of the area difference value of the wire Mask by combining historical early warning data of the terminal in an icing monitoring system, and taking a threshold value setting result as an icing degree early warning criterion; the method solves the problems that in the prior art, computer graphics are adopted for conducting traditional edge detection and deep learning based on visible light original images aiming at conducting wire icing degree evaluation. The robustness of the edge detection algorithm is poor, and the expected effect of the algorithm cannot be realized in actual situations; the existing deep learning method based on the visible light original image has the technical problems of low wire recall rate and the like.

Detailed Description

A natural icing on-line monitoring running conductor image icing degree assessment method comprises the following specific steps:

s1 preparation of standard knowledge base of ice-free conductor

Each terminal has a preset position which is relatively fixed on the tower, so that the similarity between the image of the same terminal and the ice-coating-free image is high. However, different image monitoring terminal manufacturers have different resolutions, and the distance and the view angle between the image monitoring terminal manufacturers and the shooting object are different. In order to accurately evaluate the icing thickness of the lead, the lead without icing at each terminal needs to be marked. The online monitoring system image can be exported according to the terminal number, one ice-coating-free image is selected under each terminal for conducting wire frame selection and marking, and a RGB histogram of the ice-coating-free image of the wire is generated, so that an ice-coating-free wire image knowledge base is established.

S2 icing conductor area segmentation

S2-1 image resolution review

And generating an RGB histogram of the picture to be detected, and comparing the RGB histogram with an image knowledge base without an ice-coated wire, wherein the two pictures are shot by the same terminal, so that the picture has the same resolution and the highest similarity. The comparison method comprises the following steps: and comparing the resolution of the image to be detected with the knowledge base image, and eliminating images with different resolutions. And the resolution of the rest knowledge base candidate images is the same as that of the image to be detected.

S2-2 image similarity calculation

And putting the histograms corresponding to the two images into a unified coordinate system. Since both images have the same resolution, the unified coordinate system may use the histogram coordinate system of either image. Each pixel point in the image represents a scale on a coordinate axis, each abscissa in the coordinate system corresponds to two ordinates a and b, and the number of pixels of the image under the brightness is represented. For each horizontal coordinate scale calculation (a-b)²And an average value is calculated. Defining 1/S as similarity, N as maximum abscissa, and a and b as the ordinate values of the image to be detected and the knowledge base image corresponding to the same abscissa.

S2-3 terminal image registration

And performing similarity calculation traversal on the image to be detected and all images in the knowledge base, wherein the same terminal image has the highest similarity, so that a group of images with the minimum S can be regarded as the same terminal image. In this way image registration with the terminal is performed.

S2-4 wire edge feature reinforcement

The wire is relatively slender in shape and therefore the edge of the wire is used as a feature in the wire recognition task. The edge information of the traditional wire in the natural icing visible light image of the wire is not sufficiently expressed and is easily interfered by the background with similar colors, and the wire identification accuracy is influenced, so that the input of an identification network can be converted into an edge feature enhanced image from the traditional original visible light image by adopting an edge feature enhancement mode. The edge feature strengthening steps are as follows:

the method comprises the steps of carrying out gray processing on an image to be detected, carrying out image edge identification by using a Canny operator, generating a binary image after Canny edge identification, and enabling the image to contain as many lead edges and as few background edges as possible through reasonable double-threshold setting, wherein the binary image only contains edge information in the image. And superposing the binary image and the original image to obtain the edge feature enhanced image.

S2-5 icing conductor area segmentation

And inputting a group of online monitoring images (to-be-detected images and knowledge base corresponding to the same terminal non-icing images) processed by S2-4 into a pre-trained Mask R-CNN wire recognition model for wire segmentation. Mask R-CNN is input as an edge-enhanced image, and output as a wire region bbox (bounding box), a wire category (iced or iceless), and a wire region Mask (Mask). Because the unified standard is used in the annotation, and the images come from the same terminal, the images have similar background and shooting angles. Therefore, the output images bbox identification areas of the Mask R-CNN model are basically the same. And outputting the bbox coordinate, and cutting the bbox area image from the input image to be used as the basis for evaluating the wire icing degree in the next step.

S3 assessment of conductor icing degree

S3-1 wire icing quantification index calculation

Since the ice layer is attached to the outer portion of the ice-coated wire, there is a difference in the area of the wire mask region in the bbox region image cut out at S2-5, and the larger the difference in area, the more serious the ice coating is represented. The Mask output in the Mask R-CNN network is represented by a binary image. The size of the binary image is the bbox size corresponding to the mask. Therefore, the mask area can be obtained by calculating the number of pixels of which the RGB values are (255 ) in the binary image. The method for calculating the area difference of the lead Mask in the segmentation result comprises the following steps: the area of the lead mask in the image to be detected after edge strengthening is S1, the area of the lead mask in the comparison image corresponding to the knowledge base is S2, and the difference value delta S is calculated, namely

ΔS＝S2-S1。

S3-2 wire icing degree judgment

Exporting terminal icing images which are evaluated to have icing thickness larger than the designed ice thickness of the lead according to a tension calculation model in a historical icing database, calculating area difference values delta S3 in all the exported images, and taking the minimum value of each calculated delta S3 as an evaluation threshold T3 of which the terminal icing degree is severe; exporting terminal icing images which are evaluated to have icing thickness smaller than the designed ice thickness of the lead and larger than the early warning ice thickness according to a tension calculation model in a historical icing database, calculating area difference values delta S2 in all the exported images, and taking the minimum value of each calculated delta S2 as an evaluation threshold T2 of which the terminal icing degree is medium; exporting terminal icing images which are evaluated to have icing thickness smaller than the early warning icing thickness according to a tension calculation model in a historical icing database, calculating area difference values delta S1 in all the exported images, and taking the minimum value of each calculated delta S1 as an evaluation threshold T1 that the terminal icing degree is light; the degree of icing was evaluated according to the thresholds T1, T2 and T3.

The evaluation method of the icing degree according to the threshold values T1, T2 and T3 is as follows:

when Δ S < T1, the evaluation model output is "no ice coating";

when T1< Δ S < T2, the evaluation model output is "lighter";

the evaluation model output is "medium" when T2< Δ S < T3;

the evaluation model output is "severe" when Δ S > T3.