阅读说明：本技术 基于VGG16-SegUnet和dropout的海流机叶片附着物识别方法 (Method for identifying attached matters of blades of ocean current machine based on VGG16-SegUnet and dropout ) 是由 彭海洋 王天真 于 2019-11-19 设计创作，主要内容包括：本发明属于海流机故障诊断领域,具体涉及一种基于VGG16-SegUnet和dropout的海流机叶片附着物识别方法,步骤如下：对海流机图像进行语义标注,完成原始数据集的创建；旋转增强原始数据集并进行标准化预处理；搭建VGG16-SegUnet网络；使用Adadelta优化器对网络进行训练；测试训练好的网络,完成海流机叶片附着物位置和大小的识别,同时估计识别结果的不确定度；最后计算出准确的附着物面积占比以及平均交并比。本发明解决了现有基于图像信号的海流机叶片附着物诊断方法不能定位附着物、输出准确附着物占比以及估计识别不确定度的问题,并为海流机叶片视情维护以及后续的容错控制提供了指导性建议。(The invention belongs to the field of ocean current machine fault diagnosis, and particularly relates to an ocean current machine blade attachment identification method based on VGG16-SegUnet and dropout, which comprises the following steps: performing semantic annotation on the ocean current machine image to complete the creation of an original data set; rotationally enhancing the original data set and carrying out standardized preprocessing; building a VGG16-SegUnet network; training the network by using an Adadelta optimizer; testing the trained network, completing the identification of the position and the size of the blade attachments of the ocean current machine, and simultaneously estimating the uncertainty of the identification result; and finally, calculating the accurate area ratio of the attachments and the average intersection ratio. The invention solves the problems that the existing diagnosis method for the attachments of the ocean current machine blade based on the image signal can not locate the attachments, output accurate attachment proportion and estimate and identify uncertainty, and provides guiding suggestions for the visual condition maintenance and the subsequent fault-tolerant control of the ocean current machine blade.)

1. A method for identifying attachments of blades of a current machine based on VGG16-SegUnet and dropout is characterized by comprising the following steps:

firstly, acquiring underwater images of ocean current machines with different attachment types, and then performing semantic annotation by using an open source tool labelme, thereby completing the creation of an original image-semantic label data set: background, leaves, attachments are labeled 0, 1, 2, respectively;

step two, expanding an original image-semantic tag data set by adopting a rotational data enhancement technology of [0 degrees and 360 degrees ], and then carrying out standardized preprocessing on the original image:

wherein x represents data of any dimension of R, G and B in the ocean current machine image; x is the number of_min，x_maxRespectively representing the minimum and maximum pixel values in x; x is finally coveredNormalized to [ -1,1 [ ]]；

Dividing the enhanced data into a training set, a verification set and a test set according to the ratio of 3:1: 1;

step three, building a VGG16-SegUnet novel semantic segmentation network: setting the convolution and maximum pooling models of the top 13 layers of VGG16 as feature extraction coding layers and initializing these convolution structures using ImageNet pre-training weights; the structure of the decoding layer is the same as that of the decoding layer in the SegNet, and the characteristic recovery is carried out by adopting the inverse maximum pooling; besides forward connection, the coding layer and the decoding layer are fused with feature cascade in Unet and maximum pooling index retention technology in SegNet, and a 30% dropout layer is inserted in the middle position; the introduction of Dropout can relieve the phenomenon of over-training fitting and provide different probability classification results;

inputting the preprocessed image data of the training set into VGG16-SegUnet, and outputting a pixel-by-pixel softmax probability classification result:

wherein x is⁽ⁱ⁾Representing the ith pixel point in a training image x; theta is the weight parameter matrix of the softmax classifier, an

then, an Adadelta optimizer is used for carrying out global training on the whole network, the cross entropy loss is reduced until the training times reach the set maximum value, and the final training weight is recorded:

wherein Loss (θ) represents a cross entropy Loss function; n is a radical of_trainRepresenting the number of training data; n is a radical of_nRepresenting the total number of pixels of the nth image; log (-) represents a logarithmic function; 1 {. is an illustrative function, when an expression in {. is established, 1 is output, and otherwise, 0 is output;

inputting the preprocessed test set image into VGG16-SegUnet loaded with training weight, outputting semantic segmentation graph, completing recognition of background, leaf and attachment position and size in the image, and estimating uncertainty of recognition result, wherein the specific implementation process is as follows:

i. inputting each Test image into VGG16-SegUnet fused with 30% dropout, and repeatedly performing the Test for 50 times to obtain 50 softmax probability classification results, which are recorded as Test₅₀；

ii, finding Test₅₀Mean value of

iii from

and step six, finally, calculating the accurate attachment area ratio and the identification accuracy index MIoU according to the semantic segmentation graph:

wherein AAP is the area ratio of attachments; attachment, blade respectively representing attachment and the entire ocean current machine blade area; the Count (·) is used for calculating the number of pixel points in the designated area;

wherein MIoU represents the average cross-over ratio; p is a radical of_ijThe number of pixels which represent that the real label is i and is mistakenly identified as a label j; p is a radical of_iiThe number of pixels which represent that the real label is i and are identified as the label i; p is a radical of_jiThe number of pixels representing that the true label is j and is erroneously recognized as label i.

Technical Field

The invention relates to the field of ocean current machine fault diagnosis, in particular to an ocean current machine blade attachment identification method based on VGG16-SegUnet and dropout.

Background

The ocean current energy is a renewable clean energy source known as 'blue oil field' and 'marine sauter arabia', and the following two forming modes are mainly adopted: more stable flow of seawater in subsea waterways and straits; the regular flow of seawater generated by tidal motion. Compared with wind energy and solar energy, ocean current energy has the advantages of predictability, high energy density and the like. As an ocean current energy power generation device, the ocean current machine has the advantages of low noise, reliable operation, no harsh site selection requirement and the like, and the power generation principle is as follows: the rotating machine absorbs the energy of flowing seawater and converts the energy into electric energy to be transmitted to a power grid to realize grid-connected power generation. Unlike land-based installations, ocean current machines, once put into operation officially, are placed under water for long periods of time, which creates several potential problems: (1) small marine organisms are likely to multiply in the form of attachments on the surface of the ocean current machine blades, which may cause blade imbalance failure; (2) the blades of the ocean current machine are generally made of metal materials, so that the blades can be corroded due to seawater immersion throughout the year, and the mechanical performance is influenced. Specifically, the imbalance faults caused by the attachments can cause the output voltage frequency and amplitude of the generator to be reduced, and the distortion of the waveform finally influences the quality and efficiency of power generation and even causes grid fluctuation. Therefore, for the ocean current power generation system, it is important to effectively check the corresponding fault state and give an early warning in the "germination" stage of the fault.

At present, methods for fault detection and diagnosis of ocean current machines are relatively few, and are mainly divided into two types based on electric signals (ocean current machine stator current and voltage) and image signals (ocean current machine underwater image data). However, in the case of a complex underwater environment, the analysis of the stator current and voltage signals alone is not sufficient to perform an accurate diagnosis of the extent of the attachment. In addition, the conventional diagnostic method for the attached matter of the blade of the ocean current machine based on the image signal has the following problems: (1) the position and size of the attachment are not identified; (2) precise attachment area fraction was not diagnosed; (3) the different attachment distributions cannot be identified and there is a lack of uncertainty in the diagnosis.

Disclosure of Invention

In order to solve the problems of the diagnosis method for the attached matter of the ocean current machine blade based on the image signal and realize more intuitive and accurate identification of the attached matter degree, the invention provides the identification method for the attached matter of the ocean current machine blade based on VGG16-SegUnet and dropout.

The identification method of the ocean current machine blade attachments based on VGG16-SegUnet and dropout comprises the following steps:

firstly, acquiring underwater images of ocean current machines with different attachment types, and then performing semantic annotation by using an open source tool labelme, thereby completing the creation of an original image-semantic label data set: the background, leaf, attachment are labeled 0, 1, 2, respectively.

Step two, expanding an original image-semantic tag data set by adopting a rotational data enhancement technology of [0 degrees and 360 degrees ], and then carrying out standardized preprocessing on the original image:

wherein x represents data of any dimension of R, G and B in the ocean current machine image; x is the number of_min，x_maxRespectively representing the minimum and maximum pixel values in x; x is finally normalized to [ -1,1 []。

And dividing the enhanced data into a training set, a verification set and a test set according to the ratio of 3: 1.

Step three, building a VGG16-SegUnet novel semantic segmentation network: setting the convolution and maximum pooling models of the top 13 layers of VGG16 as feature extraction coding layers and initializing these convolution structures using ImageNet pre-training weights; the structure of the decoding layer is the same as that of the decoding layer in the SegNet, and the characteristic recovery is carried out by adopting the inverse maximum pooling; besides forward connection, the coding layer and the decoding layer are fused with feature cascade in Unet and maximum pooling index retention technology in SegNet, and a 30% dropout layer is inserted in the middle position; the introduction of Dropout also provides different probabilistic classification results while mitigating the phenomenon of trained overfitting.

Inputting the preprocessed image data of the training set into VGG16-SegUnet, and outputting a pixel-by-pixel softmax probability classification result:

wherein x is⁽ⁱ⁾Representing the ith pixel point in a training image x; theta is the weight parameter matrix of the softmax classifier, an

p(y⁽ⁱ⁾＝l|x⁽ⁱ⁾(ii) a Theta) represents x⁽ⁱ⁾Predicted result y of⁽ⁱ⁾Probability of semantic label l; n represents the number of categories to be semantically labeled; exp^(·)Representing an exponential function; h is_θ(x⁽ⁱ⁾) The result vector is predicted for softmax.

Then, an Adadelta optimizer is used for carrying out global training on the whole network, the cross entropy loss is reduced until the training times reach the set maximum value, and the final training weight is recorded:

wherein Loss (θ) represents a cross entropy Loss function; n is a radical of_trainRepresenting the number of training data; n is a radical of_nRepresenting the total number of pixels of the nth image; log (-) represents a logarithmic function; 1 {. is an illustrative function, and when the expression in {. is established, 1 is output, and conversely, 0 is output.

Inputting the preprocessed test set image into VGG16-SegUnet loaded with training weight, outputting semantic segmentation graph, completing recognition of background, leaf and attachment position and size in the image, and estimating uncertainty of recognition result, wherein the specific implementation process is as follows:

i. inputting each Test image into VGG16-SegUnet fused with 30% dropout, and repeatedly performing the Test for 50 times to obtain 50 softmax probability classification results, which are recorded as Test₅₀。

ii, finding Test₅₀Mean value of

And variance

iii fromFinding out the maximum probability category of each pixel point, and then displaying a semantic segmentation graph through a visualization technology; and the variance corresponding to the maximum probability category is visually displayed in the form of an image, namely the uncertainty image.

And step six, finally, calculating the accurate attachment area ratio and the identification accuracy index MIoU according to the semantic segmentation graph:

wherein AAP is the area ratio of attachments; attachment, blade respectively representing attachment and the entire ocean current machine blade area; the Count (·) is used for calculating the number of pixel points in the designated area.

Wherein MIoU represents the average cross-over ratio; p is a radical of_ijThe number of pixels which represent that the real label is i and is mistakenly identified as a label j; p is a radical of_iiThe number of pixels which represent that the real label is i and are identified as the label i; p is a radical of_jiThe number of pixels representing that the true label is j and is erroneously recognized as label i.

Advantageous effects

Compared with the prior art, the invention has the following technical effects:

1) the invention uses image data with three dimensions (R, G, B), so that more abundant and intuitive attachment characteristic information can be extracted compared with one-dimensional current and voltage signals.

2) The semantic annotation method adopted by the invention can effectively improve the network training speed and generalization capability; the rotation data enhancement technology greatly reduces the work load of semantic annotation while simulating the rotation operation of the ocean current machine.

3) The VGG16-SegUnet novel semantic segmentation network provided by the invention can effectively segment the ocean current machine image, complete the identification of the position and size of the background, the blade and the attachment, and output the accurate attachment area ratio.

4) The attachment identification method can identify different blade attachment distributions, can estimate the uncertainty of the attachment identification result, and provides instructive suggestions for subsequent ocean current machine visual maintenance and fault-tolerant control.

Drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

FIG. 1 is a schematic diagram of an algorithm flow of a method for identifying attachments of blades of a current machine based on VGG16-SegUnet and dropout in the invention;

FIG. 2 is a schematic diagram of an image semantic annotation method according to the present invention;

FIG. 3 is a schematic diagram of the architecture of the semantic segmentation network VGG16-SegUnet proposed in the present invention.

Detailed Description

The following detailed description of embodiments of the invention is intended to be illustrative, and not to be construed as limiting the invention.

As shown in fig. 1, the invention provides a method for identifying attachments of blades of a current machine based on VGG16-SegUnet and dropout, which comprises the following steps:

firstly, acquiring underwater images of ocean current machines with different attachment types, and then performing semantic annotation by using an open source tool labelme, thereby completing the creation of an original image-semantic label data set: the blades, attachments are labeled 0, 1, 2, respectively, as shown in fig. 2.

Step two, expanding an original image-semantic tag data set by adopting a rotational data enhancement technology of [0 degrees and 360 degrees ], and then carrying out standardized preprocessing on the original image:

wherein x represents data of any dimension of R, G and B in the ocean current machine image; x is the number of_min，x_maxRespectively representing the minimum and maximum pixel values in x; x is finally normalized to [ -1,1 []。

And dividing the enhanced data into a training set, a verification set and a test set according to the ratio of 3: 1.

Step three, building a VGG16-SegUnet novel semantic segmentation network: setting the convolution and maximum pooling models of the top 13 layers of VGG16 as feature extraction coding layers and initializing these convolution structures using ImageNet pre-training weights; the structure of the decoding layer is the same as that of the decoding layer in the SegNet, and the characteristic recovery is carried out by adopting the inverse maximum pooling; besides forward connection, the coding layer and the decoding layer are fused with feature cascade in Unet and maximum pooling index retention technology in SegNet, and a 30% dropout layer is inserted in the middle position; the introduction of Dropout can relieve the phenomenon of over-training fitting and provide different probability classification results; the specific architectural design of VGG16-SegUnet is shown in FIG. 3.

Inputting the preprocessed image data of the training set into VGG16-SegUnet, and outputting a pixel-by-pixel softmax probability classification result:

wherein x is⁽ⁱ⁾Representing the ith pixel point in a training image x; theta is the weight parameter matrix of the softmax classifier, an

p(y⁽ⁱ⁾＝l|x⁽ⁱ⁾(ii) a Theta) represents x⁽ⁱ⁾Predicted result y of⁽ⁱ⁾Probability of being semantic label l(ii) a N represents the number of categories to be semantically labeled; exp^(·)Representing an exponential function; h is_θ(x⁽ⁱ⁾) The result vector is predicted for softmax.

Then, an Adadelta optimizer is used for carrying out global training on the whole network, the cross entropy loss is reduced until the training times reach the set maximum value, and the final training weight is recorded:

wherein Loss (θ) represents a cross entropy Loss function; n is a radical of_trainRepresenting the number of training data; n is a radical of_nRepresenting the total number of pixels of the nth image; log (-) represents a logarithmic function; 1 {. is an illustrative function, and when the expression in {. is established, 1 is output, and conversely, 0 is output.

Inputting the preprocessed test set image into VGG16-SegUnet loaded with training weight, outputting semantic segmentation graph, completing recognition of background, leaf and attachment position and size in the image, and estimating uncertainty of recognition result, wherein the specific implementation process is as follows:

i. inputting each Test image into VGG16-SegUnet fused with 30% dropout, and repeatedly performing the Test for 50 times to obtain 50 softmax probability classification results, which are recorded as Test₅₀。

ii, finding Test₅₀Mean value of

And variance

iii from

Finding out the maximum probability category of each pixel point, and then displaying a semantic segmentation graph through a visualization technology; the variance corresponding to the maximum probability category is visually displayed in the form of an image, namely uncertaintyAnd (4) an image.

And step six, finally, calculating the accurate attachment area ratio and the identification accuracy index MIoU according to the semantic segmentation graph:

wherein AAP is the area ratio of attachments; attachment, blade respectively representing attachment and the entire ocean current machine blade area; the Count (·) is used for calculating the number of pixel points in the designated area.

Wherein MIoU represents the average cross-over ratio; p is a radical of_ijThe number of pixels which represent that the real label is i and is mistakenly identified as a label j; p is a radical of_iiThe number of pixels which represent that the real label is i and are identified as the label i; p is a radical of_jiThe number of pixels representing that the true label is j and is erroneously recognized as label i.