阅读说明：本技术 一种基于贝叶斯优化的BiLSTM电压偏差预测方法 (BiLSTM voltage deviation prediction method based on Bayesian optimization ) 是由 王宝华 张文惠 王大飞 张弛 于 2021-06-07 设计创作，主要内容包括：本发明公开了一种基于贝叶斯优化的BiLSTM电压偏差预测方法,对电压偏差时间序列数据集进行标准差标准化处理,按照比例进行数据分割,得到训练集和验证集；利用预处理之后的电压偏差数据训练集训练BiLSTM电压偏差预测模型；将验证集输入到训练的BiLSTM电压偏差预测模型中,获取电压偏差预测值后进行反标准差处理,使用均方根误差作为BiLSTM电压偏差预测模型超参数优化的目标函数,利用贝叶斯优化算法对BiLSTM电压偏差预测模型的超参数进行优化,获取最优的超参数组合；将最优超参数组合作为BiLSTM预测模型的超参数,构建基于贝叶斯优化算法的BiLSTM电压偏差预测模型,对电压偏差时间序列数据进行预测,获得最终的预测数据。本发明精度高、预测效果可靠。(The invention discloses a BiLSTM voltage deviation prediction method based on Bayesian optimization, which comprises the steps of carrying out standard deviation standardization processing on a voltage deviation time series data set, and carrying out data segmentation according to a proportion to obtain a training set and a verification set; training a BiLSTM voltage deviation prediction model by utilizing the preprocessed voltage deviation data training set; inputting the verification set into a trained BilTM voltage deviation prediction model, obtaining a voltage deviation prediction value, then performing inverse standard deviation processing, using a root mean square error as a target function of hyperparameter optimization of the BilTM voltage deviation prediction model, and optimizing hyperparameters of the BilTM voltage deviation prediction model by using a Bayesian optimization algorithm to obtain an optimal hyperparameter combination; and (3) taking the optimal hyper-parameter combination as a hyper-parameter of the BilSTM prediction model, constructing the BilSTM voltage deviation prediction model based on the Bayesian optimization algorithm, predicting the voltage deviation time sequence data, and obtaining final prediction data. The invention has high precision and reliable prediction effect.)

1. A BiLSTM voltage deviation prediction method based on Bayesian optimization is characterized by comprising the following steps:

step 1, carrying out standard deviation standardization processing on a voltage deviation time series data set;

step 2, performing data segmentation on the voltage deviation time sequence data after standard deviation standardization according to a proportion to obtain a training set and a verification set which are respectively used for parameter learning and model result inspection of a training network model;

step 3, defining the range of the hyperparameters of the BilSTM voltage deviation prediction model, generating a random group of initial hyperparameters as the initial hyperparameters of the BilSTM model, and training the BilSTM voltage deviation prediction model by utilizing the preprocessed voltage deviation data training set;

step 4, inputting the verification set into a trained BilTM voltage deviation prediction model, obtaining a voltage deviation prediction value, then performing inverse standard deviation processing, using a root mean square error as a target function of hyperparameter optimization of the BilTM voltage deviation prediction model, and optimizing hyperparameters of the BilTM voltage deviation prediction model by using a Bayesian optimization algorithm to obtain an optimal hyperparameter combination;

and 5, using the optimal hyper-parameter combination as a hyper-parameter of the BilSTM prediction model, constructing the BilSTM voltage deviation prediction model based on a Bayesian optimization algorithm, and predicting the voltage deviation time sequence data to obtain final prediction data.

2. The Bayesian optimization-based BilSTM voltage deviation prediction method according to claim 1, wherein in step 1, the standard deviation normalization processing is performed on the voltage deviation time series data set, and the specific method is as follows:

step 101, selecting a node for voltage deviation prediction in a power distribution network, and acquiring corresponding voltage deviation data to obtain original voltage deviation time series data Z:

in the formula (1), the raw voltage deviation time-series data Z contains l data, wherein Z_lRaw voltage deviation data for the ith;

step 102: standardizing standard deviation of the time series data of the original voltage deviation to obtain normalized voltage deviation data of a continuous time series, wherein the range of the value range is [ -1,1 ];

the standard deviation normalization calculation formula is as follows (2):

in the formula (2), Z_iThe time series data of the original voltage deviation before the standard deviation standardization processing;normalizing the voltage deviation time-series data after the standard deviation processing; mu.s₁The mean value of the time series data of the original voltage deviation is obtained; sigma₁Is the standard deviation of the time series data of the deviation of the original voltage.

3. The Bayesian-optimization-based BilSTM voltage deviation prediction method as claimed in claim 1, wherein in step 2, the normalized standard deviation time-series data is divided into data in proportion, and 80% of the data is used as a training set X ═ X₁,x₂,…,x_n]For parameter learning for training the network model; the remaining 20% of the data were taken as test set Y ═ Y₁,y₂,…,y_m]And the method is used for model result inspection.

4. The BiLSTM voltage deviation prediction method based on bayesian optimization according to claim 1, wherein in step 3, a range of hyper-parameters of a BiLSTM voltage deviation prediction model is defined, a random set of initial hyper-parameters is generated as initial hyper-parameters of the BiLSTM model, and the BiLSTM voltage deviation prediction model is trained and trained by using the preprocessed voltage deviation data, and the method specifically comprises the following steps:

step 401, defining the range of the hyperparameter of the BilSTM voltage deviation prediction model, and generating a random set of initial hyperparameters.

Step 402, constructing a BilSTM voltage deviation prediction model, and performing model training by using an initial hyperparameter and a preprocessed voltage deviation data training set;

the BiLSTM voltage deviation prediction model consists of a forward LSTM layer and a backward LSTM layer, the LSTM layer controls the flow of data information of a memory unit through a forgetting gate, an input gate and an output gate, and a voltage deviation training set is firstly usedInputting data into the forward LSTM layer and the backward LSTM layer to obtain the hidden states of the forward layer and the backward layer at the current momentAndthen, the hidden states of the forward layer and the backward layer at the current moment are spliced to obtain a neuron output value O_tObtaining a predicted value of voltage deviation after passing through the full connection layer;

the method comprises the following steps of selecting information to be forgotten through a forgetting gate, wherein the specific calculation mode of the forgetting gate is as follows:

f_t＝σ(W_f·[h_t-1,x_t]+b_f) (3)

in the formula (3), W_fIs a weight matrix for a forgetting gate; h is_t-1Is the hidden state of the last-moment LSTM layer; x is the number of_tVoltage deviation data of the current moment; b_fIs the bias of the forgetting gate; σ is sigmoid function, f_tSetting the specific gravity of information forgetting for the state at the last moment;

selecting new information to be added through an input gate, firstly, selecting information i needing to be updated through the input gate_tWhile the tanh layer creates new candidate vectorsThen input the value i_tAnd candidate vectorThe multiplication is used for updating the cell state, and the specific calculation mode of an input gate is as follows:

i_t＝σ(W_i·[h_t-1,x_t]+b_i) (4)

wherein, W_iIs the weight matrix of the input gate; b_iIs the offset of the input gate; w_CIs a weight matrix of the tanh layer; b_CIs the bias of the tanh layer; c_tRepresenting the state of the currently input memory unit;

calculating the output value of the neuron through an output gate, the value o of the output gate_tDependent on the renewed cell state C_tThe output gate has the following calculation mode:

o_t＝σ(W_o[h_t-1,x_t]+b_o) (7)

h_t＝o_t×tanh(C_t) (8)

wherein: w_oIs a weight matrix of the output gates; b_oIs the offset of the output gate; h is_tThe input state is the hidden state of the LSTM layer and the input state of the next moment;

hidden states of forward layer and backward layer at current timeAndthe calculation method is as follows:

wherein f (-) is a forward information extraction function; b (-) is a backward information extraction function; w^fIs the weight matrix of the forward LSTM layer; b^fIs a bias of the forward LSTM layer;W^bis the weight matrix of the backward LSTM layer; b^bIs a bias to the LSTM layer;

the calculation formula of the neuron output value at the current moment is as follows:

in the formula (11), the reaction mixture is,for the hidden state of the forward LSTM layer,for the hidden state of the inverse LSTM layer, W_yIs a weight matrix of the BilSTM prediction output; b_yIs the bias of the BilSTM prediction output;

step 303, outputting the output value O of the neuron at the current moment_tAnd obtaining a voltage deviation predicted value after passing through the full connection layer, continuously iterating a loss value between the voltage deviation predicted value and the voltage deviation actual value through a loss function, and finishing the training of the BilSTM voltage deviation prediction model when an iteration termination condition is reached.

5. The BiLSTM voltage deviation prediction method based on bayesian optimization as claimed in claim 1, wherein the validation set is inputted into a trained BiLSTM voltage deviation prediction model, a voltage deviation prediction value is obtained and then an inverse standard deviation process is performed, a root mean square error is used as a target function of the BiLSTM voltage deviation prediction model for hyper-parameter optimization, and a bayesian optimization algorithm is used to optimize hyper-parameters of the BiLSTM voltage deviation prediction model to obtain an optimal hyper-parameter combination, the specific method is as follows:

step 501, inputting the voltage deviation validation set into the BilSTM voltage deviation prediction model trained in the step 3 to obtain a voltage deviation prediction value;

step 502, performing inverse standard deviation processing on the predicted voltage deviation value, wherein an inverse standard deviation calculation formula is as follows (12):

in the formula (12), the reaction mixture is,the data after the anti-normalization processing; y' is data before the inverse normalization processing; mu.s₁Is the mean of the output data; sigma₁Is the standard deviation of the output data;

step 503, using the root mean square error as an objective function of the hyperparametric optimization of the BilSTM voltage deviation prediction model:

in the formula (13), y_iAndthe actual value and the predicted value of the electric energy deviation are obtained, and n is the number of samples of the voltage deviation test set;

step 504, optimizing the hyper-parameters of the BilSTM voltage deviation prediction model by using a Bayesian optimization algorithm to obtain an optimal hyper-parameter combination;

the Bayesian optimization algorithm has two key parts: the probability agent model is a Gaussian process, the used collection function is an EI collection function, after the objective function of the Bayes optimization algorithm is constructed according to the step 403, the posterior probability of the Bayes optimization objective function is obtained by using the Gaussian process, and an optimal hyper-parameter combination is further selected from the hyper-parameter range by using the collection function according to the posterior probability of the Bayes optimization objective function;

(1) constructing an objective function of Bayesian optimization, and expressing as:

in the formula (14), w^*Determining an optimal parameter for Bayesian optimization, wherein W is a set of input hyperparameters, and W is a parameter space of multidimensional hyperparameters;

(2) a hyper-parametric data set D_i＝{(w₁,g(w₁)),(w₂,g(w₂)),…,(w_i,g(w_i) And the hyperparametric optimized objective function g (w) is input to the Gaussian process, assuming that the objective function g (w) obeys a Gaussian distribution, i.e.

g(w)～GP(μ(w),k(w,w′)) (15)

In equation (15), μ (w) is a mean function of g (w), and is typically set to 0; k (w, w') represents the covariance function of g (w);

(3) obtaining the posterior probability of the Bayesian optimization objective function, and expressing the posterior probability as the formula (16):

P(g_i+1|D_i)～N(μ_i+1(w),σ_i+1 ²(w)) (16)

in formula (16), g_i+1Bayesian optimization of the objective function for the next moment, D_iIn order to be a hyper-parametric data set,bayes optimization of the variance, μ, of the objective function for the next moment_i+1And (w) is the mean value of the Bayesian optimization objective function at the next moment.

(4) Based on the mean and variance of the posterior probability obtained in the previous step, the EI acquisition function is used for searching the next most potential evaluation point w_i+1；

w_i+1＝max_w∈Wα(w；D_i) (17)

In the formula (17), w_i+1The hyper-parameter, alpha (w; D), selected in step i +1_i) As a function of EI acquisition, D_iIs a hyper-parametric data set.

The calculation formula of the EI acquisition function is as shown in formula (18):

in the formula (18), v^*Expressing the current optimal function value, phi (·) is a standard normal distribution probability density function, xi is a balance function, mu (w) is a mean value function, sigma (w) is a standard deviation function, and D is a hyper-parameter data set;

(5) combining w with the obtained new set of hyper-parameters_i+1As the hyper-parameter of the BiLSTM voltage deviation prediction model, repeating the steps 401 and 403 to obtain a new hyper-parameter optimization objective function g (w)_i+1) Update the sample set D_i+1＝D_i∪w_i+1；

(6) And (5) repeating the steps (2) and (5), and when the hyper-parameter optimization objective function value corresponding to the newly selected hyper-parameter combination meets the requirement, terminating the Bayesian optimization algorithm and outputting the currently selected optimal hyper-parameter combination.

Technical Field

The invention relates to a BiLSTM voltage deviation prediction method based on Bayesian optimization, belonging to the field of electrical engineering and voltage deviation.

Background

In recent years, the power industry has been rapidly developed, and the demand of power consumers for voltage deviation is increasing. The voltage deviation problem can seriously damage the benefits of power consumers and related power industries, so that the monitoring of the voltage deviation must be increased to obtain a large amount of voltage deviation data, and a basis is provided for deeply analyzing the voltage deviation change trend. The voltage deviation data is effectively analyzed and predicted, so that the voltage deviation problem can be found by workers early, corresponding measures are taken, and the stable and safe operation of the power system is ensured.

Currently, the research efforts for predicting voltage deviation are few. Some scholars propose to predict the voltage deviation by using a linear regression method, a random time series method and a gray model, but the prediction precision of a single method is poor. Partial scholars use a method of predicting the voltage deviation by combining a clustering algorithm with a neural network, but the clustering algorithm is greatly influenced by an initial clustering center and has higher requirements on a data set. Some scholars propose to combine and predict voltage deviation by using various prediction models, but the difficulty of weight determination and modeling is improved. With the rise of deep learning algorithms, the function of a neural network in time series prediction is more obvious, but the problem of super-parameter selection is an important factor for ensuring the prediction accuracy of the prediction algorithms. The currently widely used hyper-parameter optimization is characterized by a grid search method, a random search method and a heuristic search method. The grid search method is to test each possible hyper-parameter combination once and select the optimal hyper-parameter, but the hyper-parameter optimization process is very time consuming. The random search method is evaluated through random hyper-parameter combination, and although the efficiency is improved, the training result is unstable. The heuristic search method is to perform optimization by using heuristic rules of problems, but the difficulty of establishing the heuristic rules is improved. Therefore, the existing voltage deviation prediction technology is yet to be further researched.

Disclosure of Invention

The invention aims to provide a BiLSTM voltage deviation prediction method based on Bayesian optimization to solve the problems of poor generalization capability and low prediction precision of a voltage deviation prediction model.

The technical solution for realizing the purpose of the invention is as follows: a BiLSTM voltage deviation prediction method based on Bayesian optimization comprises the following steps:

step 1, carrying out standard deviation standardization processing on a voltage deviation time series data set;

step 2, performing data segmentation on the voltage deviation time sequence data after standard deviation standardization according to a proportion to obtain a training set and a verification set which are respectively used for parameter learning and model result inspection of a training network model;

step 3, defining the range of the hyperparameters of the BilSTM voltage deviation prediction model, generating a random group of initial hyperparameters as the initial hyperparameters of the BilSTM model, and training the BilSTM voltage deviation prediction model by utilizing the preprocessed voltage deviation data training set;

step 4, inputting the verification set into a trained BilTM voltage deviation prediction model, obtaining a voltage deviation prediction value, then performing inverse standard deviation processing, using a root mean square error as a target function of hyperparameter optimization of the BilTM voltage deviation prediction model, and optimizing hyperparameters of the BilTM voltage deviation prediction model by using a Bayesian optimization algorithm to obtain an optimal hyperparameter combination;

and 5, using the optimal hyper-parameter combination as a hyper-parameter of the BilSTM prediction model, constructing the BilSTM voltage deviation prediction model based on a Bayesian optimization algorithm, and predicting the voltage deviation time sequence data to obtain final prediction data.

Further, in step 1, the standard deviation normalization processing is performed on the voltage deviation time series data set, and the specific method is as follows:

step 101, selecting a node for voltage deviation prediction in a power distribution network, and acquiring corresponding voltage deviation data to obtain original voltage deviation time series data Z:

in the formula (1), the raw voltage deviation time-series data Z contains l data, wherein Z_lRaw voltage deviation data for the ith;

step 102: standardizing standard deviation of the time series data of the original voltage deviation to obtain normalized voltage deviation data of a continuous time series, wherein the range of the value range is [ -1,1 ];

the standard deviation normalization calculation formula is as follows (2):

in the formula (2), Z_iThe time series data of the original voltage deviation before the standard deviation standardization processing;normalizing the voltage deviation time-series data after the standard deviation processing; mu.s₁The mean value of the time series data of the original voltage deviation is obtained; sigma₁Is the standard deviation of the time series data of the deviation of the original voltage.

Further, in step 2, the voltage deviation time-series data normalized by the standard deviation is subjected to data division in proportion, and 80% of the data is used as a training set X ═ X₁,x₂,…,x_n]For parameter learning for training the network model; the remaining 20% of the data were taken as test set Y ═ Y₁,y₂,…,y_m]And the method is used for model result inspection.

Further, in step 3, defining the range of the hyperparameter of the BilSTM voltage deviation prediction model, generating a random group of initial hyperparameters as the initial hyperparameters of the BilSTM model, and training the BilSTM voltage deviation prediction model by using the preprocessed voltage deviation data training set, wherein the specific method comprises the following steps:

step 301, defining the range of the hyperparameters of the BilSTM voltage deviation prediction model, and generating a random set of initial hyperparameters.

Step 302, constructing a BilSTM voltage deviation prediction model, and performing model training by using an initial hyperparameter and a preprocessed voltage deviation data training set;

the BiLSTM voltage deviation prediction model is composed of a forward LSTM layer and a backward LSTM layer, the LSTM layer controls the flow of data information of a memory unit through a forgetting gate, an input gate and an output gate, firstly, voltage deviation training set data are input into the forward LSTM layer and the backward LSTM layer, and the hidden states of the forward layer and the backward layer at the current moment are obtainedAndthen, the hidden states of the forward layer and the backward layer at the current moment are spliced to obtain a neuron output value O_tObtaining a voltage deviation predicted value through the full connection layer;

the method comprises the following steps of selecting information to be forgotten through a forgetting gate, wherein the specific calculation mode of the forgetting gate is as follows:

f_t＝σ(W_f·[h_t-1,x_t]+b_f) (3)

in the formula (3), W_fIs a weight matrix for a forgetting gate; h is_t-1Is the hidden state of the last-moment LSTM layer; x is the number of_tVoltage deviation data of the current moment; b_fIs the bias of the forgetting gate; σ is sigmoid function, f_tSetting the specific gravity of information forgetting for the state at the last moment;

selecting new information to be added through an input gate, firstly, selecting information i needing to be updated through the input gate_tWhile the tanh layer creates new candidate vectorsThen input the value i_tAnd candidate vectorThe multiplication is used for updating the cell state, and the specific calculation mode of an input gate is as follows:

i_t＝σ(W_i·[h_t-1,x_t]+b_i) (4)

wherein, W_iIs the weight matrix of the input gate; b_iIs the offset of the input gate; w_CIs a weight matrix of the tanh layer; b_CIs the bias of the tanh layer; c_tRepresenting the state of the currently input memory unit;

calculating the output value of the neuron through an output gate, the value o of the output gate_tDependent on the renewed cell state C_tThe output gate has the following calculation mode:

o_t＝σ(W_o[h_t-1,x_t]+b_o) (7)

h_t＝o_t×tanh(C_t) (8)

wherein: w_oIs a weight matrix of the output gates; b_oIs the offset of the output gate; h is_tThe input state is the hidden state of the LSTM layer and the input state of the next moment;

hidden states of forward layer and backward layer at current timeAndthe calculation method is as follows:

wherein f (-) is a forward information extraction function; b (-) is a backward information extraction function; w^fIs the weight matrix of the forward LSTM layer; b^fIs a bias of the forward LSTM layer; w^bIs the weight matrix of the backward LSTM layer; b^bIs a bias to the LSTM layer;

neuron output value O at present_tThe calculation formula of (2) is as follows:

in the formula (11), the reaction mixture is,for the hidden state of the forward LSTM layer,for the hidden state of the inverse LSTM layer, W_yIs a weight matrix of the BilSTM prediction output; b_yIs the bias of the BilSTM prediction output;

step 303, outputting the output value O of the neuron at the current moment_tAnd obtaining a voltage deviation predicted value after passing through the full connection layer, continuously iterating a loss value between the voltage deviation predicted value and the voltage deviation actual value through a loss function, and finishing the training of the BilSTM voltage deviation prediction model when an iteration termination condition is reached.

Further, inputting the verification set into a trained BilTM voltage deviation prediction model, obtaining a voltage deviation prediction value, then performing inverse standard deviation processing, using a root mean square error as a target function of hyperparametric optimization of the BilTM voltage deviation prediction model, and optimizing hyperparameters of the BilTM voltage deviation prediction model by using a Bayesian optimization algorithm to obtain an optimal hyperparametric combination, wherein the specific method comprises the following steps:

step 401, inputting the voltage deviation validation set into the BilSTM voltage deviation prediction model trained in the step 3 to obtain a voltage deviation prediction value;

step 402, performing inverse standard deviation processing on the predicted voltage deviation value, wherein an inverse standard deviation calculation formula is as follows (12):

in the formula (12), the reaction mixture is,the data after the anti-normalization processing; y' is data before the inverse normalization processing; mu.s₁Is the mean of the output data; sigma₁Is the standard deviation of the output data;

step 403, using the root mean square error as an objective function of the hyperparametric optimization of the BilSTM voltage deviation prediction model:

in the formula (13), y_iAndthe actual value and the predicted value of the electric energy deviation, and n is the number of samples of the voltage deviation test set;

step 404, optimizing the hyper-parameters of the BilSTM voltage deviation prediction model by using a Bayesian optimization algorithm to obtain an optimal hyper-parameter combination;

the Bayesian optimization algorithm has two key parts: the probability agent model is a Gaussian process, the used collection function is an EI collection function, after the objective function of the Bayes optimization algorithm is constructed according to the step 403, the posterior probability of the Bayes optimization objective function is obtained by using the Gaussian process, and an optimal hyper-parameter combination is further selected from the hyper-parameter range by using the collection function according to the posterior probability of the Bayes optimization objective function;

(1) constructing an objective function of Bayesian optimization, and expressing as:

in the formula (14), w^*Determining an optimal parameter for Bayesian optimization, wherein W is a set of input hyperparameters, and W is a parameter space of multidimensional hyperparameters;

(2) data set D_i＝{(w₁,g(w₁)),(w₂,g(w₂)),…,(w_i,g(w_i) And the hyperparametric optimized objective function g (w) is input to the Gaussian process, assuming that the objective function g (w) obeys a Gaussian distribution, i.e.

g(w)～GP(μ(w),k(w,w′)) (15)

In equation (15), μ (w) is a mean function of g (w), and is typically set to 0; k (w, w') represents the covariance function of g (w);

(3) obtaining the posterior probability of the Bayesian optimization objective function g (w), which is expressed as formula (16):

P(g_i+1|D_i)～N(μ_i+1(w),σ_iw1 ²(w)) (16)

in formula (16), g_i+1Bayesian optimization of the objective function for the next moment, D_iIn order to be a hyper-parametric data set,bayes optimization of the variance, μ, of the objective function for the next moment_i+1And (w) is the mean value of the Bayesian optimization objective function at the next moment.

(4) Based on the mean and variance of the posterior probability obtained in the previous step, the EI acquisition function is used for searching the next most potential evaluation point w_i+1；

w_i+1＝max_w∈Wα(w；D_i) (17)

In the formula (17), w_i+1The hyper-parameter, alpha (w; D), selected in step i +1_i) As a function of EI acquisition, D_iIs a hyper-parametric data set.

The calculation formula of the EI acquisition function is as shown in formula (18):

in the formula (18), v^*Expressing the current optimal function value, phi (·) is a standard normal distribution probability density function, xi is a balance function, mu (w) is a mean value function, sigma (w) is a standard deviation function, and D is a hyper-parameter data set;

(5) combining w with the obtained new set of hyper-parameters_i+1As the hyper-parameter of the BiLSTM voltage deviation prediction model, repeating the steps 401 and 403 to obtain a new hyper-parameter optimization objective function g (w)_i+1) Update the sample set D_i+1＝D_i∪w_i+1；

(6) And (5) repeating the steps (2) and (5), and when the hyper-parameter optimization objective function value corresponding to the newly selected hyper-parameter combination meets the requirement, terminating the Bayesian optimization algorithm and outputting the currently selected optimal hyper-parameter combination.

Compared with the prior art, the invention has the remarkable advantages that: 1. the BilSTM neural network is formed by combining a forward LSTM layer and a backward LSTM layer, namely, information can be transmitted back and forth at the same time to perform bidirectional learning, so that more information can be obtained, namely, the precision of a BilSTM prediction model is higher than that of a unidirectional LSTM prediction model. 2. And optimizing the hyperparameter of the BiLSTM model by using a Bayesian optimization algorithm, finding out the optimal hyperparameter combination, and further improving the generalization capability of the voltage deviation prediction model of the model. 3. The voltage deviation is predicted by using Bayesian optimization BiLSTM, the effect is good, and the accuracy is high. 4. The voltage deviation prediction method is stable and reliable, and the trained model can be directly used in future voltage deviation prediction application, so that the application is simple and convenient.

Drawings

FIG. 1 is a flow chart of a BiLSTM voltage deviation prediction method based on Bayesian optimization according to the present invention.

FIG. 2 is a graph comparing the voltage deviation predictions for the BilSTM method of the present invention and the BilSTM method without Bayesian optimization.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The performance and effect of the voltage deviation prediction model are considered to be improved. The invention uses a bidirectional long and short memory network (BilTM), and a BilTM neural network is formed by combining a forward LSTM and a backward LSTM, namely, information is transmitted back and forth at the same time, so that more information can be obtained, namely, the precision of a BilTM prediction model is higher than that of a unidirectional LSTM prediction model. And optimizing the hyperparameters by using a Bayesian optimization algorithm in the training process of the BiLSTM voltage deviation prediction model, and determining the optimal model hyperparameter combination. The Bayesian optimization method can fully utilize complete historical evaluation information when selecting the next group of most potential hyper-parameter combinations, reduce evaluation times, avoid unnecessary sampling and obtain the optimal solution of a complex objective function, namely the optimal hyper-parameter combination. And constructing a BilSTM voltage deviation prediction model based on a Bayesian optimization algorithm by using the optimal hyper-parameter combination, and effectively predicting the voltage deviation data.

A voltage deviation prediction model based on a Bayesian optimization bidirectional long and short memory network (BilSTM) is shown in figure 1, and the process comprises the following steps:

step 1, preprocessing a raw voltage deviation time series data set

In order to accelerate the convergence speed of the neural network prediction model and avoid gradient explosion in the training process, the standard deviation standardization processing is carried out on the voltage deviation time sequence data set.

Step 101, selecting a node for voltage deviation prediction in a power distribution network, and acquiring corresponding voltage deviation data to obtain original voltage deviation time series data Z:

in the formula (1), the raw voltage deviation time-series data Z contains l data, wherein Z_lThe raw voltage deviation data of the ith, and so on.

Step 102: standardizing standard deviation of the time series data of the original voltage deviation to obtain normalized voltage deviation data of a continuous time series, wherein the range of the value range is [ -1,1 ];

the standard deviation normalization calculation formula is as follows (2):

in the formula (2), Z_iThe time series data of the original voltage deviation before the standard deviation standardization processing;normalizing the voltage deviation time-series data after the standard deviation processing; mu.s₁The mean value of the time series data of the original voltage deviation is obtained; sigma₁Is the standard deviation of the time series data of the deviation of the original voltage. The voltage deviation time series data set can be made to conform to the normal distribution of the standard through an algorithm of standard deviation normalization.

Step 2, segmenting voltage deviation time series data

In order to realize a BilSTM voltage deviation prediction model, a certain amount of historical voltage deviation time sequence data is needed to complete learning training, and the aim is to fit the BilSTM voltage deviation prediction model based on Bayesian optimization; then, in order to objectively and fairly evaluate the performance of the predictive model network, a part of the voltage deviation time-series data is required to perform a test. Therefore, the voltage deviation time-series data normalized by the standard deviation in step 1 is proportionally divided into 80% of data as a training set X ═ X₁,x₂,…,x_n]For parameter learning for training the network model; the remaining 20% of the data were taken as test set Y ═ Y₁,y₂,…,y_m]The method is used for model result inspection and ensures the effectiveness and the practicability of the BiLSTM voltage deviation prediction model based on Bayesian optimization.

And 3, defining the range of the hyperparameters of the BilSTM voltage deviation prediction model, generating a random group of initial hyperparameters as the initial hyperparameters of the BilSTM model, predicting the preprocessed voltage deviation data training set by using the BilSTM voltage deviation prediction model, and obtaining a corresponding voltage deviation prediction value.

Step 301, defining the range of the hyperparameters of the BilSTM voltage deviation prediction model, and generating a random set of initial hyperparameters.

And 302, constructing a BilSTM voltage deviation prediction model by using the preprocessed voltage deviation data training set and the initial hyper-parameters.

The BilSTM voltage deviation prediction model consists of a forward LSTM layer and a backward LSTM layer. The LSTM layer controls the flow of data information of the memory unit through the forgetting gate, the input gate and the output gate. Firstly, inputting the voltage deviation training set data into the forward LSTM layer and the backward LSTM layer to obtain the hidden states of the forward layer and the backward layer at the current momentAndthen, the hidden states of the forward layer and the backward layer at the current moment are spliced to obtain a neuron output value O_t(ii) a And training a BilSTM voltage deviation prediction model through the full connection layer.

The method comprises the following steps of selecting information to be forgotten through a forgetting gate, wherein the specific calculation mode of the forgetting gate is as follows:

f_t＝σ(W_f·[h_t-1,x_t]+b_f) (3)

in the formula (3), W_fIs a weight matrix for a forgetting gate; h is_t-1Is the hidden state of the last-moment LSTM layer; x is the number of_tVoltage deviation data of the current moment; b_fIs the bias of the forgetting gate; σ is sigmoid function, f_t(range is 0-1) is the specific gravity for setting information forgetting to the state at the last moment.

New information to be added is selected through the input gate. First input gate i_tSelection of needUpdated information while the tanh layer creates new candidate vectorsThen input the value i_tAnd candidate vectorThe multiplication updates the cell state. The specific calculation method of the input gate is as follows:

i_t＝σ(W_i·[h_t-1,x_t]+b_i) (4)

wherein, W_iIs the weight matrix of the input gate; b_iIs the offset of the input gate; w_CIs a weight matrix of the tanh layer; b_CIs the bias of the tanh layer; c_tRepresenting the currently entered memory cell state.

The neuron output values are calculated through output gates. Value o of output gate_tDependent on the renewed cell state C_t. The output gate is calculated in the following way:

o_t＝σ(W_o[h_t-1,x_t]+b_o) (7)

h_t＝o_t×tanh(C_t) (8)

wherein: w_oIs a weight matrix of the output gates; b_oIs the offset of the output gate; h is_tThe hidden state of the LSTM layer is also the input state at the next time.

Hidden states of forward layer and backward layer at current timeAndthe calculation method is as follows:

wherein f (-) is a forward information extraction function; b (-) is a backward information extraction function; w^fIs the weight matrix of the forward LSTM layer; b^fIs a bias of the forward LSTM layer; w^bIs the weight matrix of the backward LSTM layer; b^bIs a bias towards the LSTM layer.

Neuron output value O at present_tThe calculation formula of (2) is as follows:

in the formula (11), the reaction mixture is,for the hidden state of the forward LSTM layer,for the hidden state of the inverse LSTM layer, W_yIs a weight matrix of the BilSTM prediction output; b_yIs the bias of the BilSTM prediction output;

step 303, outputting the output value O of the neuron at the current moment_tAnd obtaining a voltage deviation predicted value after passing through the full connection layer, continuously iterating a loss value between the voltage deviation predicted value and a voltage deviation actual value through a loss function, and finishing the training of the BilSTM voltage deviation prediction model when an iteration termination condition is reached.

And 4, inputting the voltage deviation test set into the BilSTM voltage deviation prediction model constructed in the step 3 to obtain a voltage deviation prediction value.

And 5, processing the inverse standard deviation of the predicted voltage deviation value: and carrying out inverse standard deviation processing on the voltage deviation predicted value of the BilSTM voltage deviation prediction model.

The inverse standard deviation calculation formula is as follows (12):

in the formula (12), the reaction mixture is,the data after the anti-normalization processing; y' is data before the inverse normalization processing; mu.s₁Is the mean of the output data; sigma₁Is the standard deviation of the output data.

Step 6, using the root mean square error as an objective function of the hyperparametric optimization of the BilSTM voltage deviation prediction model:

in the formula (13), y_iAndthe actual value and the predicted value of the electric energy deviation are shown, and n is the number of samples of the voltage deviation test set.

And 7, optimizing the hyperparameters of the BilSTM voltage deviation prediction model by using a Bayes optimization algorithm to obtain an optimal hyperparameter combination.

The Bayesian optimization algorithm has two key parts: a probabilistic proxy model and a collection function. The probability agent model used in the invention is a Gaussian process, and the used collection function is an EI collection function. And 6, constructing an objective function of the Bayesian optimization algorithm. And obtaining the posterior probability of the Bayesian optimization objective function through a Gaussian process. And further selecting an optimal hyper-parameter combination from the hyper-parameter range by using an acquisition function according to the posterior probability of the Bayesian optimization objective function.

Step 701, constructing a bayesian optimized objective function, which can be expressed as:

in the formula (14), w^*And determining the optimal parameters for Bayesian optimization, wherein W is a set of input hyperparameters, and W is a parameter space of the multidimensional hyperparameters.

Step 702, collecting the hyper-parameter data set D_i＝{(w₁,g(w₁)),(w₂,g(w₂)),…,(w_iG (wi)) and an objective function g (w) for hyperparametric optimization are input to the gaussian process. Assuming that the objective function g (w) follows a Gaussian distribution, i.e.

g(w)～GP(μ(w),k(w,w′)) (15)

In equation (15), μ (w) is a mean function of g (w), and is typically set to 0; k (w, w') represents the covariance function of g (w).

Step 703, obtaining the posterior probability of the Bayesian optimization objective function g (w).

The posterior probability of the Bayesian optimization objective function g (w) is expressed as formula (16):

P(g_i+1|D_i)～N(μ_i+1(w),σ_iw1 ²(w)) (16)

in formula (16), g_i+1Bayesian optimization of the objective function for the next moment, D_iIn order to be a hyper-parametric data set,bayes optimization of the variance, μ, of the objective function for the next moment_i+1And (w) is the mean value of the Bayesian optimization objective function at the next moment.

Step 704, based on the mean and variance of the posterior probability obtained in the previous step, using EI collection function to find the next most potential evaluation point w_i+1。

w_i+1＝max_w∈Wα(w；D_i) (17)

In the formula (17), w_i+1Selected in step i +1Hyper-parameter, alpha (w; D)_i) As a function of EI acquisition, D_iIs a hyper-parametric data set.

The calculation formula of the EI acquisition function is as shown in formula (18):

in the formula (18), v^*And expressing the current optimal function value, phi (·) is a standard normal distribution probability density function, ξ is a balance function, mu (w) is a mean function, sigma (w) is a standard deviation function, and D is a hyper-parameter data set.

Step 705, combine w with the new set of hyper-parameters_i+1Repeating the steps 4-6 as the hyper-parameter of the BiLSTM voltage deviation prediction model to obtain a new hyper-parameter optimization objective function g (w)_i+1) Update the sample set D_i+1＝D_i∪w_i+1。

Step 706, repeat step 702-705, when the hyper-parametric optimization objective function value corresponding to the newly selected hyper-parameter combination meets the requirement, terminate the bayesian optimization algorithm, and output the currently selected optimal hyper-parameter combination.

And 8, constructing a BilSTM voltage deviation prediction model based on a Bayesian optimization algorithm by using the optimal hyper-parameter combination obtained in the step 7 as a hyper-parameter of the BilSTM prediction model, and predicting the voltage deviation time sequence data to obtain final prediction data.

And 9, evaluating the result of the Bayesian optimization BiLSTM voltage deviation prediction model. And comparing the power quality prediction node of the target power distribution network with actual voltage deviation data corresponding to the prediction time period, and evaluating the accuracy and the fitting degree of the neural network prediction model by using the coefficients such as the root mean square error, the average absolute error and the Hilbert-Huang of the prediction result in multiple angles.

Step 901, calculating the root mean square error RMSE of the prediction result according to formula (19):

in the formula (19), y_kAndn is the number of samples in the test set.

Step 902, calculating the average absolute error MAE of the prediction result according to equation (20):

step 903, calculating the mean absolute error MAPE of the prediction result according to formula (21):

step 904, calculating the hill inequality coefficient TIC of the prediction result according to equation (22):

examples

In order to verify the effectiveness of the scheme of the invention, the voltage deviation prediction is carried out by taking an IEEE333 power distribution network as an example.

An overall block diagram of the method of the example is shown in fig. 1, comprising the steps of:

1. raw voltage offset time series dataset preprocessing

In an example, a 33-node distribution network as shown in fig. 2 is established, a node for voltage deviation prediction is selected, and corresponding voltage deviation data is collected for 720 groups of voltage deviation data. In order to make the range of the original voltage deviation data be [ -1,1], the original voltage deviation time series data are preprocessed by adopting a standard deviation standardization method in an expression (2) so that the voltage deviation time series data set conforms to the normal distribution of the standard.

2. Voltage offset time series data segmentation

Dividing the voltage deviation time series data normalized by the standard deviation in the step 1 according to a proportion, and taking 80% of the data as a training set X ═ X₁,x₂,…,x_n]576 groups for parameter learning for training the network model; the remaining 20% of the data were taken as test set Y ═ Y₁,y₂,…,y_m]And 144 groups are used for model result inspection, and the effectiveness and the practicability of the proposed BiLSTM voltage deviation prediction model based on Bayesian optimization are ensured.

3. Defining the range of the hyperparameters of the BilSTM voltage deviation prediction model, generating a random group of initial hyperparameters as the initial hyperparameters of the BilSTM model, predicting the preprocessed voltage deviation data training set by using the BilSTM voltage deviation prediction model, and obtaining a corresponding voltage deviation prediction value.

Step 301, defining the range of the hyperparameters of the BilSTM voltage deviation prediction model, and generating a random set of initial hyperparameters.

TABLE 1 hyper-parametric ranges for the BilSTM model

And 302, constructing a BilSTM voltage deviation prediction model by using the preprocessed voltage deviation data training set and the initial hyper-parameters.

The BilSTM voltage deviation prediction model consists of a forward LSTM layer and a backward LSTM layer. The LSTM layer controls the flow of data information of the memory unit through the forgetting gate, the input gate and the output gate. Firstly, inputting the voltage deviation training set data into the forward LSTM layer and the backward LSTM layer to obtain the hidden states of the forward layer and the backward layer at the current momentAndthen the current timeSplicing the hidden states of the forward layer and the backward layer to obtain o_t(ii) a And constructing a BilSTM voltage deviation prediction model through the full connection layer.

And selecting the information to be forgotten through a forgetting gate, wherein the concrete calculation mode of the forgetting gate is shown as a formula (3).

The new information to be added is selected by the input gate, which is calculated in a specific manner as equation (4).

And calculating the output value of the neuron through an output gate, wherein the calculation mode of the output gate is as shown in a formula (5).

Hidden states of forward layer and backward layer at current timeAndis calculated as equation (9) and equation (10).

Neuron output value O at present_tThe formula (2) is shown in formula (11).

Step 303, outputting the output value O of the neuron at the current moment_tAnd obtaining a voltage deviation predicted value after passing through the full-connection layer, continuously iterating the loss value between the voltage deviation predicted value and the voltage deviation actual value through a loss function, and stopping the operation of the BilSTM network when the iteration times reach 100 times or the loss function value is less than 0.001 to obtain a BilSTM voltage deviation prediction model.

4. And (4) inputting the voltage deviation test set into the BilSTM voltage deviation prediction model constructed in the step (3) to obtain a voltage deviation prediction value.

5. Inverse standard deviation processing of voltage deviation prediction values

And carrying out inverse standard deviation processing on the voltage deviation predicted value of the BilSTM voltage deviation prediction model. The inverse standard deviation calculation formula is as shown in formula (12).

6. The root mean square error is used as an objective function of the hyperparametric optimization of the BilSTM voltage deviation prediction model as shown in the formula (13).

7. And optimizing the hyperparameters of the BilSTM voltage deviation prediction model by using a Bayesian optimization algorithm to obtain an optimal hyperparameter combination.

The Bayesian optimization algorithm has two key parts: a probabilistic proxy model and a collection function. The probability agent model used in the invention is a Gaussian process, and the used collection function is an EI collection function. And 6, constructing an objective function of the Bayesian optimization algorithm. And obtaining the posterior probability of the Bayesian optimization objective function through a Gaussian process. And further selecting an optimal hyper-parameter combination from the hyper-parameter range by using an acquisition function according to the posterior probability of the Bayesian optimization objective function.

In step 701, an objective function of bayesian optimization is constructed, which can be expressed as equation (14).

Step 702, data set D is processed_i＝{(w₁,g(w₁)),(w₂,g(w₂)),…,(w_i,g(w_i) And the hyperparametric optimized objective function g (w) are input to the gaussian process. The objective function g (w) is assumed to follow a gaussian distribution, which is expressed as equation (15).

Step 703, obtaining the posterior probability of the Bayesian optimization objective function g (w). The posterior probability of the Bayesian optimization objective function g (w) is expressed as formula (16).

Step 704, based on the mean and variance of the posterior probability obtained in the previous step, using EI collection function to find the next most potential evaluation point w_i+1。w_i+1The specific calculation formula is shown in formula (17).

The calculation formula of the EI acquisition function is shown in formula (18).

Step 705, combine w with the new set of hyper-parameters_i+1Repeating the steps 4-6 as the hyper-parameter of the BiLSTM voltage deviation prediction model to obtain a new hyper-parameter optimization objective function g (w)_i+1) Update the sample set D_i+1＝D_i∪w_i+1。

Step 706, repeat step 702-705, when the hyper-parametric optimization objective function value corresponding to the newly selected hyper-parameter combination meets the requirement, terminate the bayesian optimization algorithm, and output the currently selected optimal hyper-parameter combination.

8. And (4) constructing a BilSTM voltage deviation prediction model based on a Bayesian optimization algorithm by using the optimal hyper-parameter combination obtained in the step (7) as the hyper-parameter of the BilSTM prediction model, and predicting the voltage deviation time sequence data to obtain final prediction data.

9. And evaluating the result of the Bayesian optimized BiLSTM voltage deviation prediction model. And comparing the power quality prediction node of the target power distribution network with actual voltage deviation data corresponding to the prediction time period, and evaluating the accuracy and the fitting degree of the neural network prediction model by using the coefficients such as the root mean square error, the average absolute error and the Hilbert-Huang of the prediction result in multiple angles.

Step 901, calculating the root mean square error RMSE of the prediction result according to formula (19).

Step 902, calculating the average absolute error MAE of the prediction result according to equation (20).

In step 903, the mean absolute percentage error MAPE of the prediction result is calculated according to equation (21).

Step 904, according to equation (22), calculates the hill inequality coefficient TIC of the prediction result.

In the examples, the voltage deviation prediction evaluation was performed according to the equations (19), (20), (21) and (22), as shown in table 2.

TABLE 2 predicted bias values for BilSTM voltage bias prediction based on Bayesian optimization

In the example, the Bayesian optimized BilSTM is compared with the ordinary BilSTM prediction result, and the comparison of the prediction results is shown in figure 2; the number of samples with different ranges of voltage deviation prediction errors between Bayesian-optimized BilSTM and normal BilSTM is shown in Table 3.

TABLE 3 comparison of two prediction models

Note: e represents the relative error percentage of the sample.

Example analysis shows that samples of the Bayesian optimization BilSTM prediction model in the range of the relative error less than 5% reach 70.83%, while samples of the common BilSTM prediction algorithm in the range of the relative error less than 5% are only 57.64%, which shows that the Bayesian optimization BilSTM prediction model performs good-effect prediction on voltage deviation. The Bayesian optimization BiLSTM prediction model can increase the probability that the relative error percentage falls in a low-value interval on the basis of a common BiLSTM prediction model, improves the prediction accuracy, provides technical support for electric energy quality early warning and early management, and is beneficial to reliable and safe operation of a power distribution network.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.