阅读说明：本技术 一种水处理加药量控制方法及系统 (A kind of water treatment medicine amount control method and system ) 是由 吴伟 吴海锁 丁瑞金 陈朋利 谢祥峰 李�杰 吴云波 王小祥 于 2019-09-05 设计创作，主要内容包括：本发明公开了一种水处理加药量控制方法及系统,结合深度强化学习算法不断优化加药策略。首先,通过采集当前进水水质水量等参数,来确定药剂投加量。然后,通过对系统出水水质进行监测,根据出水水质以及加药量计算得到本次加药过程的奖励值,并将奖励值返回用于训练更新强化学习系统中的神经网络,从而使得加药控制系统能够学习更好的加药策略,获取更好的污水处理效果。之后不断循环上述过程,使神经网络不断更新,加药系统能够持续学习,最终得到最优的加药策略。本发明能够综合考虑影响污水处理效果的多个特征,无需专家知识指导,可通过自身学习来选择不同进水条件下的最佳投药量,最终可实现水处理过程中加药量的智能化控制。(The invention discloses a kind of water treatment medicine amount control method and systems, continue to optimize dosing strategy in conjunction with deeply learning algorithm.Firstly, by acquiring the parameters such as current influent quality water, to determine added amount of chemical.Then, by being monitored to system effluent quality, the reward value of this dosing process is calculated according to effluent quality and dosage, and reward value is returned to the neural network being used in training update reinforcement learning system, so that control system for adding drugs can learn better dosing strategy, better wastewater treatment efficiency is obtained.Later constantly circulation the above process, constantly update neural network, medicine system can continuous learning, finally obtain optimal dosing strategy.The present invention can comprehensively consider the multiple features for influencing wastewater treatment efficiency, instruct without expertise, can be learnt by itself to select the optimal coagulant dose under different flow conditions, the intelligentized control method of dosage in final achievable water treatment procedure.)

1. a kind of water treatment medicine amount control method characterized by comprising

Sewage treatment medicine system influent quality parameter is obtained, in conjunction with M group history influent quality parameter value and corresponding to every group The history dosage of the history influent quality parameter value forms ambient condition s；

Ambient condition is inputted into default neural network and extracts feature, determines the dosage a of water treatment procedure, and send instruction control Dosing carries out water process；

Obtain the sewage treatment medicine system effluent quality parameter after carrying out water process；

Reward value r is calculated according to influent quality parameter, effluent quality parameter and the dosage before and after water process；

Reward value is used for the neural network in training update deeply study.

2. water treatment medicine amount control method according to claim 1, which is characterized in that the water quality parameter includes influencing Temperature, turbidity, coloration, flow, pH value, COD, ammonia nitrogen, total phosphorus, the conductivity of water treatment efficiency.

3. water treatment medicine amount control method according to claim 1 or 2, which is characterized in that the ambient condition includes: The Wastewater Treatment Parameters at passing M moment and selected dosage constitute the matrix of M × M, and M is positive integer.

4. water treatment medicine amount control method according to claim 1, which is characterized in that the calculation formula of the reward value r(o,a,o^-) are as follows:

Wherein o is influent quality parameter, o^-For effluent quality parameter, d (o, o^-) be pollutant reduction effect, a is dosage.

5. water treatment medicine amount control method according to claim 4, which is characterized in that the training of neural network updates strong Chemistry, which is practised, uses DDPG, including movement network and evaluation network；

Network inputs environmental state information is acted, dosage, i.e. μ (s are exported；θ), movement network weight is θ；

Network inputs environmental state information and dosage are evaluated, the evaluation for selecting the dosage under the state, i.e. Q are exported (s,a；W), evaluation network weight is w, is updated for auxiliary movement network.

6. water treatment medicine amount control method according to claim 3, which is characterized in that the structure packet of the neural network Include two layers of convolutional neural networks, one layer of LSTM model layer and one layer of full articulamentum；

The intensified learning neural network extracts feature, first layer convolutional layer convolution to ambient condition first with convolutional neural networks Core size is 3 × 3, step-length 1, and input channel number is 1, and output channel number is 4, using one layer of pond layer, pond after convolution Change layer core having a size of 2, exports the tensor for 5 × 5 × 4；

Second layer convolutional layer convolution kernel size is 3 × 3, step-length 1, and input channel number is 4, and output channel number is 8, and exporting is 5 × 5 × 8 characteristic pattern；

Using obtained feature as the input of LSTM model layer, act in network directly by LSTM layers of output as full articulamentum Input, full articulamentum finally export dosage；It is inputted in evaluation network using LSTM output and dosage as full articulamentum, Quan Lian Layer is connect finally to export for state-movement pair evaluation.

7. water treatment medicine amount control method according to claim 6, which is characterized in that the neural network uses experience Playback is extracted training data and is updated, and target network, respectively μ ' (s is respectively set for movement network and evaluation network； θ ') and Q ' (s, a；W '), to reduce correlation between data, wherein θ ' and w ' is movement target network and evaluation goal respectively The neural network weight of network.

8. water treatment medicine amount control method according to claim 7, which is characterized in that the experience replay include: by The data that each dosing generates: state-movement-reward value-new state tuple (s_i,a_i,r_i+1,s_i+1), it is stored in experience database In, experience database size is D, and after being filled with data in experience database, new data will successively replace experience database In legacy data；After experience database experience is filled with, every N_uIt is secondary to obtain new experience, N item is randomly selected from experience database Experience is updated movement network and target network.

9. water treatment medicine amount control method according to claim 8, which is characterized in that movement network and target network into Row updates, comprising:

Step 1 extracts N empirical data (s from experience database_i,a_i,r_i+1,s_i+1)；

Step 2 enables y_i=r_i+1+γQ′(s_i+1|μ′(s_i+1；θ′)；W ') Calculation Estimation network more fresh target, wherein γ is decaying system Number；

Step 3 passes through minimum loss functionCarry out more New Appraisement network；

Step 4 utilizes sampled gradients update action network

Step 5 carries out soft update to the target network of movement network and evaluation network respectively, and target network is forced to former network direction It is close:

Wherein τ is soft update coefficient.

10. a kind of water treatment medicine amount control system characterized by comprising

Data acquisition module is used for: being obtained sewage treatment medicine system influent quality parameter, is joined in conjunction with M group history influent quality Numerical value and corresponding to history influent quality parameter value described in every group history dosage form ambient condition；It obtains and carries out at water Sewage treatment medicine system effluent quality parameter after reason；

Dosage determining module, is used for: ambient condition being inputted default neural network and extracts feature, determines adding for water treatment procedure Dose, and send instruction control dosing and carry out water process；

Reward value computing module, is used for: according to the forward and backward influent quality parameter of water process, effluent quality parameter and dosing meter Calculation receives awards value；

Training module is used for: reward value is used for the neural network in training update deeply study.

Technical field

The invention belongs to water process intelligent control technology fields, and in particular to a kind of water treatment medicine amount control method and System.

Background technique

In recent years, larger for the fluctuation of sewage treatment plant's water quality and quantity, there are hysteresis qualitys and accuracy for artificial dosing method The problems such as poor, emerges a collection of water treatment medicine control technology based on water quality and quantity.Wherein, a kind of to exist for wastewater treatment Line Adding medicine control method and system (108408855 B of CN), a kind of water treatment medicine digitize on-line control system (CN 105425592 B) and a kind of Powdered Activated Carbon Automatic Dosage Control of Additives system (107512754 A of CN) for water process etc. is specially Benefit discloses the water process intelligent control method based on BP neural network, is also presently the most widely used based on influent quality Water process Automatic Dosing amount control method.It is compared with the traditional method, without artificially being judged every time, saves medicament throwing Dosage and human cost.But there are still certain technological deficiencies for such methods, give one in advance as such method is required to expert Group data instruct the judgement of neural network learning dosage, that is, need artificially to give required dosing under specific sewage state Amount, then the Generalization Capability of neural network is relied on, it went in deduction data set in the state of not occurring, required dosage. In addition, expertise needed for such method is nearly all to pass through the acquisitions such as small-scale experiment method, actual sewage system in laboratory It unites not necessarily identical as the optimal dosage under laboratory environment.Also, the response situation of different sewage treatment plant also phase not to the utmost Together, many trained models can not be directly used in other sewage treatment plants.

Summary of the invention

Purpose: in order to overcome the deficiencies in the prior art, the present invention provides a kind of water treatment medicine amount control method And system, it proposes the deeply learning method for being used for water process automatic medicament feeding system, learns from water-treated into experience, nothing Expertise is needed, and can be designed by reward value and fully consider water treatment efficiency and economic benefit, to solve the prior art Existing deficiency.

Technical solution: in order to solve the above technical problems, the technical solution adopted by the present invention are as follows:

A kind of water treatment medicine amount control method, comprising:

Sewage treatment medicine system influent quality parameter is obtained, in conjunction with M group history influent quality parameter value and is corresponded to The history dosage of history influent quality parameter value described in every group forms ambient condition s；

Ambient condition is inputted into default neural network and extracts feature, determines the dosage a of water treatment procedure, and send instruction It controls dosing and carries out water process；

Obtain the sewage treatment medicine system effluent quality parameter after carrying out water process；

Reward value r is calculated according to the forward and backward influent quality parameter of water process, effluent quality parameter and dosage；

Reward value is used for the neural network in training update deeply study.

The water quality parameter includes influencing the temperature of water treatment efficiency, turbidity, coloration, flow, pH value, COD, ammonia nitrogen, total Phosphorus, conductivity.

The ambient condition include: the passing M moment Wastewater Treatment Parameters and selected dosage, constitute M × M Matrix, M is positive integer.

Calculation formula r (o, a, the o of the reward value^-) are as follows:

Wherein o is influent quality parameter, o^-For effluent quality parameter, d (o, o^-) be pollutant reduction effect, a is dosing Amount.

The water treatment medicine amount control method, the training of neural network update intensified learning and use DDPG, including dynamic Make network and evaluation network；

Network inputs environmental state information is acted, dosage, i.e. μ (s are exported；θ), movement network weight is θ；

Network inputs environmental state information and dosage are evaluated, is exported for selecting commenting for the dosage under the state Valence, i.e. Q (s, a；W), evaluation network weight is w, is updated for auxiliary movement network.

The structure of the neural network includes two layers of convolutional neural networks, (shot and long term remembers net to one layer of LSTM model layer Network) and one layer of full articulamentum；

The intensified learning neural network extracts feature, first layer convolutional layer to ambient condition first with convolutional neural networks Convolution kernel size is 3 × 3, step-length 1, and input channel number is 1, and output channel number is 4, using one layer of pond after convolution Layer, pond layer core export the tensor for 5 × 5 × 4 having a size of 2；

Second layer convolutional layer convolution kernel size is 3 × 3, step-length 1, and input channel number is 4, and output channel number is 8, output For 5 × 5 × 8 characteristic pattern；

Using obtained feature as the input of LSTM model layer, acts in network and directly connect LSTM layers of outputs conduct entirely The input of layer is connect, full articulamentum finally exports dosage；It evaluates in network that LSTM output and dosage is defeated as full articulamentum Enter, full articulamentum is finally exported for state-movement pair evaluation.

The neural network is extracted training data using experience replay and is updated, and is movement network and evaluation network Target network, respectively μ ' (s is respectively set；θ ') and Q ' (s, a；W '), to reduce correlation between data, wherein θ ' and w ' It is the neural network weight for acting target network and evaluation goal network respectively.

The experience replay includes: the data for generating each dosing: state-movement-reward value-new state tuple (s_i,a_i,r_i+1,s_i+1), it is stored in experience database, experience database size is D, after being filled with data in experience database, newly Data will successively replace the legacy data in experience database；When experience database experience is filled with, every N_uIt is secondary to obtain new experience, N experience is randomly selected from experience database to be updated movement network and target network.

The water treatment medicine amount control method, acts network and target network is updated, comprising:

Step 1 extracts N empirical data (s from experience database_i,a_i,r_i+1,s_i+1)；

Step 2 enables y_i=r_i+1+γQ′(s_i+1|μ′(s_i+1；θ′)；W ') Calculation Estimation network more fresh target, wherein γ be Attenuation coefficient；

Step 3 passes through minimum loss functionCarry out more New Appraisement network；

Step 4 utilizes sampled gradients update action network

Step 5 to movement network and evaluates the target network of network and carries out soft update respectively, and target network is to former network side To approaching:

Wherein τ is soft update coefficient.

According to another aspect of the present invention, a kind of water treatment medicine amount control system is provided, comprising:

Data acquisition module is used for: sewage treatment medicine system influent quality parameter is obtained, in conjunction with M group history water inlet water Matter parameter value and corresponding to history influent quality parameter value described in every group history dosage form ambient condition；It obtains and carries out Sewage treatment medicine system effluent quality parameter after water process；

Dosage determining module, is used for: ambient condition being inputted default neural network and extracts feature, determines water treatment procedure Dosage, and send instruction control dosing carry out water process；

Reward value computing module, is used for: according to the forward and backward influent quality parameter of water process, effluent quality parameter and dosing Amount calculates the value that receives awards；

Training module is used for: reward value is used for the neural network in training update deeply study.

The utility model has the advantages that water treatment medicine amount control method provided by the invention and system, are learnt based on DDPG deeply Algorithm extracts the feature between 10 moment water quality parameters in the past with convolutional neural networks, with Recognition with Recurrent Neural Network LSTM to mistake It summarizes toward experience, reward value is calculated with wastewater treatment efficiency and dosage, evaluation of training network, and by evaluation network Estimation is updated movement network, significant effect.Relative to traditional intelligence agent adding device for treatment of water, it is not necessarily to expertise, is assigned Program search for identity is given, the ability learnt from interaction, universality is higher.

Adding medicine control program in this method can learn in the interaction with environment, instruct nerve without expertise Network updates.The present invention can comprehensively consider multiple water quality and quantity parameters, extract sewage feature using convolutional neural networks, consider water The water treatment efficiency for the treatment of agent dosage is as reward, training deep neural network, to obtain optimal added amount of chemical.

Detailed description of the invention

Fig. 1 is the flow diagram of water treatment medicine amount control method according to an embodiment of the invention；

Fig. 2 is movement network structure according to an embodiment of the invention；

Fig. 3 is evaluation network structure according to an embodiment of the invention；

Fig. 4 is LSTM infrastructure diagram according to an embodiment of the invention.

Specific embodiment

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and examples, how to apply to the present invention whereby Technological means solves technical problem, and the realization process for reaching technical effect can fully understand and implement.It needs to illustrate As long as not constituting conflict, each feature in each embodiment and each embodiment in the present invention can be combined with each other, It is within the scope of the present invention to be formed by technical solution.

Meanwhile in the following description, for illustrative purposes and numerous specific details are set forth, to provide to of the invention real Apply the thorough understanding of example.It will be apparent, however, to one skilled in the art, that the present invention can not have to tool here Body details or described ad hoc fashion are implemented.

In addition, step shown in the flowchart of the accompanying drawings can be in the department of computer science of such as a group of computer-executable instructions It is executed in system, although also, logical order is shown in flow charts, and it in some cases, can be to be different from herein Sequence execute shown or described step.