阅读说明：本技术 一种基于环锭纺纱线质量的全局工序共享控制系统及方法 (Global process sharing control system and method based on ring spinning yarn quality ) 是由 殷士勇 鲍劲松 于 2020-12-31 设计创作，主要内容包括：本发明公开了一种基于环锭纺纱线质量的全局工序共享控制系统及方法,包括云子系统和边缘子系统,云子系统包括获取模块、评估模块、修正模块,边缘子系统包括若干模块：Agent模块、状态识别模块、动作调节模块、奖励模块、CPPS单元,Agent模块包括Critic网络模块、Actor网络模块、误差模块；Critic网络模块与状态识别模块、奖励模块、误差模块分别连接,动作调节模块通过CPPS单元与状态识别模块连接,状态识别模块与奖励模块连接。本发明通过对纺纱工序中的质量进行控制,抑制后面单元中的质量波动,通过预测后面单元的质量特征调节前面各单元的质量控制点的值,提高全流程纱线质量。(The invention discloses a global process sharing control system and method based on ring spinning yarn quality, which comprises a cloud subsystem and an edge subsystem, wherein the cloud subsystem comprises an acquisition module, an evaluation module and a correction module, and the edge subsystem comprises a plurality of modules: the system comprises an Agent module, a state identification module, an action regulation module, a reward module and a CPPS unit, wherein the Agent module comprises a criticic network module, an Actor network module and an error module; the criticic network module is respectively connected with the state identification module, the reward module and the error module, the action adjustment module is connected with the state identification module through the CPPS unit, and the state identification module is connected with the reward module. The invention controls the quality in the spinning process, restrains the quality fluctuation in the following unit, and adjusts the value of the quality control point of each preceding unit by predicting the quality characteristic of the following unit, thereby improving the quality of the whole-process yarn.)

1. A global process sharing control system based on ring spinning yarn quality is characterized in that: the system comprises a cloud subsystem and an edge subsystem, wherein the cloud subsystem comprises: the system comprises an acquisition module, an evaluation module and a correction module, wherein the edge subsystem comprises a plurality of modules, and the modules comprise: agent module, state identification module, action adjustment module, reward module, CPPS unit, the Agent module includes: the device comprises a Critic network module, an Actor network module and an error module;

the CPPS unit is respectively connected with the state identification module and the action adjustment module, the state identification module is respectively connected with the reward module, the Critic network module and the Actor network module, the Actor network module is connected with the action adjustment module, the reward module is connected with the Critic network module, the Critic network module is connected with the Actor network module through the error module, the Critic network module is respectively connected with the acquisition module and the correction module in the cloud end, and the acquisition module in the cloud end is connected with the correction module through the evaluation module;

the CPPS unit can acquire a plurality of spinning quality characteristic data;

the state identification module is used for acquiring off-line knowledge and yarn quality characteristic data in the CPPS unit;

the Actor network module generates a regulation value of a quality control point based on the offline knowledge and the quality characteristics acquired by the state identification module and the error acquired by the error module;

the action adjusting module is used for adjusting and controlling the adjusting and controlling value of the quality control point generated by the Actor network module;

the reward module calculates a reward function to obtain reward based on the offline knowledge and the quality characteristics acquired by the state identification module.

2. The global process sharing control system based on ring spinning yarn quality as claimed in claim 1, characterized in that: the plurality of spinning quality characteristic data comprises: the sum of the human, machine, material, production process and production environment factors related to the yarn quality.

3. The global process sharing control system based on ring spinning yarn quality as claimed in claim 1, characterized in that: the reward function is used to suppress quality fluctuations of subsequent units.

4. The global process sharing control system based on ring spinning yarn quality as claimed in claim 1, characterized in that: in the adjustment of the value of the quality control point by the action adjustment module, the quality fluctuation in each subsequent unit is suppressed by adjusting the value of the quality control point in the previous unit, and the value of the quality control point in each previous unit is adjusted by predicting the quality characteristic in the unit.

5. The global process sharing control system based on ring spinning yarn quality as claimed in claim 1, characterized in that: the Agent module is constructed based on an Actor-Critic network.

6. The global process sharing control system based on ring spinning yarn quality as claimed in claim 1, characterized in that: the Critic network module and the Actor network module both adopt a BP neural network structure, the Actor network module comprises an Actor policy function module, and the Critic network module comprises a Critic value function module.

7. Method of global process sharing control system based on ring spinning yarn quality according to claims 1 to 6, characterized by: the method comprises the following specific steps:

s1, at S_tAt the moment, the state identification module acquires off-line knowledge and the number of the CPPS unitAfter the quality characteristic data of the dry yarn, the s is calculated_tThe offline knowledge and yarn quality characteristic data acquired at any moment are input into the rewarding module, and s is calculated_tAwarding the time of day by using the s_tInputting the offline knowledge and the yarn quality characteristic data acquired at any moment into a Critic value function module in the Critic network module to obtain s_tA function of time values;

s2, converting S in S1_tThe offline knowledge and yarn quality characteristic data obtained at any moment are input into an Actor network module, the regulation and control value of a quality control point is output through the Actor network module, the regulation and control value of the quality control point is input into the action regulation module to regulate the quality control point of the CPPS unit which is connected with the action regulation module, and the regulation and control value is s at the next moment after a certain time_t+1At the moment, the state identification module acquires the adjusted off-line knowledge and a plurality of yarn quality characteristic data in the CPPS unit, and inputs the adjusted off-line knowledge and the yarn quality characteristic data into a reward module to obtain s_t+1Inputting the adjusted off-line knowledge and the adjusted yarn quality characteristic data into a Critic value function module in the Critic network module to obtain s_t+1A function of time values;

s3, converting S in the S1_tTime value function and S in S2_t+1The time value function is simultaneously input into the error module to obtain an error value;

s4, updating the learning parameters in the Critic network module according to the error value to obtain Critic network parameters of the edge subsystem, and obtaining the Critic network parameters of the edge subsystem in the Critic network module in the edge subsystem through the obtaining module in the cloud end;

s5, evaluating the Critic network parameters of the edge subsystem acquired by the acquisition module through an evaluation module, inputting the Critic network parameters of the edge subsystem into a correction module after evaluation is finished, and updating the Critic network parameters of the edge subsystem through the correction module to obtain the Critic network parameters of the second cloud subsystem;

and S6, updating the Actor network parameters in the Actor network module.

8. The method of claim 7, wherein the global process sharing control system is based on the ring spinning yarn quality, and the method comprises the following steps: the evaluating in S5 includes: calculating the weight of the Critic network parameters in the Critic network modules in the edge subsystem, setting the target weight of the parameters according to the production complexity of the CPPS unit and the process requirement, evaluating whether the weight of the Critic network parameters in each Critic network module in the edge subsystem has deviation by using the target weight, correcting the deviation by using a correction module to obtain the Critic network parameters in each Critic network module in the cloud subsystem, and synchronously updating the Critic network parameters of the Critic network modules in each edge subsystem.

9. The method of claim 7, wherein the global process sharing control system is based on the ring spinning yarn quality, and the method comprises the following steps: the method for updating the Actor network parameter in the Actor network module in S6 is as follows: and updating by adopting a strategy gradient descending method.

Technical Field

The invention relates to the field of ring spinning, in particular to a global process sharing control system and method based on the quality of a ring spinning yarn.

Background

The ring spinning technology is widely applied at home and abroad at present, has simple mechanism and convenient change of process parameters, and can update and replace various related components in time according to the spinning process requirements so as to adapt to different spinning requirements; the machine equipment can be conveniently maintained; the operation is convenient, the raw material adaptability is wide, natural fibers and chemical fibers can be basically subjected to pure spinning and blended spinning by ring spinning, and the limitation on fiber processing is small.

The ring spinning technology is widely applied, and simultaneously, higher requirements are provided for the spinning quality, the quality control is an important link of the ring spinning production, and how to better finish the quality control becomes an urgent task in the prior art. In the spinning process, the quality of the unit in the previous process directly influences the quality of the unit in the next process and the quality of the final yarn, and even influences the use performance of the fabric. Therefore, the yarn quality is not only the coupling between the quality control points of the adjacent processes, but also the strong coupling of the global yarn quality control points, namely the quality control points of the previous process are regulated to inhibit the quality fluctuation of the next process, and the quality control points of the previous process are reversely regulated by predicting the quality characteristics of the next process. With the development of ring spinning production towards intellectualization, quality improvement and individuation, the requirement of yarn quality control is higher and higher. In the ring spinning CPPS, the control of the yarn quality is no longer performed in different metering departments, but in real time on the individual steps of the ring spinning line. In the quality control of ring spun yarn CPPS yarn, although a large amount of off-line knowledge data including process design data, equipment data, product quality data and the like are accumulated as a basis, the method also faces challenges: how to establish a global process sharing control system based on the quality of a ring spinning yarn, so that the quality control in the spinning process is more comprehensive and systematic.

Disclosure of Invention

The invention aims to provide a system and a method for global process sharing control based on ring spinning yarns, which are used for solving the problems in the prior art, realizing the quality control of the yarns in the whole spinning process and enabling the quality control to be more comprehensive and systematized.

In order to achieve the purpose, the invention provides the following scheme:

a global process sharing control system based on ring spinning yarn quality comprises a cloud subsystem and an edge subsystem, wherein the cloud subsystem comprises: the system comprises an acquisition module, an evaluation module and a correction module, wherein the edge subsystem comprises a plurality of modules, and the modules comprise: agent module, state identification module, action adjustment module, reward module, CPPS unit, the Agent module includes: the device comprises a Critic network module, an Actor network module and an error module;

the CPPS unit is respectively connected with the state identification module and the action adjustment module, the state identification module is respectively connected with the reward module, the Critic network module and the Actor network module, the Actor network module is connected with the action adjustment module, the reward module is connected with the Critic network module, the Critic network module is connected with the Actor network module through the error module, the Critic network module is respectively connected with the acquisition module and the correction module in the cloud end, and the acquisition module in the cloud end is connected with the correction module through the evaluation module;

the CPPS unit can acquire a plurality of spinning quality characteristic data;

the state identification module is used for acquiring off-line knowledge and yarn quality characteristic data in the CPPS unit;

the Actor network module generates a regulation value of a quality control point based on the offline knowledge and the quality characteristics acquired by the state identification module and the error acquired by the error module;

the action adjusting module is used for adjusting and controlling the adjusting and controlling value of the quality control point generated by the Actor network module;

the reward module calculates a reward function to obtain reward based on the offline knowledge and the quality characteristics acquired by the state identification module.

Further, the plurality of spinning quality characteristic data comprises: the sum of the human, machine, material, production process and production environment factors related to the yarn quality.

Further, the reward function is used to suppress quality fluctuations of subsequent units.

Further, in the adjustment of the value of the quality control point by the motion adjustment module, the fluctuation of quality in each subsequent unit is suppressed by adjusting the value of the quality control point in the previous unit, and the value of the quality control point in each previous unit is adjusted by predicting the quality characteristic in the subsequent unit.

Further, the Agent module is constructed based on an Actor-Critic network.

Further, the critical network module and the Actor network module both adopt a BP neural network structure, the Actor network module includes an Actor policy function module, and the critical network module includes a critical value function module.

A global process sharing control method based on ring spinning yarn quality is characterized in that: the method comprises the following specific steps:

s1, at S_tAt the moment, after the state identification module acquires off-line knowledge and a plurality of yarn quality characteristic data in the CPPS unit, the state identification module compares the s with the yarn quality characteristic data_tThe offline knowledge and yarn quality characteristic data acquired at any moment are input into the rewarding module, and s is calculated_tAwarding the time of day by using the s_tInputting the offline knowledge and the yarn quality characteristic data acquired at any moment into a Critic value function module in the Critic network module to obtain s_tA function of time values;

s2, converting S in S1_tThe offline knowledge and yarn quality characteristic data obtained at any moment are input into an Actor network module, the regulation and control value of a quality control point is output through the Actor network module, the regulation and control value of the quality control point is input into the action regulation module to regulate the quality control point of the CPPS unit which is connected with the action regulation module, and the regulation and control value is s at the next moment after a certain time_t+1At the moment, the state identification module acquires the adjusted off-line knowledge and a plurality of yarn quality characteristic data in the CPPS unit, and inputs the adjusted off-line knowledge and the yarn quality characteristic data into a reward module to obtain s_t+1Awarding time and time, and obtaining the adjusted off-line knowledge and yarn quality characteristic dataInputting the obtained data into a Critic value function module in the Critic network module to obtain s_t+1A function of time values;

s3, converting S in the S1_tTime value function and S in S2_t+1The time value function is simultaneously input into the error module to obtain an error value;

s4, updating the learning parameters in the Critic network module according to the error value to obtain Critic network parameters of the edge subsystem, and obtaining the Critic network parameters of the edge subsystem in the Critic network module in the edge subsystem through the obtaining module in the cloud end;

s5, evaluating the Critic network parameters of the edge subsystem acquired by the acquisition module through an evaluation module, inputting the Critic network parameters of the edge subsystem into a correction module after evaluation is finished, and updating the Critic network parameters of the edge subsystem through the correction module to obtain the Critic network parameters of the second cloud subsystem;

and S6, updating the Actor network parameters in the Actor network module.

Further, the evaluating in S5 includes: calculating the weight of the Critic network parameters in the Critic network modules in the edge subsystem, setting the target weight of the parameters according to the production complexity of the CPPS unit and the process requirement, evaluating whether the weight of the Critic network parameters in each Critic network module in the edge subsystem has deviation by using the target weight, correcting the deviation by using a correction module to obtain the Critic network parameters in each Critic network module in the cloud subsystem, and synchronously updating the Critic network parameters of the Critic network modules in each edge subsystem.

Further, the method for updating the Actor network parameter in the Actor network module in S6 is as follows: and updating by adopting a strategy gradient descending method.

The invention discloses the following technical effects:

the invention provides a global process sharing control system based on the quality of a ring spun yarn, which improves the strong coupling among global control points and the quality control effect of the system by carrying out strong coupling on the global yarn quality control points. The method can control the quality in the spinning process, not only can inhibit the quality fluctuation in the following unit by adjusting the value of the quality control point of the preceding unit, but also can predict the quality characteristic of the following unit, and conversely, the value of the quality control point of the preceding unit is adjusted, thereby improving the quality of the ring spun yarn.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

FIG. 1 is a schematic diagram of a global process sharing control system based on ring spinning quality.

Detailed Description

Reference will now be made in detail to various exemplary embodiments of the invention, the detailed description should not be construed as limiting the invention but as a more detailed description of certain aspects, features and embodiments of the invention.

It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Further, for numerical ranges in this disclosure, it is understood that each intervening value, between the upper and lower limit of that range, is also specifically disclosed. Every smaller range between any stated value or intervening value in a stated range and any other stated or intervening value in a stated range is encompassed within the invention. The upper and lower limits of these smaller ranges may independently be included or excluded in the range.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although only preferred methods and materials are described herein, any methods and materials similar or equivalent to those described herein can be used in the practice or testing of the present invention. All documents mentioned in this specification are incorporated by reference herein for the purpose of disclosing and describing the methods and/or materials associated with the documents. In case of conflict with any incorporated document, the present specification will control.

It will be apparent to those skilled in the art that various modifications and variations can be made in the specific embodiments of the present disclosure without departing from the scope or spirit of the disclosure. Other embodiments will be apparent to those skilled in the art from consideration of the specification. The specification and examples are exemplary only.

As used herein, the terms "comprising," "including," "having," "containing," and the like are open-ended terms that mean including, but not limited to.

The "parts" in the present invention are all parts by mass unless otherwise specified.

A global process sharing control system based on ring spinning yarn quality comprises a cloud subsystem and an edge subsystem, wherein the cloud subsystem comprises: the system comprises an acquisition module, an evaluation module and a correction module, wherein the edge subsystem comprises a plurality of modules, and the modules comprise: agent module, state identification module, action adjustment module, reward module, CPPS unit, the Agent module includes: the device comprises a Critic network module, an Actor network module and an error module;

the CPPS unit is respectively connected with the state identification module and the action adjustment module, the state identification module is respectively connected with the reward module, the Critic network module and the Actor network module, the Actor network module is connected with the action adjustment module, the reward module is connected with the Critic network module, the Critic network module is connected with the Actor network module through the error module, the Critic network module is respectively connected with the acquisition module and the correction module in the cloud end, and the acquisition module in the cloud end is connected with the correction module through the evaluation module;

the CPPS unit can acquire a plurality of spinning quality characteristic data;

the state identification module is used for acquiring off-line knowledge and yarn quality characteristic data in the CPPS unit;

the Actor network module generates a regulation value of a quality control point based on the offline knowledge and the quality characteristics acquired by the state identification module and the error acquired by the error module;

the action adjusting module is used for adjusting and controlling the adjusting and controlling value of the quality control point generated by the Actor network module;

the reward module calculates a reward function to obtain reward based on the offline knowledge and the quality characteristics acquired by the state identification module.

A global process sharing control method based on ring spinning yarn quality is characterized in that: the method comprises the following specific steps:

s1, at S_tAt the moment, after the state identification module acquires off-line knowledge and a plurality of yarn quality characteristic data in the CPPS unit, the state identification module compares the s with the yarn quality characteristic data_tThe offline knowledge and yarn quality characteristic data acquired at any moment are input into the rewarding module, and s is calculated_tAwarding the time of day by using the s_tInputting the offline knowledge and the yarn quality characteristic data acquired at any moment into a Critic value function module in the Critic network module to obtain s_tA function of time values;

s2, converting S in S1_tThe offline knowledge and yarn quality characteristic data obtained at any moment are input into an Actor network module, the regulation and control value of a quality control point is output through the Actor network module, the regulation and control value of the quality control point is input into the action regulation module to regulate the quality control point of the CPPS unit which is connected with the action regulation module, and the regulation and control value is s at the next moment after a certain time_t+1At the moment, the state identification module acquires the adjusted off-line knowledge and a plurality of yarn quality characteristic data in the CPPS unit, and inputs the adjusted off-line knowledge and the yarn quality characteristic data into a reward module to obtain s_t+1Inputting the adjusted off-line knowledge and the adjusted yarn quality characteristic data into a Critic value function module in the Critic network module to obtain s_t+1Function of time valueCounting;

s3, converting S in the S1_tTime value function and S in S2_t+1The time value function is simultaneously input into the error module to obtain an error value;

s4, updating the learning parameters in the Critic network module according to the error value to obtain Critic network parameters of the edge subsystem, and obtaining the Critic network parameters of the edge subsystem in the Critic network module in the edge subsystem through the obtaining module in the cloud end;

s5, evaluating the Critic network parameters of the edge subsystem acquired by the acquisition module through an evaluation module, inputting the Critic network parameters of the edge subsystem into a correction module after evaluation is finished, and updating the Critic network parameters of the edge subsystem through the correction module to obtain the Critic network parameters of the second cloud subsystem;

and S6, updating the Actor network parameters in the Actor network module.

A plurality of CPPS units in the ring spinning CPPS are as follows:

S＝{u₁，u₂...，u_i，…，u_n}

wherein S represents a ring spun yarn CPPS, u_iRepresents the ith CPPS unit, i ∈ [1, n [ ]]And n is the number of CPPS units.

The quality control points are as follows:represents the jth quality control point of the ith CPPS unit, where i ∈ [1, n ]]，j∈[1，ni],n_iThe number of the quality control points of the ith CPPS unit is shown, so that the total number m of the CPPS quality control points of the ring spinning is as follows:

as known from ring spinning production characteristics, the control value of each quality control point is adjusted within a certain range along with quality fluctuation. Adjustable at each point of quality controlMinimum value ofThe maximum value which can be adjusted isIn ring spinning CPPS, the yarn is processed by a plurality of different ring spinning CPPS units, each unit has different quality characteristics to represent the quality of the yarn of the unit, and the yarn quality characteristics are defined as follows:

the yarn has the quality characteristics that:represents the l-th quality characteristic of the i-th ring spinning CPPS unit, wherein: i is an e [1, n ]]，l∈[1，l_i]，l_iThe number of mass characteristics of the ith ring spun yarn CPPS unit is shown. For a certain ring spinning CPPS unit, the quality control point is adjusted within a certain range according to the quality characteristic value, namely:however, in actual production, after the control point acts on the ring spinning CPPS unit due to machine failure, temperature and humidity changes and the like, the quality of products is attenuated to different degrees, such as strength reduction, hairiness increase, yarn evenness increase and the like, namely the quality control pointActing on ring spinning CPPS units only in practiceProducing an effect. When in useThe extent of the adjustment of the overflow control point, or the obvious tendency to overflow, can affect the production of the subsequent units and even cause problems in the final yarn quality. Thus, for multi-process step ring spinningThe physical production system considers the compensation of each quality control point by adopting a multi-load attenuation factor so as to inhibit the possible large fluctuation of the yarn quality in the subsequent manufacturing.

Error calculation in the error module:

because of the excellent performance of Actor-critical learning in task control, the application introduces the idea thereof into the yarn quality control of ring spinning CPPS. Firstly, mapping factors such as Agent, state, environment, action and reward to a global process sharing control system based on the ring spinning quality, wherein the specific definitions are as follows:

(1) agent

The Agent is a quality controller of each unit and system of the ring spinning CPPS, and is mainly used for interacting with relevant factors of spinning quality. The Agent formulates a strategy to adjust the control value of the yarn quality control point according to the received reward, simultaneously evaluates the yarn quality characteristics and provides a flexible control strategy, and the Agent is the quality controller in the application.

(2) Status of state

For the quality control of ring spinning CPPS, the state is the quality characteristics of the yarn, such as the strength, uneven evenness, hairiness and the like of the yarn. The quality features have different physical meanings and influence the learning efficiency, and therefore, the normalization method is adopted to process all physical quantities and normalize all dimensions of the state to the same range. At time t, the yarn quality characteristics of the ith ring spinning CPPS unit, i.e., the state thereof, can be expressed as:

whereinShows the quality characteristics l, l of the ith ring spun CPPS unit yarn_iAnd the number of yarn quality characteristics of the ith ring spinning CPPS unit is shown, and the state is the state identification module in the application.

(3) Environment(s)

The environment refers to the sum of human, machine, material, method, ring and other factors related to the yarn quality in all the process units of the ring spinning CPPS.

(4) Movement of

The action is an action adjustment module of the application, and the action is to map the state space to the action space and determine the optimal adjustment strategy. The action of the ring spinning CPPS is to reasonably adjust the value of the quality control point according to the current yarn characteristic observed value. For example, quality control points such as the twist and tension of the yarn or the spindle speed of the filament yarn vehicle are selected and adjusted according to the strength fluctuation of the yarn.

At the time t, the value of each control point is adjusted by the Agent corresponding to the ith ring spinning CPPS unit, namely, one execution action is represented as:

wherein the content of the first and second substances,denotes the jth quality control point, n, of the ith ring spinning CPPS unit_iShows the number of quality control points of the ith ring spinning CPPS unit.

The strategy distribution of the action directly influences the performance of the algorithm. The strategy distribution in this chapter is normal distribution, and the probability density function is:

(5) reward

And reasonably adjusting the current yarn processing quality and the quality condition, the Agent can obtain rewards from the feedback of the environment, and the rewards are reward modules of the application.

At time t, the yarn quality produced by the ith CPPS unit is characterized by s_tWhen, the Agent performs action a_tThe yarn quality characteristic of the ith process unit of the ring spinning CPPS is s_t+1All are the same asWhile receiving a reward r from the environment_t。r_t＝r(s_t，a_t) Is a function of the yarn quality characteristics and quality control action. In this case, the value function of the quality characteristic s represents the expectation that the reward will be obtained after adjusting the quality control action a under the quality characteristic s, and can be defined as:

where γ is the discount factor.

The state-action value function Q (s, a) represents how well the quality control action a is selected for adjustment under the quality characteristic s, compared to the state-action value function which takes into account the effect of performing the action a, i.e. the function Q (s, a)

It can be seen that either the state value function or the state-action value function is a desire to accumulate rewards.

A global process sharing control system based on the quality of a ring spinning yarn is as follows:

the preceding procedure of ring spinning production not only directly influences the yarn quality of the procedure, but also influences the yarn quality of the adjacent subsequent procedures and even the final yarn quality. Therefore, the yarn quality is not only the coupling between the quality control points of the adjacent processes, but also the strong coupling of the global yarn quality control points, namely the quality control points of the previous process are regulated to inhibit the quality fluctuation of the next process, and the quality control points of the previous process are reversely regulated by predicting the quality characteristics of the next process.

The global process sharing control system based on the ring spinning yarn quality is shown in figure 1. In this system, the criticic network of each agent of the edge subsystem is mapped to the cloud subsystem over the communications network. Firstly, acquiring a criticic network parameter of each agent; then, calculating the weight of the Critic network parameters, and evaluating the deviation between the actual values and the target values of the parameters; and finally, correcting the deviation of each Critic network parameter, and compensating and updating the Critic network parameter.

(1) Reward function design in a global process sharing control system based on the ring spinning quality, the reward function shown below is designed.

Wherein, U (| a)_tI) and L (| a)_t|) represents the quality control point a of all the yarns in one motion respectively_tUpper and lower limit values of a^*Represents the optimum control value, s_tAndrespectively representing the current yarn quality characteristic and the current yarn quality optimum characteristic, beta_ijzIs the current cell quality control pointAttenuation factor for subsequent cells.

(2) Error calculation in the error module:

the Actor network and Critic network in each AC learn a policy function and a value function using a Time Difference (TD) error method. At time t, the current yarn processing quality characteristic s_tThe TD error delta is calculated by the following formula_t：

δ_t＝r_t+1+γv(s_t+1)-v(s_t)

Wherein r is_t+1Representing the current yarn quality characteristic s_tThen, a quality control action a is executed_tThen, the next quality feature s is reached_t+1The instantaneous return obtained, gamma is the discount factor. Thus, the error function can be defined as:

(3) network design

In the control system, an Actor network and a criticic network in the AC structure of each CPPS unit corresponding Agent adopt a BP neural network structure.

The Actor network adopts a multi-input multi-output structure, and the input of the subsequent unit in any two adjacent units is the off-line knowledge and yarn quality observation value of the unitThe number of nodes at the input end is the dimension of off-line knowledge and the on-line quality characteristic number l_iThe sum of (1). The output is the regulation value of the yarn quality control point of the unitThe number of the output end nodes is the number n of the quality control points_iWherein theta_iIs the learning parameter of the Actor network.

The Critic network adopts a multi-input single-output structure, and a value function v (sp) of a yarn quality observation value of a preorder unit is added to the input of the Actor network_t；ω_i-1). Outputting a value function v(s) as a yarn quality observation for the cell_t；ω_i) The output end has only one output node, where_i-1And ω_iLearning parameters of the criticic network of the previous unit and the subsequent unit are respectively.

(4) Network learning and optimization

In the control system, each unit of the edge subsystem determines the status of the production unit by using a combination of yarn quality on-line inspection data and off-line knowledge. The policy function and the value function in the AC of the current cell use the parameter theta respectively_iAnd ω_iIs expressed as σ_i1And σ_i2Respectively representing the learning step lengths, and the method for updating the parameters of each Critic network is as follows:

the Critic network parameter ω_i：

An acquirer in the cloud subsystem acquires parameters omega of each Critic network of the edge subsystem through the industrial internet technology_i。

For the evaluator, each ω is first calculated by the following formula_iWeight of (2)

According to the complexity and the technological requirements of the production of each unit of the ring spinning, the target weight of each technology is set to beSecondly, byEvaluation of eachThe deviation of (c) is adjusted and updated by the following formula_i。

Wherein epsilon_i(ε_i> 0) indicates that each cell corresponds to an upper limit value of the allowable fluctuation of the parameter in the criticc network.

And finally, transmitting each updated parameter value to a corresponding criticic network of the edge subsystem.

The Actor network parameter θ_i：

The above-described embodiments are merely illustrative of the preferred embodiments of the present invention, and do not limit the scope of the present invention, and various modifications and improvements of the technical solutions of the present invention can be made by those skilled in the art without departing from the spirit of the present invention, and the technical solutions of the present invention are within the scope of the present invention defined by the claims.