阅读说明：本技术 公共策略决策方法、装置、电子设备和存储介质 (Public policy decision method, device, electronic equipment and storage medium ) 是由 谢静文 阮晓雯 于 2021-07-28 设计创作，主要内容包括：本申请涉及人工智能技术领域,具体公开了一种公共策略决策方法、装置、电子设备和存储介质,其中,公共策略决策方法包括：根据传染病的传染参数建立传染病的传染模型；构建模拟城市群,并根据模拟城市群的参数,建立模拟城市群中的城市节点之间的人口流动模型；根据传染模型和人口流动模型,建立第一城市节点的感染人数增量模型；根据预设的封城策略和感染人数增量模型,进行数据仿真,生成仿真数据；将模拟数据输入预设的强化学习模型进行训练,得到决策模型；获取待决策城市以及第一范围内的城市的传染数据,其中,第一范围由待决策城市的位置确定；将传染数据输入决策模型,得到决策结果。(The application relates to the technical field of artificial intelligence, and particularly discloses a public policy decision method, a device, electronic equipment and a storage medium, wherein the public policy decision method comprises the following steps: establishing an infection model of the infectious disease according to the infection parameters of the infectious disease; establishing a simulation urban group, and establishing a population flow model among urban nodes in the simulation urban group according to parameters of the simulation urban group; establishing an infected person number increment model of the first city node according to the infection model and the population mobility model; performing data simulation according to a preset city-closing strategy and an infected person incremental model to generate simulation data; inputting simulation data into a preset reinforcement learning model for training to obtain a decision model; acquiring infection data of a city to be decided and cities in a first range, wherein the first range is determined by the position of the city to be decided; and inputting the infection data into the decision model to obtain a decision result.)

1. A method for public policy decision making, the method comprising:

establishing an infection model of the infectious disease according to infection parameters of the infectious disease, wherein the infection model is used for identifying the spreading and development rules of the infectious disease in the crowd;

establishing a simulation urban group, and establishing a population flow model among urban nodes in the simulation urban group according to parameters of the simulation urban group, wherein the simulation urban group comprises at least two urban nodes;

establishing an infected person number increment model of a first city node according to the infection model and the population flow model, wherein the first city node is any one city node in the simulated city group, and the infected person number increment model is used for identifying the rule of the infected person number which is increased every day by the first city node;

performing data simulation according to a preset city-closing strategy and the infected person number increment model to generate simulation data;

inputting the simulation data into a preset reinforcement learning model for training to obtain a decision model;

acquiring infection data of a city to be decided and cities in a first range, wherein the first range is determined by the position of the city to be decided;

and inputting the infection data into the decision model to obtain a decision result.

2. The method of claim 1, wherein said building an incremental model of the population of infestations for a first city node based on said infection model and said population flow model comprises:

establishing a state transformation model of the infectious disease according to the infectious disease model, wherein the state transformation model is used for identifying transformation rules among different states in the infectious disease;

establishing an internal incremental model according to the state transition model, wherein the internal incremental model is used for identifying the rule of the number of infectious people growing in each day of the first city node due to the infectious disease spreading inside the first city node;

establishing an external incremental model according to the population flow model, wherein the external incremental model is used for identifying the rule of the number of infected persons of the first city node growing every day caused by the migration of the population among the first city nodes;

determining a state type of the first city node, and determining the incremental infective person number model according to the state type, the internal incremental model and the external incremental model, wherein the state type comprises: an open state and a closed state, wherein when the state type of the first city node is the open state, the infected person number incremental model comprises the internal incremental model and the external incremental model; and when the state type of the first city node is a closed state, the infected person number incremental model is the internal incremental model.

3. The method of claim 2, wherein building an internal incremental model from the state transition model comprises:

determining an infection rate of the infectious disease of the first city node according to the state of the first city node;

determining a number of people in the first city node that are infected and in latent phase, a number of people that are infected and asymptomatic, and a number of people that are infected and symptomatic, according to the infection model and the state transition model;

establishing the internal increment model according to the infection rate of the infectious disease of the first city node, the number of people infected and in latent stage, the number of people infected and asymptomatic, and the number of people infected and symptomatic.

4. The method of claim 2, wherein building an external incremental model based on the population flow model comprises:

determining the contact strength between the first city node and a second city node according to the population flow model, wherein the second city node is any one of the simulated city groups different from the first city node;

determining an infection rate of the infectious disease of the first city node according to the state of the first city node;

determining a state value of the second city node according to a preset state rule and the state of the second city node;

determining the population number of the second city node according to the population mobility model;

and establishing the external incremental model according to the contact strength between the first city node and a second city node, the infection rate of the infectious diseases of the first city node, the state value of the second city node and the population number of the second city node.

5. The method of claim 4, wherein determining the strength of contact between the first city node and the second city node according to the population flow model comprises:

determining the population number of the first city node according to the population mobility model;

determining a distance between the first city node and the second city node according to the population mobility model;

and taking the quotient of the product of the population number of the first city node and the population number of the second city node and the square root of the distance between the first city node and the second city node as the strength of the connection between the first city node and the second city node.

6. The method according to any one of claims 1-5, wherein said establishing an infection model of an infectious disease based on infection parameters of said infectious disease comprises:

analyzing the infectious disease to determine at least one type of status of the person in the context of the infectious disease;

determining a conversion relationship between each of the at least one state type;

determining a conversion rate between each of said status types based on said infection parameters;

establishing an infection model of the infectious disease according to a transformation relation between each state type in the at least one state type and a transformation rate between each state type.

7. The method according to any one of claims 1 to 5, wherein the performing data simulation according to the preset city-closing strategy and the incremental infectious population model to generate simulation data comprises:

determining input features of the reinforcement learning model, and determining the output data type of the data simulation according to the input features;

determining a reward function of the reinforcement learning model;

performing data simulation according to the reward function of the reinforcement learning model, the parameters of the simulated urban group, the city closing strategy and the infected person number increment model to obtain initial data;

and screening the initial data according to the type of the output data to obtain the simulation data.

8. A public policy decision device, the device comprising:

the model building module is used for building an infection model of the infectious disease according to infection parameters of the infectious disease, wherein the infection model is used for identifying the transmission and development rules of the infectious disease in the crowd;

the system comprises an environment establishing module, a parameter calculating module and a parameter setting module, wherein the environment establishing module is used for establishing a simulation urban group and establishing a population flow model among urban nodes in the simulation urban group according to parameters of the simulation urban group, and the simulation urban group comprises at least two urban nodes;

the model establishing module is further used for establishing an infected person number increment model of a first city node according to the infection model and the population flow model, wherein the first city node is any one of the simulated city groups, and the infected person number increment model is used for identifying the rule of the infected person number which is increased every day by the first city node;

the simulation module is used for carrying out data simulation according to a preset city-closing strategy and the infected person number increment model to generate simulation data;

the training module is used for inputting the simulation data into a preset reinforcement learning model for training to obtain a decision model;

the system comprises an acquisition module, a decision-making module and a decision-making module, wherein the acquisition module is used for acquiring infection data of a city to be decided and a city within a first range, and the first range is determined by the position of the city to be decided;

and the decision module is used for inputting the infection data into the decision model to obtain a decision result.

9. An electronic device comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, the one or more programs including instructions for performing the steps in the method of any of claims 1-7.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which is executed by a processor to implement the method according to any one of claims 1-7.

Technical Field

The invention relates to the technical field of artificial intelligence, in particular to a public strategy decision method, a public strategy decision device, electronic equipment and a storage medium.

Background

When aiming at a great emergent public event, such as an emergent medical event, due to the lack of experience of similar events or sudden events, the decision maker is difficult to make a decision for a high-risk strategy in a short time.

At present, an analog simulation system is often used to perform data simulation in response to the above situations, and then simulated data is used to perform analysis to assist decision making. However, the conventional simulation system focuses on data transfer simulation, such as: the number of patients in a future period is simulated according to the infection coefficient of infectious diseases and the population mobility state. Such simulations, while possibly supplementing the missing information, are more difficult to make than quantitative aid decision suggestions. Namely, the current common simulation system lacks an objective function and cannot perform a reverse optimization process. In the face of a black swan event such as a major sudden public event, conditions such as attention points and experience prejudgment of experts are different, and macroscopic conditions may change instantly, so that a decision model which can provide decision suggestions timely and is high in interpretability is established, and the problem to be solved urgently is solved.

Disclosure of Invention

In order to solve the above problems in the prior art, embodiments of the present application provide a public policy decision method, an apparatus, an electronic device, and a storage medium, which can provide a decision suggestion with strong interpretability and reasonable reliability for a major public emergency.

In a first aspect, an embodiment of the present application provides a public policy decision method, including:

establishing an infection model of the infectious disease according to infection parameters of the infectious disease, wherein the infection model is used for identifying the transmission and development rules of the infectious disease in the crowd;

establishing a simulated urban group, and establishing a population flow model among urban nodes in the simulated urban group according to parameters of the simulated urban group, wherein the simulated urban group comprises at least two urban nodes;

establishing an infected person number increment model of a first city node according to the infection model and the population flow model, wherein the first city node is any one of simulated city groups, and the infected person number increment model is used for identifying the rule of the infected person number which is increased every day by the first city node;

performing data simulation according to a preset city-closing strategy and an infected person incremental model to generate simulation data;

inputting simulation data into a preset reinforcement learning model for training to obtain a decision model;

acquiring infection data of a city to be decided and cities in a first range, wherein the first range is determined by the position of the city to be decided;

and inputting the infection data into the decision model to obtain a decision result.

In a second aspect, an embodiment of the present application provides a common policy decision apparatus, including:

the model building module is used for building an infection model of the infectious disease according to the infection parameters of the infectious disease, wherein the infection model is used for identifying the transmission and development rules of the infectious disease in the crowd;

the environment establishing module is used for establishing a simulation urban group and establishing a population flow model among urban nodes in the simulation urban group according to parameters of the simulation urban group, wherein the simulation urban group comprises at least two urban nodes;

the model establishing module is further used for establishing an infected person number increment model of the first city node according to the infection model and the population flow model, wherein the first city node is any one of the simulated city groups, and the infected person number increment model is used for identifying the rule of the infected person number which is increased every day by the first city node;

the simulation module is used for carrying out data simulation according to a preset city-closing strategy and the infected person number increment model to generate simulation data;

the training module is used for inputting simulation data into a preset reinforcement learning model for training to obtain a decision model;

the system comprises an acquisition module, a decision-making module and a decision-making module, wherein the acquisition module is used for acquiring the infection data of a city to be decided and a city within a first range, and the first range is determined by the position of the city to be decided;

and the decision module is used for inputting the infection data into the decision model to obtain a decision result.

In a third aspect, an embodiment of the present application provides an electronic device, including: a processor coupled to the memory, the memory for storing a computer program, the processor for executing the computer program stored in the memory to cause the electronic device to perform the method of the first aspect.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium having a computer program stored thereon, the computer program causing a computer to perform the method according to the first aspect.

In a fifth aspect, embodiments of the present application provide a computer program product comprising a non-transitory computer readable storage medium storing a computer program, the computer operable to cause the computer to perform a method according to the first aspect.

The implementation of the embodiment of the application has the following beneficial effects:

in the embodiment of the application, the spreading characteristics of infectious diseases and the population flow characteristics among cities are analyzed under the condition of sudden infectious diseases, an infectious disease model and a population flow model among city nodes are constructed, and then an infected person number increment model reflecting the rule of the infected person number growing every day of one city node is constructed according to the infectious disease model and the population flow model. And then, performing data simulation according to a preset city-closing strategy and an infected person number increment model to obtain a large amount of simulation data to train the reinforcement learning model to obtain a decision model. And finally, inputting the actual data into a decision model to obtain a corresponding decision suggestion. Therefore, through data simulation, under the condition that no historical data can be used for reference, a large amount of simulation data is obtained, powerful data support is provided for decision making, and black swan events can be effectively responded. Meanwhile, the model for executing the data simulation is a model generated by analyzing the propagation characteristics of the infectious diseases and the population flow characteristics among cities, and the simulation reality is high, so that the rationality of decision can be further improved. Finally, the output result of the reinforcement learning model can be analyzed and traced, the deep reinforcement learning is different from the state of a pure deep neural network black box, the given decision is optimized through a reward function set by a user, and therefore the result is reasonable and interpretable.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

Fig. 1 is a schematic hardware structure diagram of a public policy decision device according to an embodiment of the present disclosure;

fig. 2 is a schematic flowchart of a public policy decision method according to an embodiment of the present disclosure;

FIG. 3 is a schematic flow chart illustrating a method for establishing an infection model of an infectious disease according to an infection parameter of the infectious disease according to an embodiment of the present application;

FIG. 4 is a schematic view of an infection model provided by embodiments of the present application;

FIG. 5 is a schematic diagram of a model for simulating population movement between city nodes in a city group according to an embodiment of the present disclosure;

fig. 6 is a schematic flowchart of a method for establishing an infection population incremental model of a first city node according to an infection model and a population mobility model according to an embodiment of the present application;

fig. 7 is a schematic flow chart of a method for performing data simulation according to a preset city-closing strategy and an infected person number incremental model to generate simulation data according to the embodiment of the present application;

fig. 8 is a block diagram illustrating functional modules of a public policy decision device according to an embodiment of the present disclosure;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments obtained by a person of ordinary skill in the art without any inventive work based on the embodiments in the present application are within the scope of protection of the present application.

The terms "first," "second," "third," and "fourth," etc. in the description and claims of this application and in the accompanying drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

Reference herein to "an embodiment" means that a particular feature, result, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.

First, referring to fig. 1, fig. 1 is a schematic diagram of a hardware structure of a common policy decision device according to an embodiment of the present disclosure. The common policy decision device 100 comprises at least one processor 101, a communication line 102, a memory 103 and at least one communication interface 104.

In this embodiment, the processor 101 may be a general processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more ics for controlling the execution of programs according to the present disclosure.

The communication link 102, which may include a path, carries information between the aforementioned components.

The communication interface 104 may be any transceiver or other device (e.g., an antenna, etc.) for communicating with other devices or communication networks, such as an ethernet, RAN, Wireless Local Area Network (WLAN), etc.

The memory 103 may be, but is not limited to, a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a Random Access Memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, optical disk storage (including compact disc, laser disc, optical disc, digital versatile disc, blu-ray disc, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

In this embodiment, the memory 103 may be independent and connected to the processor 101 through the communication line 102. The memory 103 may also be integrated with the processor 101. The memory 103 provided in the embodiments of the present application may generally have a nonvolatile property. The memory 103 is used for storing computer-executable instructions for executing the scheme of the application, and is controlled by the processor 101 to execute. The processor 101 is configured to execute computer-executable instructions stored in the memory 103, thereby implementing the methods provided in the embodiments of the present application described below.

In alternative embodiments, computer-executable instructions may also be referred to as application code, which is not specifically limited in this application.

In alternative embodiments, processor 101 may include one or more CPUs, such as CPU0 and CPU1 of FIG. 1.

In alternative embodiments, the common policy decision device 100 may include multiple processors, such as processor 101 and processor 107 in FIG. 1. Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (e.g., computer program instructions).

In an alternative embodiment, if the public policy decision apparatus 100 is a server, the public policy decision apparatus 100 may further include an output device 105 and an input device 106. The output device 105 is in communication with the processor 101 and may display information in a variety of ways. For example, the output device 105 may be a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display device, a Cathode Ray Tube (CRT) display device, a projector (projector), or the like. The input device 106 is in communication with the processor 101 and may receive user input in a variety of ways. For example, the input device 106 may be a mouse, a keyboard, a touch screen device, or a sensing device, among others.

The common policy decision device 100 may be a general-purpose device or a special-purpose device. The embodiment of the present application does not limit the type of the common policy decision device 100.

Secondly, it should be noted that the common policy decision method provided by the present application can be applied to various major emergencies, such as: and (4) carrying out policy decision making in major emergencies such as whether to close a city under infectious diseases, whether to discharge flood under flood, whether to evacuate under storm and the like. The process of the public strategy decision method is mainly described by taking the scene of whether the city is closed or not under the infectious disease as an example. The public policy decision method in other scenarios is similar to the public policy decision method in the case of whether to close a city under an infectious disease, and details are not repeated here.

Hereinafter, a common policy decision method disclosed in the present application will be explained:

referring to fig. 2, fig. 2 is a schematic flowchart of a public policy decision method according to an embodiment of the present disclosure. The public policy decision method comprises the following steps:

201: and establishing an infection model of the infectious disease according to the infection parameters of the infectious disease.

In this embodiment, the infection model is used to identify the transmission and development rules of infectious diseases in the population. Illustratively, the present application provides a method for establishing an infection model of an infectious disease according to an infection parameter of the infectious disease, as shown in fig. 3, the method comprising:

301: an infection is analyzed to determine at least one type of condition of the person in an infectious disease environment.

Generally, in a disease environment, due to the nature of the disease, a population often presents a number of different types of states before and after the disease, such as: susceptible state, infected state, dead state, recovered state, etc. In this embodiment, at least one status type present in a population in the environment may be determined by analyzing historical infection data for the infectious disease.

Specifically, in the present embodiment, after analyzing the historical infection data of the infectious disease, it is found that there are people who immediately develop symptoms and people who are asymptomatic in the infected people. Therefore, the population under the infectious disease environment can be classified into 6 status types, i.e., susceptible population S, infected population E, symptomatic population I after infection, asymptomatic population a after infection, dead population D, and convalescent population R.

302: a translation relationship between each of the at least one state type is determined.

In the present embodiment, after analyzing the historical infection data of the infectious disease, the population belongs to the susceptible population S after being found to be in the environment of the infectious disease. After the infection, the susceptible population S is converted into an infected population E, and then the infected population E is converted into a corresponding post-infection symptomatic population I or post-infection asymptomatic population A according to the condition of the presence or absence of symptoms after the infection. And finally, according to the condition that whether the recovery or the death is cured or not, converting into the corresponding death population D or recovery population R. Based on this, the conversion relationships between the 6 state types can be determined.

303: based on the infection parameters, the conversion rate between each status type is determined.

In the present embodiment, the infection parameter of the infectious disease can be obtained by analyzing historical infection data of the infectious disease. In particular, infection parameters may include: the infection probability of the susceptible population and the case by the virus, the probability of the case with symptoms, the virus transmission initial stage, the death probability of the case with symptoms, the incubation period, the rehabilitation period and the like are shown in table 1.

Table 1:

(symbol)	value of	Explanation of the invention
			Dcub	5-10 (Tian)	Incubation period
Dinf	7-14 (Tian)	In the convalescent period
			Dtrf	1 (sky)	1 day after infection with the virus, the virus began to spread
PEI	0.8	Probability of case with symptoms
			PID	0.02	Probability of death in symptomatic cases
Ptrf	1	Probability of virus infection of susceptible population and case

Thus, it can be seen that the probability of a susceptible population S being transformed into an infected population E upon exposure to an infectious disease is 1; the probability of immediate symptom after infection is 0.8, namely the probability of transforming the infected people E into the symptomatic people I after infection is 0.8, and the probability of transforming the corresponding infected people E into the asymptomatic people A after infection is 0.2; the death probability of the symptomatic case is 0.02, that is, the probability of transforming the symptomatic population I into the death population D after infection is 0.02, and the probability of transforming the symptomatic population I into the rehabilitation population R after infection is 0.98.

304: and establishing an infection model of the infectious disease according to the conversion relation between each state type in the at least one state type and the conversion rate between each state type.

In the present embodiment, following the above example, the 6 state types may be connected by an arrow according to the conversion relationship between each of the 6 state types, and the conversion rate between the two state types is marked on the arrow between the two corresponding state types. Thus, an infection model as shown in FIG. 4 can be obtained.

Specifically, in the infection model shown in fig. 4, the numbers represent the percentage of the previous state type that translates to the corresponding state type, and not the absolute transition probability. For example, when performing state transitions from state type E, 80% of the people in E are converted to I and 20% are converted to A. The infected people E have a latent period D of 5-10 days from infection to symptom_cub(i.e., from E to I or A) with a post-symptomatic convalescence of 7-14 days D_inf(i.e., from I to D or R) and is in the form of a latent periodThe probability of state transition is the same, and the 1/D is obeyed_cubPolynomial distribution, i.e., the person in E has a probability of 1/10 being converted to I or A each day during the latency period. Likewise, the transition probability from I to D, R is 1/14. In 6 states, the person in three states, E, I, a, was infectious, and none of the remaining states were infectious.

202: and establishing a simulated urban group, and establishing a population flow model among urban nodes in the simulated urban group according to the parameters of the simulated urban group.

In the present embodiment, in order to restore the real environment as much as possible, a city node network including 100 city nodes is first generated when constructing the simulated city group. The coordinates of each city node are randomly generated from the [100,2000] uniform distribution, and then the number of people per city node is randomly generated from the [25000,100000] uniform distribution.

Next, in the present embodiment, in order to simulate the propagation of an epidemic disease between different city nodes, the strength of contact between the different city nodes is defined. The strength of contact is used to identify the population flow of two cities on the day, and illustratively, the strength of contact is proportional to the product of the population of the two cities and inversely proportional to the square root of the distance between the two cities. Specifically, the strength of the connection between any two city nodes can be represented by formula (i):

wherein, P_xRepresenting the number of people in the x city node; p_yRepresenting the number of people in the y-th city node; d_xyRepresents the distance between the x-th city node and the y-th city node, x is not equal to y, and x and y are positive integers other than 0.

Therefore, as shown in fig. 5, a population flow model between the city nodes in the simulated city group can be obtained, where a circle represents a city node, a larger circle represents that the population of the city node is more, a connecting line between every two city nodes represents the connection strength between the two city nodes, and only the connecting line with the highest connection strength is shown in fig. 5.

203: and establishing an infected person number increment model of the first city node according to the infection model and the population mobility model.

In this embodiment, the first city node is any one of the simulated city groups, and the infection number incremental model is used to identify the rule of the number of infected persons that the first city node increases every day. Illustratively, the present application provides a method for building an infection population incremental model of a first city node based on an infection model and a population mobility model, as shown in fig. 6, the method comprising:

601: and establishing a state transformation model of the infectious diseases according to the infectious disease model.

In this embodiment, the state transition model is used to identify transition laws between different states in infectious diseases. Illustratively, the state transition model may be established based on an infection model of an infectious disease and infection characteristics of the infectious disease. Specifically, infectious diseases are divided into two steps to be calculated respectively when being transmitted, wherein one step is state classification, namely, the population in E or I is divided into two classes through conversion rate; the second step is state transition, namely after determining which state type to transition to, the corresponding crowd passes through 1/D_cubOr 1/D_infThe transition probabilities of (a) are transformed accordingly. After the two steps of calculation are finished, the number of people in each state type is calculated based on an infection model of the infectious disease, and meanwhile, the number of people in each state type is updated by taking days as a unit.

In the present embodiment, the conversion rate may be used as a classification threshold value when classifying the states. For example, a random number less than 1 may be generated each time, and when the random number is less than the classification threshold, the random number is classified as the corresponding state type. Specifically, the method comprises the following steps: when the state of the crowd in E is classified, the conversion rate of transferring E to I is 0.8, so that the crowd with the state of less than 0.8 in the generated random numbers is classified as I and the crowd with the state of more than 0.8 in the generated random numbers is classified as A.

Furthermore, in an alternative embodiment, if no latency is required from E to A and E to I requires a latency of 5-10 days, i.e., it is determined that the population in E that will be converted to A is directly converted toA. When the number of state transition people is calculated, the state temporary storage operation can be adopted, namely, a length D is maintained_cub-1 temporary memory in which the next D is stored_cubThe number of people who will experience state transitions each day for 1 day. The daily state transitions can be determined by a polynomial distribution, obtained by polynomial experiments at D_cubAnd (3) adding the list to the temporary storage according to the position, wherein the first element is the number of people who perform state transition on the day, and the rest elements are used as new temporary storage to be stored, and repeating iteration until the termination condition is reached.

602: and establishing an internal incremental model according to the state conversion model.

In this embodiment, the internal incremental model is used to identify the regularity of the daily growth of the number of infectious agents in the first city node due to the spread of infectious diseases within the first city node. For example, the infection rate of an infectious disease of a first city node may be determined based on the status of the first city node. The number of infected and latent people, infected and asymptomatic people, and infected and symptomatic people in the first city node are then determined according to the infection model and the state transition model. Finally, an internal incremental model is established based on the infection rate of the infectious disease of the first city node, the number of people who have been infected and are in the latent stage, the number of people who have been infected and are asymptomatic, and the number of people who have been infected and are symptomatic.

Specifically, the internal incremental model can be represented by the formula (II):

wherein E is₁Representing a number of persons in the first city node that have been infected and are in a latent state; a. the₁Representing the number of persons in the first city node who have been infected and are asymptomatic; i is₁Indicating the number of infected and symptomatic persons in the first city node,to representAn infection rate of an infectious disease of the first city node.

603: and establishing an external incremental model according to the population flow model.

In this embodiment, the external incremental model is used to identify the regularity of the daily growth of the infected population at the first city node due to the population migrating between the first and second city nodes. For example, the strength of the contact between the first city node and the second city node can be determined according to a population mobility model, wherein the second city node is any one of the simulated city groups different from the first city node. And determining the infection rate of the infectious diseases of the first city node according to the state of the first city node. And then, determining the state value of the second city node according to a preset state rule and the state of the second city node, and determining the population number of the second city node according to a population flow model. And finally, establishing an external incremental model according to the contact strength between the first city node and the second city node, the infectious disease infection rate of the first city node, the state value of the second city node and the population number of the second city node.

Specifically, the internal incremental model can be represented by formula (c):

wherein, K_1,2Representing the strength of the connection between the first city node and the second city node; u. of_2,dA state value indicating a second city node, specifically, u if the city node 2 is in an open state_2,dIs 1, if the city node 2 is in a closed state, u_2,dIs 0; rho₂Representing the population of the second city node.

604: and determining the state type of the first city node, and determining the incremental model of the infected person number according to the state type, the internal incremental model and the external incremental model.

In this embodiment, first, two state types of the city node are defined:

(1) open type (open) city nodes, asymptomatic and other people can freely enter and exit the city nodes, but symptomatic individuals cannot enter/leave the city nodes, and if cases are forcibly isolated immediately after symptoms appear, 10% of people in symptomatic people still flow inside the city nodes;

(2) closed type (lockdown) city node: the access of the city nodes is closed, and only part of people move in the city nodes.

In this respect, in the present embodiment, when the city node is open, the infectious disease spreads among the city nodes as the population flows, and in this case, the newly increased number of infected persons in the city node can be divided into the newly increased number of infected persons due to the flow of the population at the outside city node and the newly increased number of infected persons due to the spread of the infectious disease inside the city node. That is, when a city node is open, the delta model of the number of infected people includes an internal delta model and an external delta model.

When the urban nodes are closed, the external urban nodes do not influence the infectious disease transmission condition of the closed urban nodes any more, and the newly increased number of infected people only generates a small amount of population flow of the internal nodes. That is, when the city node is closed, the number of increments generated by the external increment model is constantly 0, and the newly increased number of infected persons only takes into account the number of increments generated by the internal increment model, that is, the incremental model of infected persons is the internal increment model.

In addition, the infection rate of the city node is also related to the current state of the city node. In particular, following the example of the first city node described above, when the first city node is open, the infection rate corresponding to the city nodeInfection rate in open stateInfection rate corresponding to urban node when the urban node is closedInfection rate in a closed state

In alternative embodiments, other parameters of the infectious disease transmission between city nodes may also be obtained, and a corresponding transmission model may be established, for example, some relevant parameters that may be used are given in table 2:

table 2:

204: and performing data simulation according to a preset city-closing strategy and the infected person number increment model to generate simulation data.

In the present embodiment, the generated simulation data is used to train a predetermined reinforcement learning model, and the data for training the reinforcement learning model needs to have a state (state), an action (action), and a reward (reward). The action is whether to take a binary value of closing city, for example: action 1 is taken as a closed city decision, and action 0 is not closed city.

Based on this, in order to ensure that the generated simulation data can be directly provided to the reinforcement learning model for use, in the present embodiment, a method for generating simulation data by performing data simulation according to a preset city-closing strategy and an infected person number increment model is proposed, as shown in fig. 7, the method includes:

701: determining input features of the reinforcement learning model, and determining the output data type of the data simulation according to the input features.

In the present embodiment, the states of the reinforcement learning model are 8-dimensional input features, which are: the number of single city nodes, the number of single city node infected persons, the total number of all city nodes infected persons, the number of single city node restored persons, the number of single city node dead persons, the number of single city node susceptible persons, the number of single city node symptom infected persons and the number of all city node total symptom infected persons are increased. Such feature definitions allow model decision mechanisms to be interpretable while accounting for human casualties and economic losses. Specifically, in data simulation, 8-dimensional input feature results are calculated before the beginning of each week and stored in one tuple.

702: a reward function of the reinforcement learning model is determined.

In this embodiment, the cost of closing a city may be defined as 10/day, the cost of infection and symptoms as 10/infected individual, the cost of death of a person as 25/dead individual, and the reward function as the opposite of the cost. Meanwhile, the infection simulation of the infectious diseases is performed in units of weeks, namely, before each week, whether to adopt the closed city is selected according to the closed city strategy, and the disease transmission condition of each subsequent city node is calculated based on the decision result. That is, the reinforcement learning model will affect the spread of infectious diseases in the week once action is taken.

Meanwhile, since the algorithm of the reinforcement learning model needs to consider the long-term return, in the present embodiment, the reward function is composed of two parts: one is an intermediate status award and one is a final status award. Since not only the control of the number of patients but also the economic loss due to city closing need to be considered in the decision making process. In the present embodiment, penalty coefficients for the number of infected persons, the number of dead persons, and the number of days in an urban area are set. The specific calculation formula is as follows:

the intermediate state reward may be represented by the formula (iv):

wherein Δ represents the variation of one week in the city node; c_lock(1-u_n,d) A penalty term representing a closed city; rho_nRepresenting the population of the city node; c_dead、C_infAnd C_lockIs radix Ginseng; x_n，deadRepresenting the number of people in the city node who died due to infection; x_n,infRepresenting the population size of the infection in the city node.

The final status reward may be represented by the formula (v):

R_n,term＝a-b(C_deadX_n，dead+C_infX_n,inf+C_lockd_n,lock).........⑤

wherein, a, b and C_dead、C_infAnd C_lockFor hyperginseng, in this scenario, a is 2, b is 0.01, C_dead＝25、 C_inf＝10、C_lock＝10；d_n,lockIndicating the number of closing days for the city node.

Based on this, the final establishment function of each state can be expressed by a formula (I):

R_{n,step_final}＝R_n,step+γ^N-vR_n,term.........⑥

wherein γ represents a discount coefficient, and in this scenario γ is 0.9; n represents the total number of iterations; v represents the current iteration week number.

703: and performing data simulation according to the reward function of the reinforcement learning model, the parameters of the simulated urban group, the city closing strategy and the infected person increment model to obtain initial data.

In this embodiment, each simulation may continue for up to 52 weeks, or the simulation may end until E, I, a are all 0 in the model of infectious disease infection. At the end of the weekly simulation, calculating stepforward (R) of each node according to a formula_n,step) Meanwhile, whether the node propagation reaches the simulation stop condition is judged, if the simulation is finished, the final rewarded (R) of the node is calculated according to a formula (v)_n,term). To obtain R_n,termThen, the information is propagated forwards by a discount coefficient of gamma being 0.9, and the target reward of each step is R_n,stepAnd discount R_n,termAnd (4) summing. Finally, a large amount of simulation data with a format of 5-tuple (state, action, reward, next _ state, done) is generated.

704: and screening the initial data according to the type of the output data to obtain simulation data.

205: and inputting the simulation data into a preset reinforcement learning model for training to obtain a decision model.

In the embodiment, a DQN model is adopted, the state of each node and the associated nodes thereof and simulation data are used as the input of the model, the probability distribution of the closing strategy is output as yes/no, and meanwhile, the reward value of the whole sequence is calculated according to the selected decision, so that the infection and death number of population are reduced, and the influence on economy is minimum.

Specifically, the RL algorithm selects DQN to calculate an optimal closing strategy, and the value of each action strategy is calculated through a fully-connected neural network. The fully connected neural network model structure is as follows: the input is 8-dimensional characteristic, 2 hidden layers are respectively provided with 50 and 30 neurons, each layer of network is activated by relu function, the output layer is 2 neurons, and the return value of each action is calculated. In the epidemic propagation simulation process, the model calculates a higher reward strategy according to the node state information before starting every week, so as to determine whether to close the city. Once DQN has computed all the nodes' next actions, the epidemic propagation simulation advances forward for 7 days.

Training DQN network parameters: 220000 pieces of simulation data (5 balanced man-made intervention city-closing strategy data) are trained, the Batch _ size is 20000, the learning rate LR is 0.00001, the Loss function is mse, the optimization function is Adam, the epoch is 100, and finally the Reward is calculated by using an environment Reward instead of a Bellman formula.

206: and acquiring infection data of the city to be decided and cities in the first range.

In the present embodiment, the first range is determined by the location of the city to be decided, and specifically, the first range may be a circular area determined under a preset radius with the coordinates of the city to be decided as the center.

207: and inputting the infection data into the decision model to obtain a decision result.

In summary, in the public policy decision method provided by the present invention, the spreading characteristics of infectious diseases and the inter-city population flow characteristics are analyzed under the condition of sudden infectious diseases, so as to construct an infectious disease model and a population flow model between city nodes, and then an infectious population incremental model reflecting the rule of the infectious population that increases every day for one city node is constructed according to the infectious disease model and the population flow model. And then, performing data simulation according to a preset city-closing strategy and an infected person number increment model to obtain a large amount of simulation data to train the reinforcement learning model to obtain a decision model. And finally, inputting the actual data into a decision model to obtain a corresponding decision suggestion. Therefore, through data simulation, under the condition that no historical data can be used for reference, a large amount of simulation data is obtained, powerful data support is provided for decision making, and black swan events can be effectively responded. Meanwhile, the model for executing the data simulation is a model generated by analyzing the propagation characteristics of the infectious diseases and the population flow characteristics among cities, and the simulation reality is high, so that the rationality of decision can be further improved. Finally, the output result of the reinforcement learning model can be analyzed and traced, the deep reinforcement learning is different from the state of a pure deep neural network black box, the given decision is optimized through a reward function set by a user, and therefore the result is reasonable and interpretable.

Referring to fig. 8, fig. 8 is a block diagram illustrating functional modules of a common policy decision device according to an embodiment of the present disclosure. As shown in fig. 8, the common policy decision device 800 includes:

the model establishing module 801 is used for establishing an infection model of the infectious disease according to infection parameters of the infectious disease, wherein the infection model is used for identifying the transmission and development rules of the infectious disease in the crowd;

the environment establishing module 802 is configured to establish a simulated urban group, and establish a population flow model between urban nodes in the simulated urban group according to parameters of the simulated urban group, where the simulated urban group includes at least two urban nodes;

the model establishing module 801 is further configured to establish an infected person number increment model of the first city node according to the infection model and the population flow model, wherein the first city node is any one of the simulated city groups, and the infected person number increment model is used for identifying a rule of an infected person number which increases every day for the first city node;

the simulation module 803 is used for performing data simulation according to a preset city-closing strategy and an infected person number increment model to generate simulation data;

the training module 804 is used for inputting the simulation data into a preset reinforcement learning model for training to obtain a decision model;

an acquisition module 805, configured to acquire infection data of a city to be decided and cities within a first range, where the first range is determined by a location of the city to be decided;

the decision module 806 is configured to input the infection data into the decision model to obtain a decision result.

In an embodiment of the present invention, in terms of establishing an infection population incremental model of a first city node according to an infection model and a population mobility model, the model establishing module 801 is specifically configured to:

establishing a state transformation model of the infectious disease according to the infectious disease model, wherein the state transformation model is used for identifying a transformation rule among different states in the infectious disease;

establishing an internal incremental model according to the state conversion model, wherein the internal incremental model is used for identifying the rule of the number of infectious people which are grown every day in the first city node due to the spreading of the infectious diseases in the first city node;

establishing an external incremental model according to the population flow model, wherein the external incremental model is used for identifying the rule of the number of infected persons of the first city node growing every day caused by the migration of the population among the first city nodes;

determining the state type of a first city node, and determining an infected person number incremental model according to the state type, an internal incremental model and an external incremental model, wherein the state type comprises the following steps: the method comprises the steps that the number of infected people is in an open state and a closed state, and when the state type of a first city node is in the open state, the incremental model of the number of infected people comprises an internal incremental model and an external incremental model; and when the state type of the first city node is a closed state, the infected person number increment model is an internal increment model.

In an embodiment of the present invention, in terms of building an internal incremental model according to a state transition model, the model building module 801 is specifically configured to:

determining the infection rate of the infectious diseases of the first city node according to the state of the first city node;

determining a number of infected and latent, a number of infected and asymptomatic, and a number of infected and symptomatic persons in the first city node, according to the infection model and the state transition model;

an internal increment model is established based on the infection rate of the infectious disease of the first city node, the number of people who have been infected and are in the latent stage, the number of people who have been infected and are asymptomatic, and the number of people who have been infected and are symptomatic.

In an embodiment of the present invention, in terms of building an external incremental model according to a population flow model, the model building module 801 is specifically configured to:

determining the contact strength between a first city node and a second city node according to a population flow model, wherein the second city node is any one of simulated city groups different from the first city node;

determining the infection rate of the infectious diseases of the first city node according to the state of the first city node;

determining a state value of a second city node according to a preset state rule and the state of the second city node;

determining the population number of the second city node according to the population mobility model;

and establishing an external incremental model according to the contact strength between the first city node and the second city node, the infectious disease infection rate of the first city node, the state value of the second city node and the population number of the second city node.

In an embodiment of the present invention, in determining the strength of connection between a first city node and a second city node according to a population mobility model, the model establishing module 801 is specifically configured to:

determining the population number of the first city node according to the population flow model;

determining the distance between the first city node and the second city node according to the population mobility model;

and taking the quotient of the product of the population number of the first city node and the population number of the second city node and the square root of the distance between the first city node and the second city node as the strength of the connection between the first city node and the second city node.

In an embodiment of the present invention, in establishing an infection model of an infectious disease according to an infection parameter of the infectious disease, the model establishing module 801 is specifically configured to:

analyzing the infectious disease to determine at least one type of condition of the person in the context of the infectious disease;

determining a conversion relationship between each of the at least one state type;

determining a conversion rate between each status type according to the infection parameters;

and establishing an infection model of the infectious disease according to the conversion relation between each state type in the at least one state type and the conversion rate between each state type.

In the embodiment of the present invention, in terms of performing data simulation and generating simulation data according to a preset city-closing strategy and an incremental infected person model, the simulation module 803 is specifically configured to:

determining input characteristics of a reinforcement learning model, and determining the output data type of data simulation according to the input characteristics;

determining a reward function of the reinforcement learning model;

performing data simulation according to a reward function of the reinforcement learning model, parameters of a simulated urban group, an urban closing strategy and an infected person increment model to obtain initial data;

and screening the initial data according to the type of the output data to obtain simulation data.

Referring to fig. 9, fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. As shown in fig. 9, the electronic device 900 includes a transceiver 901, a processor 902, and a memory 903. Connected to each other by a bus 904. The memory 903 is used to store computer programs and data, and may transfer the data stored in the memory 903 to the processor 902.

The processor 902 is configured to read the computer program in the memory 903 to perform the following operations:

establishing an infection model of the infectious disease according to infection parameters of the infectious disease, wherein the infection model is used for identifying the transmission and development rules of the infectious disease in the crowd;

establishing a simulated urban group, and establishing a population flow model among urban nodes in the simulated urban group according to parameters of the simulated urban group, wherein the simulated urban group comprises at least two urban nodes;

establishing an infected person number increment model of a first city node according to the infection model and the population flow model, wherein the first city node is any one of simulated city groups, and the infected person number increment model is used for identifying the rule of the infected person number which is increased every day by the first city node;

performing data simulation according to a preset city-closing strategy and an infected person incremental model to generate simulation data;

inputting simulation data into a preset reinforcement learning model for training to obtain a decision model;

acquiring infection data of a city to be decided and cities in a first range, wherein the first range is determined by the position of the city to be decided;

and inputting the infection data into the decision model to obtain a decision result.

In an embodiment of the present invention, in establishing an incremental model of the number of infected persons of the first city node according to the infection model and the population flow model, the processor 902 is specifically configured to perform the following operations:

establishing a state transformation model of the infectious disease according to the infectious disease model, wherein the state transformation model is used for identifying a transformation rule among different states in the infectious disease;

establishing an internal incremental model according to the state conversion model, wherein the internal incremental model is used for identifying the rule of the number of infectious people which are grown every day in the first city node due to the spreading of the infectious diseases in the first city node;

establishing an external incremental model according to the population flow model, wherein the external incremental model is used for identifying the rule of the number of infected persons of the first city node growing every day caused by the migration of the population among the first city nodes;

determining the state type of a first city node, and determining an infected person number incremental model according to the state type, an internal incremental model and an external incremental model, wherein the state type comprises the following steps: the method comprises the steps that the number of infected people is in an open state and a closed state, and when the state type of a first city node is in the open state, the incremental model of the number of infected people comprises an internal incremental model and an external incremental model; and when the state type of the first city node is a closed state, the infected person number increment model is an internal increment model.

In an embodiment of the present invention, in establishing the internal incremental model according to the state transition model, the processor 902 is specifically configured to perform the following operations:

determining the infection rate of the infectious diseases of the first city node according to the state of the first city node;

determining a number of infected and latent, a number of infected and asymptomatic, and a number of infected and symptomatic persons in the first city node, according to the infection model and the state transition model;

an internal increment model is established based on the infection rate of the infectious disease of the first city node, the number of people who have been infected and are in the latent stage, the number of people who have been infected and are asymptomatic, and the number of people who have been infected and are symptomatic.

In an embodiment of the present invention, in establishing the external incremental model according to the population flow model, the processor 902 is specifically configured to perform the following operations:

determining the contact strength between a first city node and a second city node according to a population flow model, wherein the second city node is any one of simulated city groups different from the first city node;

determining the infection rate of the infectious diseases of the first city node according to the state of the first city node;

determining a state value of a second city node according to a preset state rule and the state of the second city node;

determining the population number of the second city node according to the population mobility model;

and establishing an external incremental model according to the contact strength between the first city node and the second city node, the infectious disease infection rate of the first city node, the state value of the second city node and the population number of the second city node.

In an embodiment of the present invention, in determining the strength of contact between a first city node and a second city node according to a population mobility model, the processor 902 is specifically configured to:

determining the population number of the first city node according to the population flow model;

determining the distance between the first city node and the second city node according to the population mobility model;

and taking the quotient of the product of the population number of the first city node and the population number of the second city node and the square root of the distance between the first city node and the second city node as the strength of the connection between the first city node and the second city node.

In an embodiment of the present invention, in establishing an infection model of an infectious disease according to an infection parameter of the infectious disease, the processor 902 is specifically configured to:

analyzing the infectious disease to determine at least one type of condition of the person in the context of the infectious disease;

determining a conversion relationship between each of the at least one state type;

determining a conversion rate between each status type according to the infection parameters;

and establishing an infection model of the infectious disease according to the conversion relation between each state type in the at least one state type and the conversion rate between each state type.

In an embodiment of the present invention, in terms of performing data simulation according to a preset city-closing policy and an incremental infectious population model, and generating simulation data, the processor 902 is specifically configured to perform the following operations:

determining input characteristics of a reinforcement learning model, and determining the output data type of data simulation according to the input characteristics;

determining a reward function of the reinforcement learning model;

performing data simulation according to a reward function of the reinforcement learning model, parameters of a simulated urban group, an urban closing strategy and an infected person increment model to obtain initial data;

and screening the initial data according to the type of the output data to obtain simulation data.

It should be understood that the public policy decision device in the present application may include a smart Phone (e.g., an Android Phone, an iOS Phone, a Windows Phone, etc.), a tablet computer, a palm computer, a notebook computer, a Mobile Internet device MID (MID), a robot, a wearable device, etc. The above-mentioned common policy decision device is merely an example, not an exhaustive list, and includes but is not limited to the above-mentioned common policy decision device. In practical applications, the common policy decision device may further include: intelligent vehicle-mounted terminal, computer equipment and the like.

Through the above description of the embodiments, those skilled in the art will clearly understand that the present invention can be implemented by combining software and a hardware platform. With this understanding in mind, all or part of the technical solutions of the present invention that contribute to the background can be embodied in the form of a software product, which can be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods according to the embodiments or some parts of the embodiments.

Accordingly, the present application also provides a computer readable storage medium, which stores a computer program, the computer program being executed by a processor to implement part or all of the steps of any one of the public policy decision methods as set forth in the above method embodiments. For example, the storage medium may include a hard disk, a floppy disk, an optical disk, a magnetic tape, a magnetic disk, a flash memory, and the like.

Embodiments of the present application also provide a computer program product comprising a non-transitory computer readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any one of the common policy decision methods as set out in the above method embodiments.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are all alternative embodiments and that the acts and modules referred to are not necessarily required by the application.

In the above embodiments, the description of each embodiment has its own emphasis, and for parts not described in detail in a certain embodiment, reference may be made to the description of other embodiments.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is merely a logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of some interfaces, devices or units, and may be an electric or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The integrated unit may be implemented in the form of hardware, or may be implemented in the form of a software program module.

The integrated units, if implemented in the form of software program modules and sold or used as stand-alone products, may be stored in a computer readable memory. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a memory, and including several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned memory comprises: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable memory, and the memory may include: flash Memory disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.

The foregoing detailed description of the embodiments of the present application has been presented to illustrate the principles and implementations of the present application, and the above description of the embodiments is only provided to help understand the methods and their core ideas of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.