阅读说明：本技术 一种5g网络切片资源分配方法及系统 (5G network slice resource allocation method and system ) 是由 杨国民 于 2021-08-26 设计创作，主要内容包括：本发明公开了一种5G网络切片资源分配方法,包括：步骤1：获取用户提出的切片请求,根据CoS等级将获取的切片请求分队排列,得到8个SFC队列；步骤2：每次从8个SFC队列的队首取出切片请求,根据uRLLC、eMBB和mMTC的业务分类,为取出的切片请求配置通信路径；步骤3：采用预先学习的资源分配方法为取出的切片请求分配计算资源、存储资源、带宽资源；步骤4：将完成资源分配的切片请求从SFC队列删除,将新获取的切片请求按CoS等级置于对应SFC队列的队尾,返回步骤2。本发明从多方面降低时延,根据业务对时延要求的不同分配不同数量的资源,在资源充分利用的前提下满足各种业务的时延特性要求。(The invention discloses a 5G network slice resource allocation method, which comprises the following steps: step 1: acquiring slicing requests provided by users, and arranging the acquired slicing requests in a queue according to the CoS grade to obtain 8 SFC queues; step 2: taking out a slice request from the head of 8 SFC queues each time, and configuring a communication path for the taken out slice request according to the business classification of uRLLC, eMBB and mMTC; and step 3: allocating computing resources, storage resources and bandwidth resources for the taken slice request by adopting a pre-learned resource allocation method; and 4, step 4: and deleting the slice request for completing the resource allocation from the SFC queue, placing the newly acquired slice request at the tail of the corresponding SFC queue according to the CoS grade, and returning to the step 2. The invention reduces time delay from multiple aspects, allocates resources with different quantities according to different time delay requirements of the services, and meets the time delay characteristic requirements of various services on the premise of fully utilizing the resources.)

1. A5G network slice resource allocation method is characterized by comprising the following steps:

step 1: acquiring slicing requests provided by users, and arranging the acquired slicing requests in a queue according to the CoS grade to obtain 8 SFC queues;

step 2: taking out a slice request from the head of 8 SFC queues each time, and configuring a communication path for the taken out slice request according to the business classification of uRLLC, eMBB and mMTC;

and step 3: allocating computing resources, storage resources and bandwidth resources for the taken-out slice request by adopting a pre-learned resource allocation method to obtain a slice resource allocation strategy meeting the time delay requirements of each service, and taking the slice resource allocation strategy as a resource allocation method of the slice request;

and 4, step 4: and deleting the slice request for completing the resource allocation from the SFC queue, placing the newly acquired slice request at the tail of the corresponding SFC queue according to the CoS grade, and returning to the step 2.

2. The 5G network slice resource allocation method of claim 1, wherein when the SFC queue is empty, the queue resource requirement is 0, and any one of the computing resource, the storage resource and the bandwidth resource allocated to the queue is 0.

3. The 5G network slice resource allocation method of claim 1, wherein the CoS classes comprise 0-7 classes, where CoS-7 denotes that a slice has the highest requirement on end-to-end delay and CoS-0 denotes that a slice has the lowest requirement on end-to-end delay.

4. The 5G network slice resource allocation method of claim 1, wherein the end-to-end delay comprises propagation delay, transmission delay, queuing delay, and processing delay; the propagation delay is optimized when a communication path is configured, and the transmission delay, the queuing delay and the processing delay are optimized in the resource allocation process.

5. The 5G network slice resource allocation method of claim 4, wherein the configuration communication path satisfies: and the service with higher end-to-end delay requirement configures a shorter communication path, and the shortening of the communication path is realized by UPF (unified power flow), MEC (media independent component) function sinking and DU (unit data) and CU (unit operation Unit) combination methods.

6. The 5G network slice resource allocation method of claim 5, wherein the UPF, MEC function convergence and DU, CU merging method comprises:

configuring a data plane of a 5G system, the data plane comprising: an access data center DC1, an edge data center DC2, and a core data center DC 3;

the DU, CU, MEC and UPF of the uRLLC service are arranged in the access data center DC1, and the DU and the CU are arranged on the same physical device;

the DU of the eMBB service is arranged in an access data center DC1, and the CU, MEC and UPF functions are arranged in an edge data center DC 2;

the DU of mMTC service is arranged in an access data center DC1, the CU function is arranged in an edge data center DC2, and the UPF and MEC functions are arranged in a core data center DC 3.

7. The 5G network slice resource allocation method according to claim 4, wherein the preset resource allocation method comprises:

multiplying the sum of the transmission delay, the queuing delay and the processing delay of each service by 1+ CoS and summing;

with the minimum sum as an optimization target, allocating computing resources, storage resources and bandwidth resources for each slice request by using a reinforcement learning method; the computing resources and the bandwidth resources participate in the calculation of the optimization target, and the storage resources are used as constraint conditions during optimization.

8. The 5G network slice resource allocation method of claim 7, wherein the optimization objective is represented by the following formula:

in the formula (1), T represents the result of weighted summation; CoS_iRepresenting the CoS grade of the ith slice, and satisfying i belongs to 1-8; c_nRepresenting the computing resource distributed by the nth physical node of a certain time slot for the ith slice; b is_nIndicating the egress bandwidth allocated to the ith slice by the nth physical node of a certain time slot; x represents a calculation resource required for processing unit bit data; l represents the average length of the packet; the arrival process of the packets conforms to Poisson distribution with the arrival rate of lambda packets/s; e_iIndicating whether the queue is empty, and when the queue is empty, E_iNot equal to 0, otherwise E_i＝1。

9. The 5G network slice resource allocation method of claim 7, wherein the single step reward of the reinforcement learning method is a negative correlation function of an optimization objective.

10. A 5G network slice resource allocation system, comprising:

a slicing and grading module: the system comprises a queue management module, a queue management module and a queue management module, wherein the queue management module is used for acquiring slicing requests provided by users, and arranging the acquired slicing requests in a queue according to CoS grades to obtain 8 SFC queues;

a queue scheduling module: the method comprises the steps that a slice request is taken out from the head of 8 SFC queues each time, and a communication path is configured for the taken-out slice request according to the business classification of uRLLC, eMBB and mMTC;

a resource allocation module: the resource allocation method is used for allocating computing resources, storage resources and bandwidth resources to the taken slice request by adopting a pre-learned resource allocation method to obtain a slice resource allocation strategy meeting the time delay requirements of each service, and the slice resource allocation strategy is used as the resource allocation method of the slice request;

a queue update module: and deleting the slice request for completing resource allocation from the SFC queue, and placing the newly acquired slice request at the tail of the corresponding SFC queue according to the CoS level.

Technical Field

The invention relates to a 5G network slice resource allocation method and a system, and belongs to the technical field of communication networks.

Background

Currently, wireless communication has entered the 5G era. 5G services are classified into 3 types: the method is characterized in that mobile broadband (eMBB), ultra-high reliability low-delay communication (uRLLC) and massive machine type communication (mMTC) services are enhanced, and according to the requirements of different services on network characteristics, a physical network is cut into a plurality of virtual end-to-end networks by a 5G communication network through a slicing technology, so that customized services are provided for different services. The virtual end-to-end network comprises sub slices of an access network, a bearer network, a core network and the like. Compared with the 4G service, the 5G service has the characteristics of ultrahigh speed, ultralow time delay, large capacity and the like.

In order to solve the problem of the harsh requirement of the time delay, the industry makes a great deal of research on the aspects of reducing the propagation time delay, the transmission time delay, the processing time delay, the queuing time delay and the like, for example: introducing technologies such as CU/DU merging and MEC at an access network part, introducing an OTN technology at a bearer network part, introducing SDN and NFV technologies at a core network part, introducing a reinforcement learning technology in the aspect of resource allocation and the like. However, these technologies are basically to reduce the end-to-end delay of the slice network service unilaterally, and at present, there is no technology for comprehensively solving the delay problem from multiple aspects, and some of these technologies are not reasonable, for example, the reinforcement learning technology in the aspect of resource allocation initially takes the delay as a constraint condition rather than an optimization target, and later, although the delay is taken as an optimization target, the optimization target is the average delay rather than the single service delay, and this method cannot guarantee the delay characteristic requirement of the urrllc service with extremely high delay requirement.

Disclosure of Invention

The invention aims to overcome the defects in the prior art and provides a 5G network slice resource allocation method and a system, which can reduce time delay from multiple aspects and allocate different quantities of resources according to different time delay requirements of services, thereby meeting the time delay characteristic requirements of various services on the premise of fully utilizing the resources.

In order to achieve the purpose, the invention is realized by adopting the following technical scheme:

in a first aspect, the present invention provides a 5G network slice resource allocation method, including:

step 1: acquiring slicing requests provided by users, and arranging the acquired slicing requests in a queue according to the CoS grade to obtain 8 SFC queues;

step 2: taking out a slice request from the head of 8 SFC queues each time, and configuring a communication path for the taken out slice request according to the business classification of uRLLC, eMBB and mMTC;

and step 3: allocating computing resources, storage resources and bandwidth resources for the taken-out slice request by adopting a pre-learned resource allocation method to obtain a slice resource allocation strategy meeting the time delay requirements of each service, and taking the slice resource allocation strategy as a resource allocation method of the slice request;

and 4, step 4: and deleting the slice request for completing the resource allocation from the SFC queue, placing the newly acquired slice request at the tail of the corresponding SFC queue according to the CoS grade, and returning to the step 2.

With reference to the first aspect, further, when the SFC queue is empty, if the resource requirement of the queue is 0, any resource of the computing resource, the storage resource, and the bandwidth resource allocated to the queue is 0.

With reference to the first aspect, further, the CoS rank includes 0 to 7 ranks, where CoS ═ 7 denotes that the slice has the highest requirement on the end-to-end delay, and CoS ═ 0 denotes that the slice has the lowest requirement on the end-to-end delay.

With reference to the first aspect, further, the end-to-end delay includes propagation delay, transmission delay, queuing delay, and processing delay; the propagation delay is optimized when a communication path is configured, and the transmission delay, the queuing delay and the processing delay are optimized in the resource allocation process.

With reference to the first aspect, further, the configuring the communication path satisfies: and the service with higher end-to-end delay requirement configures a shorter communication path, and the shortening of the communication path is realized by UPF (unified power flow), MEC (media independent component) function sinking and DU (unit data) and CU (unit operation Unit) combination methods.

With reference to the first aspect, further, the 5G system includes a data plane, where the data plane is composed of computers and a network providing communication between the computers; according to different functions, computers belong to an access data center DC1, an edge data center DC2 and a core data center DC 3.

With reference to the first aspect, further, the method for UPF, MEC function convergence and DU, CU combination includes:

configuring a data plane of the 5G system;

the DU, CU, MEC and UPF of the uRLLC service are arranged in the access data center DC1, and the DU and the CU are arranged on the same physical device;

the DU of the eMBB service is arranged in an access data center DC1, and the CU, MEC and UPF functions are arranged in an edge data center DC 2;

the DU of mMTC service is arranged in an access data center DC1, the CU function is arranged in an edge data center DC2, and the UPF and MEC functions are arranged in a core data center DC 3.

With reference to the first aspect, further, the preset resource allocation method includes:

multiplying the sum of the transmission delay, the queuing delay and the processing delay of each service by 1+ CoS and summing;

with the minimum sum as an optimization target, allocating computing resources, storage resources and bandwidth resources for each slice request by using a reinforcement learning method; the computing resources and the bandwidth resources participate in the calculation of the optimization target, and the storage resources are used as constraint conditions during optimization.

With reference to the first aspect, preferably, the slicing request includes a CoS class of the request.

With reference to the first aspect, preferably, the computing resource refers to virtual computing capability allocated by a computer when a VNF is mapped to a specific physical device, and the VNF includes a DU, a CU, a MEC, and a UPF.

In connection with the first aspect, preferably, the storage resource refers to a virtual memory allocated by a computer.

With reference to the first aspect, preferably, the bandwidth resource refers to a virtual bandwidth obtained by a packet at an egress; when the virtual bandwidth is inside the access data center DC1, inside the edge data center DC2, or inside the core data center DC3, the virtual bandwidth is a portion of the bandwidth between computers; when the virtual bandwidth exits at the access data center DC1, the edge data center DC2, the virtual bandwidth belongs to a portion of the bearer network bandwidth between the access data center DC1 and the edge data center DC2, and between the edge data center DC2 and the core data center DC 3.

With reference to the first aspect, preferably, the sum of the virtual resources allocated to each slice request is not greater than the maximum value of the actual physical resources of the computer in which each slice is located.

With reference to the first aspect, preferably, after VNFs related to SFC queues are placed in corresponding physical devices for different situations of urrllc, eMBB, and mtc services, propagation delay is a fixed value, propagation delay does not need to be optimized in reinforcement learning, and whole-process propagation delay is not included in reinforcement learning.

With reference to the first aspect, preferably, the sum of the transmission delay, the queuing delay, and the processing delay of each service in reinforcement learning is represented by the following formula:

in the formula (1), T_iThe sum of transmission delay, queuing delay and processing delay generated by each slice entering resource allocation is represented, and the condition that i belongs to 1-8 is met; c_nThe unit of the computing resource distributed by the nth physical node of a certain time slot for the ith slice is CPU cycles/s; x represents the computing resource needed by processing unit bit data, and the unit is CPU cycles/bit; l represents the average length of the packet in bit; the arrival process of the packets conforms to Poisson distribution with the arrival rate of lambda packets/s; b is_nAnd the unit of the egress bandwidth which is allocated to the ith slice by the nth physical node of a certain time slot is bit/s.

With reference to the first aspect, further, the sum of the transmission delay, the queuing delay, and the processing delay of each service is multiplied by 1+ CoS and summed, which is represented by the following formula:

in the formula (2), T represents the result of weighted summation; CoS_iRepresenting the CoS grade of the ith slice, and satisfying i belongs to 1-8; c_nRepresenting the computing resource distributed by the nth physical node of a certain time slot for the ith slice; b is_nIndicating the egress bandwidth allocated to the ith slice by the nth physical node of a certain time slot; x represents a calculation resource required for processing unit bit data; l represents the average length of the packet; the arrival process of the packets conforms to Poisson distribution with the arrival rate of lambda packets/s; e_iIndicating whether the queue is empty, and when the queue is empty, E_iNot equal to 0, otherwise E_i＝1。

With reference to the first aspect, further, the single-step reward of the reinforcement learning method is a negative correlation function of the optimization objective.

In combination with the first aspect, preferably, the reinforcement learning method needs to satisfy the requirement that the high bandwidth bearer networks between the AAU and the access data center DC1, between the access data center DC1 and the edge data center DC2, between the edge data center DC2 and the core data center DC3, such as OTN, are directly reached by one hop, and the high bandwidth internal networks between the computers inside the access data center DC1, inside the edge data center DC2, and inside the core data center DC3 are also based on EPON.

In a second aspect, the present invention provides a 5G network slice resource allocation system, including:

a slicing and grading module: the system comprises a queue management module, a queue management module and a queue management module, wherein the queue management module is used for acquiring slicing requests provided by users, and arranging the acquired slicing requests in a queue according to CoS grades to obtain 8 SFC queues;

a queue scheduling module: the method comprises the steps that a slice request is taken out from the head of 8 SFC queues each time, and a communication path is configured for the taken-out slice request according to the business classification of uRLLC, eMBB and mMTC;

a resource allocation module: the resource allocation method is used for allocating computing resources, storage resources and bandwidth resources to the taken slice request by adopting a pre-learned resource allocation method to obtain a slice resource allocation strategy meeting the time delay requirements of each service, and the slice resource allocation strategy is used as the resource allocation method of the slice request;

a queue update module: and deleting the slice request for completing resource allocation from the SFC queue, and placing the newly acquired slice request at the tail of the corresponding SFC queue according to the CoS level.

Compared with the prior art, the 5G network slice resource allocation method provided by the embodiment of the invention has the beneficial effects that:

the invention obtains the slicing request provided by the user, and arranges the obtained slicing request in a queue according to the CoS grade to obtain 8 SFC queues; the requirements of each slice request on time delay can be effectively distinguished;

the invention takes out the slice request from the head of 8 SFC queues each time, and configures a communication path for the taken out slice request according to the business classification of uRLLC, eMBB and mMTC; according to the invention, communication paths are configured according to service classification, and optimization is carried out from the perspective of propagation delay so as to reduce delay to the greatest extent;

the invention adopts a pre-learned resource allocation method to allocate computing resources, storage resources and bandwidth resources for the taken-out slice request; the resource allocation method adopted by the invention is to perform reinforcement learning from the perspective of transmission delay, processing delay and queuing delay, and can meet the end-to-end delay requirements of various services while fully utilizing resources;

the invention distributes resources to the services with high real-time requirement as much as possible, and can ensure the time delay characteristic requirement of the uRLLC service with extremely high end-to-end time delay requirement.

Drawings

Fig. 1 is a flowchart of a 5G network slice resource allocation method provided in embodiment 1 of the present invention;

fig. 2 is a schematic diagram of a 5G system architecture in a 5G network slice resource allocation method according to embodiment 1 of the present invention;

fig. 3 is a flowchart of SFC queue scheduling in a 5G network slice resource allocation method according to embodiment 1 of the present invention;

fig. 4 is a flowchart of an algorithm of a preset resource allocation method in the 5G network slice resource allocation method according to embodiment 2 of the present invention.

Detailed Description

The invention is further described below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and the protection scope of the present invention is not limited thereby.

Example 1:

as shown in fig. 1, a 5G network slice resource allocation method includes:

step 1: acquiring slicing requests provided by users, and arranging the acquired slicing requests in a queue according to the CoS grade to obtain 8 SFC queues;

step 2: taking out a slice request from the head of 8 SFC queues each time, and configuring a communication path for the taken out slice request according to the business classification of uRLLC, eMBB and mMTC;

and step 3: allocating computing resources, storage resources and bandwidth resources for the taken-out slice request by adopting a pre-learned resource allocation method to obtain a slice resource allocation strategy meeting the time delay requirements of each service, and taking the slice resource allocation strategy as a resource allocation method of the slice request;

and 4, step 4: and deleting the slice request for completing the resource allocation from the SFC queue, placing the newly acquired slice request at the tail of the corresponding SFC queue according to the CoS grade, and returning to the step 2.

As shown in fig. 2, a 5G network slice resource allocation method is implemented based on a 5G system, where the 5G system includes a data plane, and the data plane is formed by computers and a network providing communication between the computers; according to different functions, computers belong to an access data center DC1, an edge data center DC2 and a core data center DC 3.

The method comprises the following specific steps:

step 1: and acquiring a slicing request made by a user.

Step 2: and arranging the acquired slice requests according to the CoS grade to obtain 8 SFC slice grading queues.

As shown in table 1, the CoS grades in the industry include 0-7 grades, where CoS ═ 7 denotes that the slice has the highest requirement on the end-to-end delay, and CoS ═ 0 denotes that the slice has the lowest requirement on the end-to-end delay.

TABLE 1 CoS rating Scale Chart

And step 3: and taking out the slice requests from the head of 8 SFC queues each time, and configuring a communication path for the taken out slice requests according to the service classification of uRLLC, eMBB and mMTC.

End-to-end delay includes propagation delay, transmission delay, queuing delay, and processing delay. The propagation delay is optimized when a communication path is configured, and the transmission delay, the queuing delay and the processing delay are optimized in the resource allocation process.

Note that the SFC request includes the CoS class of the request.

As shown in fig. 2 and fig. 3, the data packet at the head of the hierarchical queue of 8 SFC slices is read at the same time in each timeslot, and a shorter communication path is arranged for the service with higher end-to-end delay requirement according to the service classification method of urrllc, eMBB, and mtc. The shortening of the communication path is realized by UPF, MEC function sinking and DU, CU merging method.

The UPF and MEC function sinking and DU and CU merging method comprises the following steps:

setting a data plane of a 5G system;

the DU, CU, MEC and UPF of the uRLLC service are arranged in the access data center DC1, and the DU and the CU are arranged on the same physical device;

the DU of the eMBB service is arranged in an access data center DC1, and the CU, MEC and UPF functions are arranged in an edge data center DC 2;

the DU of mMTC service is arranged in an access data center DC1, the CU function is arranged in an edge data center DC2, and the UPF and MEC functions are arranged in a core data center DC 3.

And 4, step 4: and allocating computing resources, storage resources and bandwidth resources for the taken slice request by adopting a pre-learned resource allocation method to obtain a slice resource allocation strategy meeting the time delay requirement of each service, and taking the slice resource allocation strategy as a resource allocation method of the slice request.

The invention distributes resources to services with high real-time requirement as much as possible, and meets the end-to-end time delay requirement of various services while fully utilizing the resources.

The computing resources refer to virtual computing capacity allocated by the computer when the VNF is mapped to a specific physical device, and the VNF comprises DU, CU, MEC and UPF. Storage resources refer to virtual memory allocated by a computer. The bandwidth resource refers to virtual bandwidth obtained by grouping at an outlet; when the virtual bandwidth is inside the access data center DC1, inside the edge data center DC2, or inside the core data center DC3, the virtual bandwidth is a portion of the bandwidth between computers; when the virtual bandwidth exits at the access data center DC1, the edge data center DC2, the virtual bandwidth belongs to a portion of the bearer network bandwidth between the access data center DC1 and the edge data center DC2, and between the edge data center DC2 and the core data center DC 3.

It should be noted that the sum of the virtual resources allocated to each slice request is not greater than the maximum value of the actual physical resources of the computer in which each slice is located.

As shown in fig. 3, the preset resource allocation method includes:

multiplying the sum of the transmission delay, the queuing delay and the processing delay of each service by 1+ CoS and summing;

and allocating computing resources, storage resources and bandwidth resources for each slice request by using a reinforcement learning method with the minimum sum as an optimization target.

When the SFC queue is empty, if the resource requirement of the queue is 0, any one of the computing resource, the storage resource, and the bandwidth resource allocated to the queue is 0.

After VNFs related to SFC queues are placed to corresponding physical equipment according to different conditions of uRLLC, eMBB and mMTC services, propagation delay is a fixed value, the propagation delay is not required to be optimized in reinforcement learning, and the whole-process propagation delay is not included in the reinforcement learning.

The state of the reinforcement learning method is the request condition of all SFCs in 8 SFC queues in each time slot; the action is to allocate computing resources, storage resources and bandwidth resources to each queue head SFC at a corresponding physical node; the single step reward being a negative correlation function of the optimization objective, i.e.

The reinforcement learning method simultaneously optimizes from the perspective of transmission delay, processing delay and queuing delay so as to reduce the delay to the greatest extent. The sum of the transmission delay, queuing delay and processing delay of each service is represented by the following formula:

in the formula (1), T_iThe sum of transmission delay, queuing delay and processing delay generated by each slice request entering resource allocation is represented, and the sum satisfies that i belongs to 1-8; c_nThe unit of the computing resource distributed by the nth physical node of a certain time slot for the ith slice is CPU cycles/s; b is_nThe unit of the outlet bandwidth which is distributed by the nth physical node of a certain time slot for the ith slice is bit/s; x represents the computing resource needed by processing unit bit data, and the unit is CPU cycles/bit; l represents the average length of the packet in bit; the arrival process of the packet conforms to the poisson distribution with the arrival rate of lambda packets/s.

Specifically, the optimization objective is represented by the following formula:

in the formula (2), T represents the result of weighted summation; e_iIndicating whether the queue is empty, and when the queue is empty, E_iNot equal to 0, otherwise E_i＝1。

It should be noted that the reinforcement learning method needs to be satisfied on the basis of high-bandwidth bearer networks such as OTN through one hop between the AAU and the access data center DC1, the access data center DC1 and the edge data center DC2, and the edge data center DC2 and the core data center DC3, and also on the basis of high-bandwidth internal networks such as EPON between computers inside the access data center DC1, inside the edge data center DC2, and inside the core data center DC 3.

And 5: and deleting the slice request for completing the resource allocation from the SFC queue, placing the newly acquired slice request at the tail of the corresponding SFC queue according to the CoS grade, and returning to the step 2.

Example 2:

the present embodiment describes an intensive learning process for implementing resource allocation by taking a SAC algorithm for automatically adjusting a temperature coefficient α as an example.

To speed up the algorithm convergence process, the SAC algorithm introduces a strategic entropy (usingRepresenting the concept calculated by the inverse of the action probability), the training process of which is shown in fig. 4, includes:

step 1: a neural network is initialized.

Step 1.1: neural network parameters are initialized.

The neural network parameters include: 1 strategy network pi, and the parameter is phi; 2 action value networks Q with parameters theta₁、θ₂. The purpose of setting 2Q networks is to select the smaller of the Q values, avoiding an overestimation of the Q values. In order to reduce the 2 target networks corresponding to the 2 action value networks Q introduced by data correlation, the parameters are respectivelyIn order to reduce data dependency, a playback memory is provided

The hyper-parameters of this embodiment are set as: strategy network pi learning rate lambda pi and action value network Q learning rate lambda_QTemperature coefficient alpha learning rate lambda_αAll 0.001, discount rate gamma 0.99, playback memorySize 10⁶Byte, hidden layer of neural network is 2 layers, target smooth coefficientMeanwhile, the objective function is set as:

wherein θ represents θ₁、θ₂，To represent

Step 1.2: will theta₁Is copied toWill theta₂Is copied to

Step 1.3: clearing playback memory

Step 2: the iteration number i is set to 0.

And step 3: the number of steps j that have been performed for each iteration is set to 0.

And 4, step 4: selecting an action according to the policy: a is_t～π_φ(a_t|S_t)。

And 5: and (3) updating the state: s_t+1～p(S_t+1|S_t,a_t)。

Step 6: storing data in a playback memory:

and 7: updating Q network parameters:

and 8: updating the weight of the pi network:

and step 9: updating the temperature coefficient:

step 10: updating the target Q network parameters:

step 11: the number of steps j +1 that the iteration has performed is set.

Step 12: judging whether the number of steps executed by iteration reaches the maximum number of steps: if the maximum step number is reached, setting the iteration number i as i + 1; and if the maximum step number is not reached, returning to the step 4.

Step 13: judging whether iteration reaches the maximum number of rounds: if the maximum number of rounds is reached, the parameters phi and theta of the neural network are output₁、θ₂(ii) a And if the maximum step number is not reached, returning to the step 3.

Example 3:

the embodiment provides a 5G network slice resource allocation system, including:

a slicing and grading module: the system comprises a queue management module, a queue management module and a queue management module, wherein the queue management module is used for taking a slicing request provided by a user, and queuing the obtained slicing request according to the CoS grade to obtain 8 SFC queues;

a queue scheduling module: the method comprises the steps that a slice request is taken out from the head of 8 SFC queues each time, and a communication path is configured for the taken-out slice request according to the business classification of uRLLC, eMBB and mMTC;

a resource allocation module: the resource allocation method is used for allocating computing resources, storage resources and bandwidth resources to the taken slice request by adopting a pre-learned resource allocation method to obtain a slice resource allocation strategy meeting the time delay requirements of each service, and the slice resource allocation strategy is used as the resource allocation method of the slice request;

a queue update module: and deleting the slice request for completing resource allocation from the SFC queue, and placing the newly acquired slice request at the tail of the corresponding SFC queue according to the CoS level.

The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.