阅读说明：本技术 一种Rados Gateway的细粒度QoS控制方法 (Fine-grained QoS control method of Rados Gateway ) 是由 曹磊 高传集 王腾飞 李超 于 2021-08-05 设计创作，主要内容包括：本发明公开一种Rados Gateway的细粒度QoS控制方法,涉及网络服务技术领域,包括：开放设置Qos、查询Qos以及删除Qos的API接口；通过API接口,管理员向Rados Gateway发送请求,设置针对用户级别和桶级别两种粒度的QoS,同时记录两种粒度的QoS状态并进行持久化；发送请求后,依次检测两种粒度的QoS,进行请求的处理；请求处理完毕后,归还该请求占用的令牌数。本发明通过用户和桶两种细粒度的QoS,可以更合理的控制资源分配,无需单独开辟线程每秒向令牌桶中添加令牌,通过记录最早令牌时间的方式,将添加令牌的过程放在每次消耗和释放令牌之前的检测里,用少量的存储空间换取了计算时间和减少了cpu消耗,提高了QoS的效率。(The invention discloses a fine-grained QoS control method of Rados Gateway, relating to the technical field of network service and comprising the following steps: opening an API interface for setting Qos, inquiring Qos and deleting Qos; through an API (application program interface) interface, an administrator sends a request to a Rados Gateway, sets QoS (quality of service) aiming at two granularities of a user grade and a bucket grade, and records and persists QoS states of the two granularities; after the request is sent, QoS with two granularities is detected in sequence, and the request is processed; and after the request is processed, returning the number of tokens occupied by the request. The method can more reasonably control resource allocation through two kinds of fine-grained QoS (quality of service) of the user and the bucket, does not need to separately develop threads to add tokens to the token bucket every second, puts the process of adding the tokens into detection before consuming and releasing the tokens each time in a mode of recording the earliest token time, replaces the calculation time with a small amount of storage space, reduces the CPU consumption and improves the QoS efficiency.)

1. A Rados Gateway fine-grained QoS control method is characterized by comprising the following steps:

opening an API interface for setting Qos, inquiring Qos and deleting Qos;

through an API (application program interface) interface, an administrator sends a request to a Rados Gateway, sets QoS (quality of service) aiming at two granularities of a user grade and a bucket grade, and records and persists QoS states of the two granularities;

after the request is sent, QoS with two granularities is detected in sequence, and the request is processed;

and after the request is processed, returning the number of tokens occupied by the request.

2. The fine-grained QoS control method for Rados Gateway of claim 1, wherein the three API interfaces are admin interfaces for the administrator user of ceph cluster to call;

all three API interfaces specify the QoS sub-resources in the operation cluster in the URl, and the specific parameter values involved in the operation are contained in the body expressed in XML form.

3. The fine-grained QoS control method for Rados Gateway of claim 2, wherein when the QoS is set through the API, the requested body needs to contain the granularity of QoS, the name of the resource instance corresponding to the granularity, the category of the network resource to be limited by QoS, and the maximum value of the corresponding network resource;

after the administrator sends a request to the Rados Gateway, the Rados Gateway parses the parameters in the body, and records the name of the resource instance of QoS, the category of the network resource, the maximum value of the network resource, the earliest token time and the current token number in the attrs attribute of the object corresponding to the metadata pool.

4. The method of claim 3, wherein the QoS granularity includes a user level and a bucket level;

(a) if the QoS granularity is at a user level, recording the name of a QoS resource instance, the category of network resources and the maximum value of the network resources in attrs attributes of a user metadata object under a user uid name space in a metadata pool, simultaneously recording the current time as the earliest token time, initializing the token number to the maximum value of the corresponding network resources, and persistently storing the two token related parameters in the attrs attributes of the uid object;

(b) if the granularity of the QoS is bucket level, recording the name of the resource instance of the QoS, the category of the network resource and the maximum value of the network resource in attrs attribute of the instance object under root name space in the metadata pool.

5. The fine-grained QoS control method for Rados Gateway as claimed in claim 4, wherein when inquiring QoS and deleting QoS, the requested body needs to specify QoS granularity and resource instance name;

when deleting the QoS, judging the granularity of the QoS which needs to be deleted, and further deleting the attrs attribute of the QoS of the object in the corresponding metadata pool.

6. The fine-grained QoS control method for Rados Gateway according to claim 4, wherein the QoS of the user level is detected, and the request is processed, and the specific process is as follows:

carrying out authority authentication on the user;

after passing the authentication, checking the uid object of the user;

if QoS is set in attrs, firstly, the difference value between the current time and the recorded earliest token time is judged, when the difference value is larger than 1 second, the current token number is updated to be the maximum value of the corresponding network resource, meanwhile, the earliest token time is updated to be the current time, and then the difference between the current token number and the number of tokens to be consumed by the current request is judged:

(i) when the difference is not negative, the processing of the request is continued,

(ii) and when the difference value is negative, the current request is put back into the work queue at the front end of the Rados Gateway again, and the next taking-out processing of the work thread is waited.

7. The method for fine-grained QoS control of Rados Gateway as claimed in claim 6, wherein the QoS of bucket level is detected, and the request processing is performed, the specific process is as follows:

checking whether QoS setting exists in attrs attribute of instance in a metadata pool corresponding to the bucket;

if the token number is greater than 1 second, updating the current token number to be the maximum value of the corresponding network resource, updating the earliest token time to be the current time, and then judging the difference between the current token number and the token number to be consumed by the current request:

(i) when the difference is not negative, the processing of the request is continued,

(ii) and when the difference is a negative number, the current request is placed back into the work queue at the front end of the Rados Gateway, and the flag bit and the state attribute corresponding to the request instance are required to be restored to the state before the request is processed before the current request is placed back into the work queue.

8. The method for fine-grained QoS control of Rados Gateway as claimed in claim 7, wherein the number of tokens occupied by the request is returned after the request is processed, the specific process is as follows:

reading the current token state, and simultaneously judging the difference value between the current time and the recorded earliest token time:

(i) when the difference is more than 1 second, updating the current token number to be the maximum value of the corresponding network resource, updating the earliest token time to be the current time, and directly returning the result to the client sending the request;

(ii) and when the difference value is not more than 1 second, adding the current token number to the token number occupied by the request, if the added value is larger than the maximum value of the corresponding network resource, taking the current token number as the maximum value of the corresponding network resource, recording the added token number in the current token state, and finally returning the result to the client sending the request.

Technical Field

The invention relates to the technical field of network service, in particular to a fine-grained QoS control method of Rados Gateway.

Background

QoS (quality of service) is a short term for quality of service, QoS of a network service includes multiple aspects, such as bandwidth, delay, concurrency number of transmission, and packet loss rate of data, and a common QoS scheme includes setting a firewall rule, a routing policy, and the like. The quality of service of some services is guaranteed in network services while the quality of service of other services is also compromised, since network resources are always limited. Therefore, the QoS of the network service pays more attention to the reasonable utilization of limited resources, the token bucket is a common method for realizing the QoS of the network service, but a common token bucket algorithm cannot meet the requirement for the Rados Gateway, firstly, the QoS granularity of a client layer is too large, resource competition also exists between different users and the bucket, and the QoS is also required to be carried out; secondly, the number of users and buckets is huge, it is not practical to establish a thread for each user and bucket to maintain its token, and even if supported by hardware, the efficiency of the Rados Gateway under the influence of a large number of distributed locks and distributed variable synchronization is difficult to guarantee.

Disclosure of Invention

Aiming at the requirements and the defects of the prior art development, the invention provides a fine-grained QoS control method of Rados Gateway.

The invention discloses a Rados Gateway fine-grained QoS control method, which adopts the following technical scheme for solving the technical problems:

a Rados Gateway fine-grained QoS control method comprises the following steps:

opening an API interface for setting Qos, inquiring Qos and deleting Qos;

through an API (application program interface) interface, an administrator sends a request to a Rados Gateway, sets QoS (quality of service) aiming at two granularities of a user grade and a bucket grade, and records and persists QoS states of the two granularities;

after the request is sent, QoS with two granularities is detected in sequence, and the request is processed;

and after the request is processed, returning the number of tokens occupied by the request.

Optionally, all three involved API interfaces are admin interfaces for the administrator user of the ceph cluster to call;

all three API interfaces specify the QoS sub-resources in the operation cluster in the URl, and the specific parameter values involved in the operation are contained in the body expressed in XML form.

Further optionally, when the Qos is set through the API interface, the requested body needs to include the Qos granularity, the name of the resource instance corresponding to the granularity, the category of the network resource to be restricted by the Qos, and the maximum value of the corresponding network resource;

after the administrator sends a request to the Rados Gateway, the Rados Gateway parses the parameters in the body, and records the name of the resource instance of QoS, the category of the network resource, the maximum value of the network resource, the earliest token time and the current token number in the attrs attribute of the object corresponding to the metadata pool.

Further optionally, the granularity of the QoS involved includes a user level and a bucket level;

(a) if the QoS granularity is at a user level, recording the name of a QoS resource instance, the category of network resources and the maximum value of the network resources in attrs attributes of a user metadata object under a user uid name space in a metadata pool, simultaneously recording the current time as the earliest token time, initializing the token number to the maximum value of the corresponding network resources, and persistently storing the two token related parameters in the attrs attributes of the uid object;

(b) if the granularity of the QoS is bucket level, recording the name of the resource instance of the QoS, the category of the network resource and the maximum value of the network resource in attrs attribute of the instance object under root name space in the metadata pool.

Further optionally, when querying Qos and deleting Qos, the requested body needs to specify Qos granularity and resource instance name;

when deleting the QoS, judging the granularity of the QoS which needs to be deleted, and further deleting the attrs attribute of the QoS of the object in the corresponding metadata pool.

Further optionally, the QoS of the user level is detected, and the request is processed, where the specific process is as follows:

carrying out authority authentication on the user;

after passing the authentication, checking the uid object of the user;

if QoS is set in attrs, firstly, the difference value between the current time and the recorded earliest token time is judged, when the difference value is larger than 1 second, the current token number is updated to be the maximum value of the corresponding network resource, meanwhile, the earliest token time is updated to be the current time, and then the difference between the current token number and the number of tokens to be consumed by the current request is judged:

(i) when the difference is not negative, the processing of the request is continued,

(ii) and when the difference value is negative, the current request is put back into the work queue at the front end of the Rados Gateway again, and the next taking-out processing of the work thread is waited.

Further optionally, the QoS of the bucket level is detected, and the request is processed, where the specific process is as follows:

checking whether QoS setting exists in attrs attribute of instance in a metadata pool corresponding to the bucket;

if the token number is greater than 1 second, updating the current token number to be the maximum value of the corresponding network resource, updating the earliest token time to be the current time, and then judging the difference between the current token number and the token number to be consumed by the current request:

(i) when the difference is not negative, the processing of the request is continued,

(ii) and when the difference is a negative number, the current request is placed back into the work queue at the front end of the Rados Gateway, and the flag bit and the state attribute corresponding to the request instance are required to be restored to the state before the request is processed before the current request is placed back into the work queue.

Further optionally, after the request is processed, the number of tokens occupied by the request is returned, and the specific process includes:

reading the current token state, and simultaneously judging the difference value between the current time and the recorded earliest token time:

(i) when the difference is more than 1 second, updating the current token number to be the maximum value of the corresponding network resource, updating the earliest token time to be the current time, and directly returning the result to the client sending the request;

(ii) and when the difference value is not more than 1 second, adding the current token number to the token number occupied by the request, if the added value is larger than the maximum value of the corresponding network resource, taking the current token number as the maximum value of the corresponding network resource, recording the added token number in the current token state, and finally returning the result to the client sending the request.

Compared with the prior art, the fine-grained QoS control method of Rados Gateway has the beneficial effects that:

(1) according to the invention, through two kinds of fine-grained QoS (quality of service) of the user and the bucket, the resource allocation can be more reasonably controlled, a process of adding the token into the token bucket every second is not required to be independently developed, the process of adding the token is placed in the detection before the token is consumed and released every time in a mode of recording the earliest token time, the calculation time is replaced by a small amount of storage space, the CPU consumption is reduced, and the QoS efficiency is improved;

(2) the token state is persisted in the metadata pool, the execution efficiency of the Rados Gateway is prevented from being influenced by the synchronization of the distributed lock and the distributed variable, and the problem that the token state of the system cannot be recovered after the power failure of the machine is solved;

(3) the invention strictly separates and decouples IO concurrency number and bandwidth control, and limits different network resources can not be influenced mutually.

Drawings

FIG. 1 is a flow chart of the method of the present invention;

FIG. 2 is a flow chart illustrating sequential detection of QoS of two granularities in the present invention.

Detailed Description

In order to make the technical scheme, the technical problems to be solved and the technical effects of the present invention more clearly apparent, the following technical scheme of the present invention is clearly and completely described with reference to the specific embodiments.

The first embodiment is as follows:

with reference to fig. 1, this embodiment provides a fine-grained QoS control method for a Rados Gateway, which includes the following steps:

and (I) opening API interfaces for setting Qos, inquiring Qos and deleting Qos.

In this embodiment, the three API interfaces are admin interfaces for the administrator user of the ceph cluster to call.

All three API interfaces specify the QoS sub-resources in the operation cluster in the URl, and the specific parameter values involved in the operation are contained in the body expressed in XML form.

When the Qos is set through the API interface, the requested body needs to include the Qos granularity, the name of the resource instance corresponding to the granularity, the category of the network resource to be restricted by the Qos, and the maximum value of the corresponding network resource.

And (II) through an API (application programming interface) interface, an administrator sends a request to the Rados Gateway, sets QoS (quality of service) aiming at two granularities of a user level and a bucket level, and records the QoS states of the two granularities and carries out persistence.

After the administrator sends a request to the Rados Gateway, the Rados Gateway parses the parameters in the body, and records the name of the resource instance of QoS, the category of the network resource, the maximum value of the network resource, the earliest token time and the current token number in the attrs attribute of the object corresponding to the metadata pool.

In this embodiment, the QoS granularity includes a user level and a bucket level;

(a) if the QoS granularity is at a user level, recording the name of a QoS resource instance, the category of network resources and the maximum value of the network resources in attrs attributes of a user metadata object under a user uid name space in a metadata pool, simultaneously recording the current time as the earliest token time, initializing the token number to the maximum value of the corresponding network resources, and persistently storing the two token related parameters in the attrs attributes of the uid object;

(b) if the granularity of the QoS is bucket level, recording the name of the resource instance of the QoS, the category of the network resource and the maximum value of the network resource in attrs attribute of the instance object under root name space in the metadata pool.

When inquiring QoS and deleting QoS, the requested body needs to specify the granularity of QoS and the name of resource instance.

When deleting the QoS, judging the granularity of the QoS which needs to be deleted, and further deleting the attrs attribute of the QoS of the object in the corresponding metadata pool.

And thirdly, with reference to fig. 2, after the request is sent, the QoS of two granularities is detected in sequence, and the request is processed.

(1) Detecting QoS of a user level, and processing a request, wherein the specific process is as follows:

carrying out authority authentication on the user;

after passing the authentication, checking the uid object of the user;

if QoS is set in attrs, firstly, the difference value between the current time and the recorded earliest token time is judged, when the difference value is larger than 1 second, the current token number is updated to be the maximum value of the corresponding network resource, meanwhile, the earliest token time is updated to be the current time, and then the difference between the current token number and the number of tokens to be consumed by the current request is judged:

(i) when the difference is not negative, the processing of the request is continued,

(ii) and when the difference value is negative, the current request is put back into the work queue at the front end of the Rados Gateway again, and the next taking-out processing of the work thread is waited.

(2) Detecting the QoS of the bucket level, and processing the request, wherein the specific process is as follows:

checking whether QoS setting exists in attrs attribute of instance in a metadata pool corresponding to the bucket;

if the token number is greater than 1 second, updating the current token number to be the maximum value of the corresponding network resource, updating the earliest token time to be the current time, and then judging the difference between the current token number and the token number to be consumed by the current request:

(i) when the difference is not negative, the processing of the request is continued,

(ii) and when the difference is a negative number, the current request is placed back into the work queue at the front end of the Rados Gateway, and the flag bit and the state attribute corresponding to the request instance are required to be restored to the state before the request is processed before the current request is placed back into the work queue.

And (IV) after the request is processed, returning the number of tokens occupied by the request, which comprises the following specific processes:

reading the current token state, and simultaneously judging the difference value between the current time and the recorded earliest token time:

(i) when the difference is more than 1 second, updating the current token number to be the maximum value of the corresponding network resource, updating the earliest token time to be the current time, and directly returning the result to the client sending the request;

(ii) and when the difference value is not more than 1 second, adding the current token number to the token number occupied by the request, if the added value is larger than the maximum value of the corresponding network resource, taking the current token number as the maximum value of the corresponding network resource, recording the added token number in the current token state, and finally returning the result to the client sending the request.

In summary, by adopting the fine-grained QoS control method of Rados Gateway of the invention, resource allocation can be more reasonably controlled through two kinds of fine-grained QoS of users and buckets, tokens are not required to be added into the token bucket every second by independently opening up threads, and the QoS efficiency is improved.

The principles and embodiments of the present invention have been described in detail using specific examples, which are provided only to aid in understanding the core technical content of the present invention. Based on the above embodiments of the present invention, those skilled in the art should make any improvements and modifications to the present invention without departing from the principle of the present invention, and therefore, the present invention should fall into the protection scope of the present invention.